First off, I'm a cadet at the USAF Academy and know what you're going through to get in here. Secondly I'm red-green deficient and hence, partially color blind too. The way the academy has worked so far is allowing people with color blindness to join, but are labeled as "Commission Qualified" rather than "Pilot Qualified." This may or may not prevent you from getting a pilot slot, you'll just have to talk to the doctors here at the military flight screening to find out because you may not have a problem as bad as you think.
On another note, the academy is a great place. The education is great, the scenery is amazing, and the comradery is one of a kind. However, I must warn you. If you do come, be prepared to become for a rough first year. As you may have heard, the Fourth Class (freshman) year is the most difficult physically, academically, and militarily with few privileges. After that year, though you are accepted as an upperclassmen and things are a lot easier...except the academics, but then again you do have a 4.4 GPA.
Good luck and maybe we'll see you in a few years!