There have always been three reasons for preventing women for joining, and all have been related to combat effectiveness:
1) Physical. This reason is now dead in the water, but an understandable reason in the past. We now know that women can do the job, physically (and emotionally) as well as men. This is why pretty much all civilised armies permit women in a wide variety of support arms and non-front line combatant arms (such as combat pilots, on ships, etc.)
2) Small unit cohesion. This was the reason for segregation in the US army until the 50s, "don't ask, don't tell", etc. Without such cohesion, a unit cannot be an effective fighting force, and is a reasonable concern regarding military effectiveness - to use an example from the past, a platoon or section composed of white guys from Mississippi with a black NCO in the 1940s would not have been any use in a fight, and as far as the military is concerned, they quite rightly think that "using unfair discrimination to ensure that the army can do its job" is better than "we must be just, the fact that our troops are lynching one another for looking at them funny is an acceptable consequence". In fact, in the cases of black serviceman, segregation was probably the best solution at the time, ensuring small unit cohesion whilst ensuring that all Americans could serve their country. Similarly, don't ask don't tell was a surprisingly pragmatic approach allow gay men and women to serve without it affecting the combat efficiency of the unit - in both cases the only practical alternative was "no blacks" and "no homosexuals".
Regarding women in the military in the 21st century, this is of course trickier, but I think we're certainly getting to the stage where this is becoming less and less of an issue due to societal shift. There may be a couple of things that could cause problems, such as a CMT prioritising a female for casualty evacuation over male, or an infantryman forgetting that evacuation of casualties comes from the rear when faced with an injured servicewoman, but these are things that can be sorted. All in all, I'd say we're on the cusp of this not being a problem any more.
3) How the enemy treat women. This is where it gets tricky, and not wishing to go into too much detail...well, how well do you think some of the chaps we're fighting today, or are likely to face, are going to treat a captured woman? This, for me, is the main issue holding us back at the moment. Though, of course, it becomes less of an issue if our likely enemies have a somewhat more enlightened view of women.
In summary, I can see it happening in the future, but the reasons it hasn't happened yet are reasonably sound, and I wouldn't raise a particular hue and cry if it were changed.