To women shorter than me, I'm not fussed, my wife is 4ft 9(she's legally a Dwarf, now if only I could get her to take up blacksmithing, which is still a reasonably big thing in Finland....), in theory I wouldn't be bothered about my girlfriend being taller than me, though in practice I might feel a bit silly if she was taller than me by a large margin. I'm 5ft 7, so a little below average in the UK, but quite a bit more in Finland where I'm surrounded by vikings. Strangely though, ceilings, lamps and hanging things are all really low hanging in Finland compared to the UK, to the point where I consistantly hit my head on things, so how do the vikings not do it?
But in society at large, the general view is still that men should be taller than women, just because of the general manliness-feminine social construct. Even my wife said that she'd feel silly if I were the same hight as her or smaller, granted that's never been an issue for her, but yeah.