Well, as I live in Pittsburgh where the three seasons are Summer, winter, and RAIN...where the word 'drought' is a laughable term...where many of us have insurance against flooding, landslide, and MINE SUBSIDANCE...I would say hot weather is worse.
You see, it takes far more energy to make hot to cold than vice versa. Living in foul weather territory, I can tell you that winter happiness is thick clothes, a decent coat, and a reasonble space heater, and that's for those with low funds. One furnace heating made payable by utility taxes is probably less of a to-do than the strain and consumption on electricity during times of heat. Suffice to say, I'm definitely use to ther cold, so that sort of makes the win.