I know, we're awesome! *preens* Well, when I say "we", I mean the people who had to learn it in school; I got it gratis because of my Irish mother, with little to no effort on my part...
I don't know how hard Swedish is to learn (obviously) but both my American friend and my Irish brother-in-law learned to speak casual Swedish fluently (if with a thick accent) after about 9 months, through studying, work and, in my friend's case, playing The Sims in Swedish. Then again, I've also met English speakers who've lived here for 20-30 years and never bothered to learn Swedish because WHO NEEDS TO AMIRITE? /sarcasm
I've heard the hardest part of Swedish is the cadence; we have a very sing-songy pitch accent that can be hard to pick up, what with depending on how you stress words they can mean totally different things. (stegen = the ladder VS stegen = the steps (you take), or gifter = marrying VS gifter = poisons) My friend still has problems with this and she's lived here now for 7 years.
But if all you're doing is visiting, Hej, Hej då, and Tack is all you really need =) Swedes LOVE to speak English at you given the chance anyway!