To me the only real difference between sci-fi and fantasy is setting. Fantasy is set in a bygone era (often set in a world looking back at a lost greater age that is now gone) while science fiction is set in some near of distant future. What is shared between them is, a physical and understandable world but with something that pushes the limits of physical reality. In fantasy this device is magic while in sci-fi it is science and technology.
Someone quoted Arthur C. Clark earlier and that quote kinda says it all (Any sufficiently advanced technology is indistinguishable from magic). I am a great fan of Dan Simmons Hyperion Cantos so I will use that as a perfect illustration of this. In the Hyperion Cantos you have timetravel, forceshields, faster then light travel, teleportation, immortality and a loads of other things that current scientific theory deem either impossible or highly unlikely. It even has things that seem more like magic then science but the only reason why it seems like magic is because it is never explained how it is done. Here then comes the coup de grace if you will. In this setting there is even something beyond the comprehension of any sentient being presented (including some rather impressive AI's). Strange creatures at the edge of the system they use to communicate presented almost as creatures of myth and magic and definitely as something to be feared. These mysterious creatures are explained in time but I won't spoil the surprise. What we see is a setting where seemingly impossible thing happen while being explained by science.
Now The Hyperion Cantos is what you might call soft sci-fi. The reason of course is that it holds rather precariously on to the science bit while focusing on the fiction. That is ok. It works well for it. Personally I can't of the top of my head come up with some hard science fiction novels but I suppose if we look at movies and games I could find some. Alien springs to mind as well as Halo, Gears of War and Dreampod 9s Jovian Chronicles (roleplaying game). what is the key to understand is that sci-fi is more of a spectrum then on clear genre.
Incidentally the way middle-ground sci-fi often glosses over the more fantastic bits of what they do is with the nicely coined "technobable". Just say a bunch of stuff that sounds scientific and hope it sticks. Star-Trek uses this a LOT.
Now on to fantasy. You can probably guess that my point here is the same. Fantasy also covers a broad spectrum. In this case it is often the gritty versus the fantastic. You have your high-magic setting with wizards and magic, elves and goblins, your basic Lord of the Rings, on the one side and you low-magic, rare (to non-existent) exotic races story on the other. I would say the legend of King Arthur would fit precariously in the second category. The category can get even broad (and more muddied) if we take a look at the D&D setting of Eberron in which magic IS science and technology. Where exotic races have their own nations and where even sentient constructs walk the world. This is still fantasy.
So we can see that fantasy is an umbrella that can cover a lot of things. Is it so strange to think that sci-fi can be just as broad?
Then of course there are the cross-overs. The places where science and magic meet. Science Fantasy. I have no reason to mention this group except to say that I love it. Star Wars (up till mediclorians which I will ignore), Warhammer 40k and a lot of the Final Fantasies out there fall beautifully into this entertaining genre. It truly shows how well the two sides work together and how futile trying to splitting them up is.
Anyway, those are my thoughts. I hope they are of use to someone.