I've always defined immersion in a game as the point where you forget that you're playing. When you're actually afraid of death in-game instead of going back to your last quicksave, when you make decisions based on what you or your character would really do instead of what benefits you most at the time, when you're excited by your new find or you really want to defeat the villain or really care about the person you're saving. Realism is easy and can be fun; Mount and Blade is the best melee combat simulator I've found and one of my favorite "toys" but I can't get immersed in the flat world and nonexistant story, and there are hundreds of shooters that have beautiful graphics, perfectly bullet paths, and realistic damage effects, but I've never enjoyed playing one.
My top pick for immersion (disclaimer: I have not played Skyrim yet in order to not drop out of university) would probably be Morrowind. I might be influenced a bit by nostalgia here (I spent hundreds of hours in that game as a kid, when I had free time), and I'll be the first to admit that the graphics are crap, the AIs are idiotic, and it's glitchy as hell, but I've never forgotten myself in a game quite as well. The excitement of exploration... the quick and deadly combat... the massive variety of quests and people... the infinite possibilities for your hero that force you to play again and again... the sheer openness of the world...
I was very disappointed by Oblivion in comparison. Sure it was prettier, and smoother, and the combat and control systems were vastly improved, but it lost everything that made the previous game great. There was no point to exploration. Where Morrowind allowed anyone to die, no matter what that did to their precious plot, Oblivion arbitrarily marked half the characters as immortal until they'd served their purpose to some side story that doesn't matter to you. Invisible walls abounded and transportation spells (levitation, mark and recall) were removed to prevent the player from leaving the rails, and replaced with a fast travel system that kills any sense of scope, isolation, etc. Instead of hundreds of quests such that even I haven't found them all in my extensive travels, there were few enough that you could finish them in an afternoon or two. Instead of a complex world filled with competition, politics, and outright war between factions, forcing you to choose between your possible loyalties, you could literally make the same character into the head of every organization in Cyrodil without anyone questioning it. Instead of starting you off as a normal person, barely stronger than a mudcrab, and gradually building up to demigodhood and deicide, you're put up against daedra right from the start and win effortlessly. Overall it just felt so small, and I was always acting for the best game result because I couldn't get past the breaks in immersion.
Enough off-topic ranting. Especially since Oblivion was honestly a good game and decently immersive, just a huge dropoff from its predecessor. Other games that did the immersion well, in my opinion, include the original two Fallout games, New Vegas, Mass Effect (the first more than the second, but both were good), and The Witcher.
Vampire the Masquerade: Bloodlines came close, but spent too much time on the rails for me. Though I suppose it makes sense to be railroaded if you're a young vampire being forced along by the domination of your elders...
One that did surprisingly well was Half Life 2; I never had my own choices to make, but the atmosphere of it was such that I lost track in spite of myself and when I hit Ravenholm my first time I was paranoid enough to waste all my ammo making sure corpses would stay down, and while frantically bashing zombies with my crowbar (I was absolutely rubbish with the gravity gun) I discovered that I was actually laughing maniacally and shaking a little IRL.