I apologize if this has already been said, as I don't have the time to read all of the comments, but this is the first time I've disagreed with Jim, although it is only partial disagreement.
I agree that games have to utilize their interactive nature. They have to let you be a part of them. However, I think Dear Esther does this and does it well. The problem with it, in my estimation, is that it simply wont appeal to a lot of people. If you go in WANTING to be pulled into the world, TRYING to be pulled in, the story works. If you are the kind of person who goes in to see IF it will pull you in, it probably wont. There is little it does to pull you into the story, but once you are in it the story is fascinating. I don't think it is even that deep, but it is a story that is different for everyone.
Dear Esther utilizes a semi-random story, where the framework is the same, but the actual bits of narration change each time you play (selecting from a list of possible lines). The story is very, very vague, and the fact that the details are different for everyone means that everyone has their own interpretation. The game is essentially a puzzle game, but you don't interact with the environment, you interact with the story, trying to piece together disparate narrative threads to reach a conclusion as to what is going on. This makes the story so much more personal, because YOU figured out what's going on, the game didn't tell you.
Dear Esther really wouldn't work as a short-film or novel, as you suggested, because if it just told you the story it would be boring. The fact that it is a game lets the story be non-linear and it also slows the pace. The vague details don't make for a very meaty story, so just expressing it straight out would lose the pacing and immersion.
Now I realize that it isn't perfect, and I agree that it needs to let you interact with the environment a bit more. What I think would work perfectly for this style of 'interactive story' is Amnesia:The Dark Descent's click-drag physics. They let you interact with the environment in a fairly natural, immersive way. This is why I think it is awesome that Dear Esther devs TheChineseRoom are making a game under Amnesia devs Frictional Games. They will get to use the immersive Amnesia interaction system along with their talent for storytelling. I don't think Dear Esther is perfect but I love it for its uniqueness. It is the only game I've ever played that has a story that is simultaneously so linear and yet so personal. I do think that many games are guilty of just having you walk through a story so that they seem deep and arty though. Limbo almost fell into this trap, but the puzzles and atmosphere were enough to pull it through IMO. Every Day the Same Dream on the other hand, was fucking boring. I know that is was partially supposed to be to make its point, but just because you wanted your game to be boring and repetitious doesn't stop it from being, well, shit.
One game (well, mod really) actually made use of this lack of interactivity. The Stanley Parable is a Half-Life mod in which the entire point is the lack of control. I don't want to say too much more for fear of spoiling, but it is fantastic and you should definitely play it (it only takes about an hour or so to see all of the endings.) A lot of people say that these art games Jim speaks of ARE interactive because you walk around and take it at your own pace, its just that the interactivity isn't combat-based (hmm yes, go back to your 'shooty' games, plebian.) This argument is BS in my opinion because while walking around freely is technically interactivity, it is boring interactivity, and usually contributes nothing to the work in question. Dear Esther uses it to slow the pacing and to add to the atmosphere and immersion by letting you walk around and take in the scenery at your own pace.
In the end, I think you are right Jim, but I think Dear Esther is better than you give it credit for, even if it isn't the best.I don't mind the problems because it was an experiment. It has no interactivity in the environment because it was designed to see if a game could be held up by narrative ALONE. I think it succeeded, but I can easily see how someone who doesn't actually TRY to get pulled in like I did may have found it face-meltingly boring. It is similar to how some people (like me) find Amnesia terrifying while others think it is dumb. If you don't actively immerse yourself in it, the game's subtle dread-inducing tactics will have no effect on you and the actual scares will be made far less scary. Whatever, to each his own I guess, I just wanted to voice another point of view.
Oh and uh, sorry for the wall of text, I tend to do that a lot =/