I have to say yes!*
*It HAS to be done right. I JUST finished replaying OoT, and there were so many dark things about the game that could have been adjusted to be even darker and scarier. First of which, Gannondorf puking bile instead of coughing up blood. I played the original "Red Blood" edition in '96, so playing the GC remake (for convenience sake. I didn't feel like setting up my N64 again) I was quite disturbed at the small changes they made to make it more kid friendly.
One thing I really wanted to see more of was that freaky-as-all-hell undead sub-boss that had the hands coming out of the ground and would chase you and try to bite you. Seriously! that was some fucked up shit! Last time I saw a monster that scary was in Pan's labyrinth! Now going strictly horror genera wouldn't work for Zelda, but there are plenty of opportunities to throw stuff like that in there.
There were the Re-dead in burnt-out Hyrule town, and though it is implied that they are the dead walking, if they actually looked like the decaying corpses of some of the villagers that you talked to as a kid, I think it would be a lot more "M rated" of a theme.
Another thing that would be nice is seeing the carnage that the villains lay down on Hyrule. I could totally see Gannondorf floating magically above the castle, using his new-found Triforce of Power to rain horrible destruction down on the helpless people. Anyone remember the destruction of the world in Final Fantasy 3 (6)? It made you just wanna rip Kafka to shreads, and HE WAS LAUGHING about it! Zelda could use some new villains like this. Some one you hate. Some one that MUST be destroyed.
All in all, there is plenty of room for Zelda to go M rated without going over the top. It has to be dark, and it has to be real. It can't have stupid things like T&A and swearing just so that there will be more "cool." M ratings are much more broad of a classification that it should be. There are plenty of M games that are so tame they could almost make a PG movie. I know that Thief 1 and 2 are M rated, and the worst thing in the game is watching the silhouette of some one get his eye ripped out. It's implied violence more than on-screen blood and gore, and yet the games got a gaming equivalent of an "R" rating. Suggested viewing age of 17 year olds.