I've always felt similarly. It was one of the reasons I skipped Majora's Mask even when it came out, an action I later discovered to be a huge mistake, for it is practically the template for how to use timers to add rather than subtract from a game.
I think there's a fundamental difference between a timer mechanic linked to the core game mechanics versus a superfluous one. Is the timer punishing players for being slow, or is is designed to force strategic decisions, creating emergent gameplay? Is it part of the puzzle, or just an obstacle? The timer in Majora's Mask for instance is just a natural extension of the infinite day/night cycle seen in practically every game these days, only with more depth and three times as long. Every three day cycle (much the same as every day in any other game) your goals as a player are different. The timer in Doom or Duke Nukem 3D on the other hand exists solely for speedrunners as the goal of the level is the same every time you begin a given level.
As a more recent example of the same exploration-heavy genre to Zelda, Pandora's Tower's timer is the core source of the non-linearity in the game. All the dungeons are essentially long linear hallwayss, but with tonnes of unlockable shortcuts interlinking parts you've cleared. Without a timer, this whole game would've been boring as shit (OK, perhaps not the bosses), but combined with a day night cycle for (predictable) resource collection, the core game challenge itself becomes one of planning your route through the dungeon across a series of short campaigns (sorta like an MMO minus the grind). You feel well rewarded for a successful dungeon dive, and you are given opportunities to use rare consumables without feeling like they are going to waste: when the timer is in the red, that is a clear time when the game is screaming at you: use your hoarded shit now, you greedy bugger, or risk losing stuff!
Actually, thinking back to Majora's Mask again, three of the four dungeons in that game were basically linear hallways too, as were the final four dungeons in Twilight Princess and pretty much every other game since. So too in fact was the infamous Water Temple of Ocarina of Time; that one was ludicrously linear. The catch with the water temple is that recalibrating the system should you leave the dungeon half finished was insanely difficult, going a great deal towards explaining the contrasting difficulty expereinces players report. Those who did the dungeon in one sitting had an easy time, those who took two sittings spent hours recalibrating the water level. Hypothetically, were there a timer in that dungeon, all players would have had the non-linear, exceedingly challenging experience the Water Temple was famed for: and that's pretty much Pandora's Tower's timer mechanic in a nutshell. Using clocks both to standardise the experience between players, and as a source of emergent gameplay.
(Is there a second half to this article that is missing perchance? The site is telling me its one page long, but it lacks any sort of snarky summary, or the author tagline.)
