I think death is its own penalty. The goal of the game is to succeed, and the opposite of it is failure. The entire idea behind a game is that success is not easy, but it is achievable; it's just a matter of figuring out how to succeed. Our brains key in to this, rewarding us for the success and punishing us for failure. So long as we can distinguish between success and failure, we reward ourselves for our successes and punish ourselves for our failures. That's why we keep trying; one of these times, we're going to succeed and we will be rewarded.
Adding a "death penalty" just increases the amount of work that has to be done to try again. The longer this is, the less likely we are to try again, because we have an intrinsic expectation of successes-to-failures. If we go too long without a success, and don't get enough rewards, we grow frustrated and decide to look for something else that gives more rewards. Conversely, if we succeed too quickly and get too many rewards, we decide that the game offers no challenge and that the reward isn't "real", because we didn't have to work for it. The longer it takes to try again, the less often we will receive those rewards, and the more likely we are to give up trying.
It may seem that the opposite could be true; that too little time between tries would make the game too easy. This may be true, but only if the choices are reasonably exhaustible. For example, if your only "choice" is between two possible answers, then if you fail once, you automatically succeed the next time. This eventual success isn't rewarding because you didn't come up with the solution yourself, the answer was given to you because you failed, with the final outcome being that you failed entirely and will never get a chance to succeed. If this is what each decision in the game looks like, and the time between attempts is short, a player can easily enumerate all of the choices until they happen upon the correct answer and no challenge remains; like cracking a password, all you have to do is keep trying until it works.
However, a game with real decisions will not be simply enumerable. The player will need to solve the system, figuring out what is happening, such that he can choose the correct answer out of the multitude of possible answers. Solving the system is the success and the player knows it's a success because without solving the problem, they couldn't have reasonably expected to discover the answer. There's a clear line between success and failure.
If this is the case, the time it takes between attempts can be infinitesimally small and still the cost of failure will not be small enough to eliminate the challenge, because it's infeasible to bypass the challenge through sufficient attempts; the only way to succeed to is solve the problem.
For a great example of this, you need look no further than games like Super Meat Boy or Braid, where the time between attempts is nearly instant, yet the player does not feel that the game fails to pose a challenge. The time between attempts doesn't matter because repeated attempts will not eventually lead to success, only solving the problem (either solving the puzzle or developing the proper muscular responses) will lead to a success.
In my own opinion, long delays between attempts, especially due to explicit death penalties, only serve to interfere with game play. Once I have made my decision and failed, I immediately want to try another solution; I don't want to spend 10 minutes waiting for the opportunity. The challenge of the game was to solve the system and find the correct decision and the death penalty is an external thing which is preventing me from attempting to solve the system; it's getting in the way of my game play.
So why do online games, especially MMOs, insist on long death penalties? Part of the reason is that every problem they pose you with is a decision between "safe and slow" or "fast and risky". If you are hunting for a lost item, it's only a matter of time until you explore the limited space to find it; death penalties force you to decide between fighting an enemy to take the shortest path and save time (at the risk of dying and actually increasing the time) or taking the longer, safer route. Likewise when choosing which quest to complete: the harder ones may give you more experience, causing you to progress faster, but they carry a higher risk of dying, which -- with a sufficient death penalty -- will cause you to progress slower. Even the choice of whether or not to get somebody to help you is a tradeoff between the time it will take to find a helper and the inherent loss of experience for completing the goal, versus the amount of time wasted repeatedly failing if you don't have the help. The goal is always to progress faster, without knowing how long it will take to complete or how much time you risk losing due to deaths. The cost of death, therefore, is the great counterbalance -- too small, and you will always choose to risk dying; too long and you will always choose to get avoid the risk.
But clearly, the cost of a death doesn't always have to be used to maintain the difficulty of a decision. If time is not an issue, such as when solving a puzzle, the cost of the death doesn't affect the difficulty of the decision; the decision is not a time-risk trade-off. The same is also often true for FPSes, depending on how success is measured; where fast and precise reactions are the deciding factor of success, the time between attempts doesn't improve or impair your success rate. However, there can be a time-risk trade-off, such as choosing between risking a costly death to get a kill or avoiding the risk and waiting for a better opportunity, in a game where you must maximize the number of kills in a fixed amount of time.
