I only really remember the Water Temple from Legend of Zelda Ocarina of Time.
It's not that it was hard to figure out (although that does contribute), but mainly because it was frustratingly slow and required a ton of pausing to switch boots so you can sink/walk underwater or rise/swim, as well as a couple of other repetitive puzzle pieces.
It took absolutely forever, and then you had to go up against a mini-boss that I considered one of the hardest bosses in the entire game, even over the actual bosses. I may have only been a kid back then, but Dark/Shadow Link (aka mirror version of Link) was damn tricky to beat. I don't even remember how I got past him. He copies your exact same moves, but if you keep spamming attacks he'll counter you. If you use items he just dodges it. If you stab at him, he jumps on your sword like some kind of ninja, then attacks you.