There've been a few times where I've turned difficulty down for a spell because one part was aggravating, then turned it back up after. Games I thoroughly enjoy and/or am good at I'll either play hard, or normal 1st and hard 2nd playthru. If a game offers new game+ in particular, I'll probably go on hard.
Mass Effect 2 is a little annoying in this regard. I play Vanguard (have tried infiltrator and would like to try adept sometime) but the difficulty settings have a direct consequence on gameplay. After Vet difficulty (ie. Hardcore, Insanity) the enemies all have additional shields and armour. Having any form of protection in ME2 completely nullifies biotic abilities (except warp AFAIK). So biotic charge, pull, singularity or whatever have little effect, rendering the vanguard/adept almost useless (at least until the silly teammate AI strips shields/armour off for you). Great game, but how BioWare missed that feature and/or thought it would be fine in the final game is truly beyond me. So yeah, just Vet on ME2 then.
Managed DS2 on Nightmare, whew, won't do that again! Currently (re)playing Resident Evil 5 on professional difficulty with maxed out, infinite ammo L. Hawk, M3 and Dragunov...so does that count as hard or easy? :-D