In America, from experience, I'd say CAG (Combat Applications Group, AKA Delta Force). CAG soldiers are generally selected from special ops units across all branches (including SEAL) and they function in a very different manner than most special ops units. SEAL team 6 in particular would probably be up there with them.
Outside of America, I'd probably say the Israelis (forget the name) or the Indians (again, forget the name). I know the Indians win the competitions enough, at least.
Also, just as an aside, the Republic of Korea has an all female special forces unit. I forget the actual name, but we called them Black Widows. They were pretty fucking awesome, we actually got to meet a couple of them on a jump while I was over there. Also, they are hot.
Edit: This is not a personal photograph, just one I found on the internet.