The endermen are scary if you let them be. They're not really threatening in a dangerous way, just in more of a...psychological way. You CAN'T look at them. If you do, they tear your face off. And when they die in the morning, the block they're holding goes with them. That means they're taking the world apart one block at a time per night per enderman. You know what that makes them?
Now tell me they're not scary.