Someone may have said this already, but I couldn't read all of the people thinking English is hard for the same reasons everyone is hard. Every language has words that sound the same but mean something different (or at least many do), many languages have weird words that don't follow the rules.
But, having studied a little Japanese and a lot of Spanish, I think one of the main difficulties is the way our words sound vs. the way they're written. In Japanese and Spanish, words are very easy to sound out, because vowels (or certain combinations of vowels) always make the same sound. Think of the Japanese word "domo," is pronounced the same way we would say "homo." But, if we were to see the word in an English book and were unfamiliar with it, some of us might get it right, but some of us might say it like "da-mo".
Some of our things just plain don't make sense, too, like gh being an f sound, (cough), "o" being a small "i" sound (women), and "ti" making a sound like "sh" (caution). An old demonstrations using these regularities spelling fish like "ghoti."