In terms of black-and-white, Casablanca is an absolutely must-see. I would also recommend Best Years of Our Lives. On a much harder to find note, there was an old film made in the late 30s or so called Gunga Din: a truly amazing adventure film that is little-known, but rather respected in film communities.
As for Citizen Kane... you can see how it influenced/created modern cinematography, but it's really not that great of a film. Watch it for its influence, but don't expect all that incredibly much.
You must see Equilibrium, if you haven't already. It is a brilliant sci-fi that is far too little-known.
Pan's Labyrinth, if you haven't seen it, absolutely must be seen. A truly unique and brilliant piece of artistic film.
Then, play Final Fantasy VII and watch Final Fantasy VII Advent Children Complete. Assuming you get Complete (with an extra 30 minutes), watch it with subtitles (they screwed up the English translation horribly), and understand the events in Final Fantasy VII, it is a truly great film. Plus, it's just awesome. Some of the most beautiful CG you will ever see. EVER.
Shawn of the Dead and Hot Fuzz are both by the same two guys, and two of the best British comedies you will ever see (thus, two of the best comedies you will ever see, because America completely fails at that genre right now).
Eternal Sunshine of the Spotless Mind is the best romance film you will ever see. Ever. Because it not only messes with the formula, it messes with your head. Bigtime. That movie will confuse the heck out of you until understanding comes crashing violently down on you at the end, and you will be blown away.
On a similar note, The Sixth Sense. Holy crap. Any of Shyamalan's first four films, really (The Sixth Sense, Unbreakable, Signs, The Village).
Anything by Miyazaki. Spirited Away, Princess Mononoke, Nausicaa of the Valley of the Wind, Howl's Moving Castle, Ponyo... it's all amazing.
An American Tail. If you've already seen this as a kid, watch it again. Yes, it's an animated movie about mice. But take that out of it for a second and look at it again. It's a movie about a family that moves from Russia to America, expecting its promises of glory and good life, but their child is lost at sea and washes up on the shores of New York, and he now has to try and survive in the underbelly of New York while his family searches for him and a gang of cats is controlling everything in the area. Yeah. It's intense, depressing, and extremely serious. Good film.
On that note, watch all the classic childhood movies again. The Brave Little Toaster, The Land Before Time, The Lion King, The Great Mouse Detective all that stuff: there are some truly amazing animated films that we only remember nostalgically until we see them again.
I am assuming you've seen things like The Princess Bride, the original Star Wars Trilogy, and The Matrix, so I won't even bother mentioning them.
District 9, if you haven't seen it, is a great film.
And... that's all for now.