Since I got my first Sony Ericsson, I have played many java games. Most of them were bad (boring, impractical controls...), but some were excellent.
My favourites are Doom RPG, Ancient Empires II (Advance Wars-like medieval fantasy strategy, insane fun to play with friends), Gravity Defied Trial Racing (yeah, I hate some of those names... But the game is addicting), Galaxy on Fire (3D space shooter with mind-blowing graphics for a phone and fun gameplay), Playman World Soccer (mobile football finally done right), Playman Beach Volleyball, Alpha Wing 2 (classic 2D spaceship shooter, hard as hell), Prince of Persia trilogy (3rd one being the best), Splinter Cell quadralogy (Chaos Theory was excellent) - Ubisoft makes great mobile games, by the way - there were some others, but these I think are the best.
I rarely play games on my phone nowadays, except for the Absolute Minesweeper, because I love minesweeper and it's great time-waster when I'm waiting for something.