Basically dig until you can make a compass and try not to leave home without it.

And set up your main base really close to your spawn.
If you can't be bothered to place signs in caves, but still don't want to get lost - use cobblestone to wall off the tunnels that you've fully explored. That way you'll usually only have one way to go.
And read up on the recipes on the minecraft wiki. You'll learn loads of interesting stuff.
And for those of you who've lost entire worlds due to conversion or HD failure I have just one word: Backup. It's especially easy with the new format since copying a 30MB world takes just a few seconds rather than 10 minutes like it used to.