Oh, wow, this is going WAY back. My memory is pretty fuzzy, since I was a little dunce back then. I was probably around five...
I THINK it was Sonic Adventure.
I vaguely remember trying to learn how to work a controller while my family gathered around the magic light box and watched me play. I vaguely remember having no idea how Pokemon worked as well (I kept trying to catch the trainer Pokemon). I actually got stuck on the first badge for like a year, until I was old enough to figure out how the mechanics worked, so that must have been one of the earliest games I played as well. Crash Bandicoot came along around that time too.
EDIT: Crash was released two years prior to the others, so I suppose it could be that one. I really can't remember.