I would recommend starting with Persona 3 FES. 1 and 2 are good (especially 2), but they haven't aged particularly well, and 3 is basically a soft reboot. Persona 4 is definitely the easiest to get into, and 5 is the most polished, but it's kind of hard to go back and play 3 after having the improvements made in both of them, so if you're going to play 3 at all, I would play it before 4 and 5. Just keep in mind the various versions of the games:
Persona 3 FES is an updated/expanded version that is basically superior in every way, and is the definitive version of the game. Persona 3 Portable added some stuff, but also removed some of the stuff that FES added, and also removed being able to walk around the world outside of dungeons (reducing it to maps and menus) and all the cutscenes were reduced to just dialogue portraits, so it's overall inferior to FES.
Persona 4 Golden added a huge amount of content and is without question the better version of 4, but not a lot of people have a Vita. If you have one, go for this, but if not, the original is still excellent.
For what it's worth, Persona 4 Golden is my favorite game of all time.