I've been trying to play through the series before Persona 5 releases. So far I've finished Persona 1 (PSP), Persona 2 Innocent Sin (PSP) and Persona 2 Eternal Punishment (Playstation 1 Classics) and I'm about to start Persona 3 Portable (PSP).
Persona 1 is a 1st person dungeon crawler with a cool story, two cool stories in fact as there is a hidden story line that you can only activate under certain conditions at the start of the game. Battles are fought on two opposing grids with different weapons having different ranges of attack across the grid, where as magic has infinite range but uses SP, this type of battle system requires more micromanaging but I found is more engaging as a result. The game also has demon negotiation that allows you to recruit enemies, take their money or gain information, the game does a poor job of explaining this feature but it involves filling up different meters for Happy, Sad, Surprised and Angry, if you get 3 points in a category or in multiple categories the demon will react appropriately. A super neat feature of the game is that you can choose your party members for a playthrough and then those party members will have completely unique dialogue during the story. However the game is very difficult and if you want to get the good ending you will NEED to use a FAQ or guide, especially during a certain obtuse section starring a ginger bread house.
Persona 2 Innocent Sin is a 3rd person RPG with another great story with plenty of twists and fun characters. There is more emphasis on exploring and talking to NPCs as after every dungeon or story event all the people you can talk to in the city get new dialogue, making it feel like a city that reacts to the actions made by characters. Another interesting system in the game is that you can learn rumors from rumor mongers, pay to spread these rumors, and then they will come true. These rumors may be about items or shops or even side bosses. The battle system is no longer grid based but the new aspect of battles are that you can perform combo attacks for massive damage by using different types magic in certain orders, these combinations are never told to you so it is up to you to keep trying different combinations to see what works. The issue with Innocent Sin is that they seem to have taken Persona 1 feedback to heart and the game is completely devoid of challenge for the bulk of the game, you can go through most of it without ever upgrading armor or weapons or Personas. All magic costs the exact same amount of SP for a given Persona on a given character meaning as you level up some skills become completely redundant. Demon negotiation returns and you now can do combo negotiations with different combinations of characters.
Persona 2 Eternal Punishment is an alternate story set in an alternate timeline to Innocent Sin that follows a few new characters along with a few old ones. It reuses a lot of assets from Innocent Sin and a couple dungeons feel like they are straight copy and pasted with a small amount of changes made. The battle system though is overall more balanced and sits in the Goldilocks zone of difficulty between Persona 1 and Innocent Sin. Providing a decent amount of challenge without needing you to ever grind. It was fun learning about the new characters that were given a more prominent role than in Innocent Sin, but since the main story runs somewhat parallel to Innocent Sin there are less plot twists providing you have already played the first part.