PSP in my opinion is the best but you are better of with a game boy advanced cheap easy to use, user friendly and maybe after 2 years or so go for a upgrade
+1 For vamsi_krishna
I would also recommend, puzzles like Rubik's Cube and other similar ones.
I bet when your son reaches 8 Years, he will be able to solve mathematical problems better than his buddies who may be using a PSP or NDS today
Think about it.