I have both, I prefer the PSP.
Reason is because I have a PC, and most games that comes out to 360 comes out to PC. So my 360 was kind of a waste of money. I only have 5 games on the 360, and those games are the ones that are exclusively to 360 or home consoles. Though there are 20 360 games in the household, I only own 5. I am awaiting for Forza 3 as well.
The 360 looks like crap on an small SDTV, I can barely read any texts or see an details that wowed me. So make sure you are going to play on an HDTV or an monitor. That's my opinion.
There arent much PSP games out that are fun either. I have been playing the same exact 3 games since I got my PSP. Monster Hunter, Initial D, and Crisis Core. I play other games now and then, but I get bored of them after a few hours and stop. It's nice if you have CFW, because then you can play your old PSX classics. I can't wait till Peace Walker comes out. Sony did say this is the year of the PSP with games like Assassins Creed, LBP, GT, Resident Evil, etc that were announced at E3. I think the 3000 series kind of fixed the ghosting problem.
I dont really care if my PSP gets outdated, as long as I can enjoy it, I'm fine. The Go! doesnt intrigue me, so I am fine with my 2000 series. I wanted an 1000 series because of better homebrew, but my mom made me buy an 2000 series instead because of the fake stuff in China...
Heres some Pros and Cons I made from my experience
Home Exclusive games
Cheap home console
Not much games if PC gamer
Looks bad on small SDTV
Cost of accessories
Xbox Live (some would disagree)
Achievements (some would disagree)
PSP exclusive games
Can't see under the sun.
Load times/battery consumption of UMDs
That's all I can think of now. Kind of pulling an all nighter, so I cant really think, and I may not make any sense.
There are ways around some of the PSP cons, but you need CFW for those.