In my opinion I think you should stick with the 32, although it does depend on how you are financially.
I love the series 5 rx7 too and almost got one in place of my r32 skyline, but I decided that the race look of the 32 and afterburners was the way to go.
The series 5 looks better than ther series 4 coz the brake lights are round in the back like the series 6.
If you decide to go the rotary, like the skyline you can do just about anything to it. I also think the buzz of a rotary plus the spool of a turbo = mad sound.
The 32 looks a little more modern in my opinion, although they are both very fine cars, Good Luck with you decision buddy!