I think 3 had the best driving of the series. Now, taking them at base, un-modded state.
Now the modding community have people that go through, look at each car, and try to tweak each car accordingly, more than likely, spending more time on that single aspect than the original developer(s) had allotted to them for the same project. Most of the vehicle roster seems to have just fallen somewhere on the table numerically, regardless of whether those numbers made sense for the car model or not. In 3 it seemed that whoever was in charge of the vehicles had their favorites, and tried to make those drive as well as they could, without calling for someone to re-tweak the entire physics engine. Heck I fire up Saints Row 2 just to cruise around in my various vehicles in hopes of The Boss singing along to the stereo. Now, having said that, I personally prefer driving cars around. Probably similar to why they just dropped boats from IV altogether and kept the (largely physics ignoring) super powered travel instead. I've accepted that, for the sake of the rest of the game, they couldn't make the driving portions as sharp as they could have been.
