Well of course, it depends what you are running BobbyDRZA, and the switch might just do what you said, but 5.1 is to immerse you more with a centre speaker for dialogue and centre effects (though with 4.1, this is a virtual centre speaker)
With anything though, if the signal has to be stretched farther (music and games that aren't meant for 5.1) then of course some of the other speakers are going to loose out on some of the output since it has to range the sound through an extra speaker. Get me?
If you watched movies using Power DVD and such on 5.1 surround sound mode, you would notice an improvement.
I hope this helps