Well, I have a $25 set of 2.1 Creative speakers, and they rock the house - even at high volumes. It is possible that the speaker quality isn't good, but there are a lot of other possibilities.
If you have a CRT monitor, make sure the speakers and their wires aren't too close to it. The magnetic fields could cause some interference. Also, some appliances and other products can cause interference like this if they are plugged into the same wall outlet. A variety of other things could cause interference as well. A few examples are cell phones, radios, televisions, power cables, and anything with a motor in it.
Barring outside interference, the sound card or software could be causing some trouble. Borrow a friend's sound card to see if you get the same results. I suggest doing this with speakers, too - try another set to see if it really is just the speakers. Also make sure all your audio drivers are updated to the max.
EDIT> And yes, make sure they are plugged in all the way.