I suggest you put in the origional card, boot it up and see if it works, and uninstall all the drivers for the card. Then turn the computer off, and insert the new card and cd. If that doesn't work then... well; you MAY have to reformat!
Since you can't reall get into your windows box because it glitches out, that's all I can think of. You might want to try checking to see if it's not using BOTH integrated graphics AND the card. They could be conflicting. Make sure integrated video is OFF, and make sure it knows there is a card in there. Also, pull out the card, blow on it, and put it back in. Then turn it on and see what's up. If none of this works, make sure the card is securely in place on the motherboard and do a clean reformat.