Hmmm, i was going to say it was the video card too...
My best guess at this point is your motherboard is sequentially failing. Rather then crap out all at once, certain components are going. Is it onboard audio? It could be that you got a bad mobo, or its going bad. First the GPU slot failed, then the onboard audio, and tommorow if this is the problem the computer might not start at all.
If you have anyway of getting a hold of a free compatible motherboard, try it. You probably won't be able to boot to windows as using a new mobo will make windows think your installing a new harddrive in a different computer to void copyright laws and activate a self-shutdown, but you should be able to turn it on and get to the windows loading screen and see the windows logo before that happens.
Now, you said,
When the dispay turned off as well.
Does this mean it stays on, then goes off? Often it's the stupid things that get us. Check your audio connections and your VGA cable. Make sure their tight and inserted properly.
Heat has also been known to be a major culprit of strange problems. Although this doesn't seem like heat is to blame in this situation, it can't hurt to check. If you can't check your internal temperatures place a thermometer in the case for an hour with the computer running and get as accurate a reading as possible.
Most of my options are longshots except for the motherboard. If the motherboard isn't the culprit, and neither are the other things i listed, then i think i speak for most of us when i say i'm truly stumped.