Don't be so quick to rule out the power supply... I had a similar issue when I was using a standard ATX power supply with a motherboard that required a non-standard, HP, power supply (Voltages were different). It started, then beeped once, then shut off - as if the protection kicked in.
Is this a custom build, or is it an of the shelf computer? What is the make and model? Has anything been changed? Was it ok before this?
It could be a multitude of things. But a memory, graphics card, or any other device usually does not force the machine to shut down - they usually keep the thing powered and have warning beeps with specific patterns to signify the fault.
I would check the CPU is properly seated, and fully compatible with the motherboard, check the wiring, perhaps unplug devices and test to see if something is causing a problem. Do this for the power supply for each device too - as if it is an overload issue of sorts, this might help find the culprit.
Anyway, as Johnny Five says: more input...More input! Tell us more about this machine.