Well, I thought of this concept a while ago also. You need a router, you can use one broadband modem in your house and plug a cat5 wire from the modem to a router and then take 2 cat5 wires and plug them into the computers and the ports on the router.
Where it says WAN (Wide Area Network) is where you plug in a CAT5 or Ethernet cable from your broadband modem to the WAN port.
The 1,2,3, and 4 Ports are where CAT5 or Ethernet cables plug into from the 1-4 computers you want connected to internet.
You can get a cheap router that will work easily at newegg ranging from 25-100 dollars depending on wireless or wired.
If wired may I suggest this:
D-Link, Linksys, and Netgear make the best routers in my opinion.
P.S.- Like daemon said, yeah you could use 2 modems. Also, your internet on both computers will be split in half if they are both in use. I personally cannot tell a difference between the router in my home.
BTW, By above I wasn't trying to make you feel stupid giving you such specific instructions, I myself didn't even know the concept of what a router did until not too long ago.