Best bet is to get some type of flash video player for your site (you'll most likely need to convert the videos into .flv format).
Alternatively, you can look up in w3schools how to use the <object> and <embed> tags. I would not recommend this practice however because all browsers and users and operating systems have different codecs / implementations for embedding video in a web-page and it may not appear as expected. A flash wrapper is the only sure-fire way to give the user play, pause, seek, volume, etc... Plus using a flash player you can track how many times a video was watched, etc...
**EDIT: @CoolRabbit, I am also from Ohio. Whereabouts are you located?