RPM stands for Red Hat Package Management. It's the system used by Red Hat based Linux distros for downloading certain packages, me thinks, I'm a Windows user.
RPM stands for Redhat Package Manager. It's like the .exes for Windows, only it's for GNU/Linux. Tar.gz is not a file format. Its actually a compression format. It's kinda like zip files. However, most source code for programs are using tar.gz so people have the misconception that tar.gz is a source code file format.
GNU/Linux is not as straightforward as Windows. In Windows, formats are rather limited such as .doc for documents and .exe for installers. In GNU/Linux, there can be numerous formats. It takes a little getting used to but most of the time you'll be sticking to the same few formats such as .sxw for documents, .ogg for media files and RPMs for installing software.