The reason why there has to be something between the CPU and heatsink is that the two mating surfaces of processor and heatsink aren't perfectly flat. They may look flat and have a mirror polish. But, on the microscopic scale, they look like a scale model of the Rocky Mountains. And the mountains on one item do not match the valleys on the other. It's like what Marxsoccer was saying.
Without thermal transfer compound, everywhere heat sink metal doesn't mate with CPU package material is a very small, eensy weensey, air gap. Air is a good thermal insulator. As long as your heat sink looks flat when you lay a ruler on it then there'll be a decent amount of actual contact, of course, but the amount of heat that'll actually make it around the air gaps may be surprisingly small which introduces a thermal layer.