VGA is an analog signal and is more susceptible to interference, while DVI is digital, and is not. Usually, you will not notice much of a difference in image quality between the two. Although in some cases I have noticed a marginal difference. If DVI is available, I use that. If only VGA is available, I don't worry about it.
I have heard that DVI might have some advantages such as higher refresh rates and such that would benefit gamers. But I don't have a source on that.
EDIT: sorry I was thinking of DVI, not HDMI. But essentially they are comparable as they are both digital signals that support high resolutions. But as celery said, HDMI can support audio. For a computer I would stick with DVI and for TV, use HDMI.