It really depends on what firewall you're using. Norton's has a little icon of a globe in your system tray (bottom right hand corner of the screen. On WinXP you may have to click the little arrow to show all of the icons and find it) Many other firewalls have a similar icon placement.
To test if it is, in fact, the fault of the firewall, right click on the icon and click disable. Most firewalls have that option. Then start up your internet. Make sure that IE is working online (file->work offline and make sure that the option isn't checked).
Service Pack 2 for WinXP has firewall options that I believe can be accessed and modified from your control panel (My Computer->Control Panel) and similar steps can be taken to disable it.
Sorry if that's too vague. Without knowing the setup of your computer it's hard to tell you what to do. What firewall is installed? What version of Windows?
Hope that helps a little any way.