You need to navigate to the visual effects tab in the "Performance Options" window inside the advanced system settings.
This mini-guide should work on Windows 7. Use either of these methods to get to the setting.
Method 1:
Click the start button down the bottom left. In the search provided copy and paste the following into the search:
%windir%\system32\SystemPropertiesPerformance.exe
This will open up the window you require. You will see a screen with lots of tickboxes. Tick the box (if it isn't already) that says "Show thumbnails instead of icons." Now click "OK." Your pictures should know show as mini versions of themselves instead of as the icon.
Method 2 (same as 1, but you get to the required window differently):
Step 1: Click the start button down the bottom left.
Step 2: Click "control panel"
Step 3: Click "System and Security," then click "System." (or just click "system" if you're using view by small icons)
Step 4: Click "Advanced system settings" at the top left of the screen
Step 5: On the window that pops up, click the "Settings" button next to the "Performance" box
Step 6: In the window that pops up there will be lots of tickboxes, tick the "Show thumbnails instead of icons" box (if not already ticked) and click "OK." Your pictures should now be displayed as thumbnails, not just an icon.