this was discussed before...
and there is even a sticky about it...
yuo're not going to be able to find out who's left you rep other than when they sign their name, this is to stop people just negativly repping and flaming because negative rep was recieved etc...
if the rep is random and meaninles then you shuold report it to david, who will investigate, remove the negative rep and add the person to the list of named and shamed pointless negative reppers...
what's more... their ability to leave rep will also be taken away.