Depends on how strict to the rules you are. If you're severely strict on the rules and make sure everyone follows them, sooner or later you'll find yourself reminding a popular member of the rules, and bang, you're hated by quite a few. This was what happened to me before I decided I wanted out of the staff position because I was sick of the tantrums and the PM's telling me how I should do my job. So, yes, it can be.
Plus, lets face it, this place is massive. Keeping an eye on every single user and their post can become quite tedious at times. But hey; I'm back as mod so it can't be that bad.