Everyone is quiet, but they're shouting at me

I am experiencing a strange situation at the company I work for lately. My manager has warned me about leaving late, but when my other colleagues act the same way, he doesn't say anything. It clearly feels like I am being singled out. I can't understand the reason; I try to pay attention to what he says, but I still feel like he is keeping an eye on me. What do you think I should do, or am I thinking wrongly?