It was my experience as a Muslim in the US that most people were conspicuously deferential towards me, very self-conscious about demonstrating their tolerance and lack of bigotry. However, it would also happen that when I would let people know I am not an easily offended, hyper-sensitive Muslim, some people would assume from this that I am not serious in my religion, and they would begin leaking hostility towards Islam and Muslims, mistakenly thinking I was going to either agree, or feel the need to acquiesce in order to prove I am not a rigid extremist.
I think a similar dynamic takes place oftentimes between Black people and White people in the US too.
Because of this, everyone keeps their guard up.