The Basement Stairs

Going up the basement stairs alone in the dark, you feel like something's behind you even though you know nothing is there. You're not in control of this feeling, but it's one of the strongest feelings you'll ever experience. You're wired to feel it. It's self preservation. That feeling exists because it helped keep our ancestors alive while they were evolving toward us.

In the same way, humans are vigilant for differences. If there's something obviously different about you, even if that difference is meaningless, people will notice. They don't notice on purpose; they're wired to look for differences. Again, it's self preservation. In a purely Darwinian sense, outsiders and people who act on independent impulses might be dangerous. Even if you don't want to feel this way, you will - just as you'll feel that fear of the basement steps whether you want to or not. Humans naturally isolate and exclude people that aren't like them.  Value judgments aside, humans need to be able to engage in exclusion for their mental wellbeing.  When they can't, they feel like they're the ones being excluded and they suffer.

You don't have to run up those last two steps. Even though you're afraid, you can stand in the dark and let the fear exist. That takes conscious effort, and accepting people who aren't like you takes conscious effort, too. The unconscious impulse, at least in humans, is always prejudice.