That's one possibility. I suspect there's some randomness involved, too. Families or other surrounding people can be more or less tolerant of whatever features a person has. Sometimes people just invent stuff to fuss about.