One thing I learned in the kindergarten schoolyard: if you're visibly disabled or different, NO ONE wants you in their group. Heck, because I was mainstreamed, the kids in the special ed classes wouldn't speak to me except to call me names.

Just extrapolate that to larger groups and that's American society in general.