>The question of pejorative seems, as usual, more tied to the inclinations/sympathies of whoever is doing the talking...

Yes, I've often been the only xxx amongst a group of yyy, female/male, white/black, able-bodied/disabled, heterosexual/homosexual - I've learnt to be very, very careful with some of the words I've come across. In most instances words which are OK within a group are strictly not OK if you are not part of the group.