But when you are talking to people do you say it exactly like that... "I am a cat person" ?

Maybe it is because there is really no French version of this. I generally only hear this on English television and always kinda assumed that, like everything on TV, it doesn't represent the general population.