Logically there's no difference between 'people of color' and 'colored people.' Any problem I might have with 'people of color' is that it sounds so awkward.