we might be able to limit ourselves to 11 basic names for colors

Well, the eleven basic colours of English are really a compromise list. I've found it varies much more by individual, but you can't put idiosyncrasies of perception into Roget's Thesaurus.

I have a twelfth basic colour, crimson: anything deep red, claret, burgundy, or plum is to me not red. It's crimson. (Hungarian also has a separate dark red.) I cannot see crimson as a kind of red. I have no idea why.

My family are artists (I'm not), and they have interesting variations. They all maintain that aqua/turquoise is a disinct basic colour, and so is cream/ochre. (Or possibly cream and ochre/tan were two extra colours, I can't remember.) My father also distinguishes scarlet and red, insisting that if something is scarlet it isn't red (= a deeper shade, but not my very deep crimson).

Then there's the different question of what you feel ought to be distinct basic colours. I call both light blue and dark blue "blue", but I'd be eminently happy using distinct terms, as they do in Russian (siniy and goluboy). So to a lesser extent with green, except that there are a lot of middle-range greens in nature to bind the light and dark ones.

I also feel it's wrong to lump light brownish shades such as tan or ochre with cardinal brown. (French distinguishes beige from marron IIRC -- the word brun isn't the real equivalent of 'brown' in all its English use; but I am open to correction by native speakers.) Nevertheless, I don't feel that tan names a basic colour. As far as names go, tan is a kind of brown, even though perceptually I want to elevate it to basic status.

But I don't feel this with aqua. I am happy to say an aqua thing is either blue or green or something intermediate: that is shades of blue gradually give way to shades of green.

I suspect anyone would come up with equally subtle complications once you examined them in detail.