That must be a difference between UK English and American English - I've never heard it called goat cheese in the UK.

I've looked up Delia Smith and Nigel Slater (both highly regarded British Food Writers) and thought it was very straightforward - they both say "goat's cheese". That was fine until I looked up Josceline Dimbleby and Claire Macdonald (similarly highly regarded in the UK) who both say "goats' cheese". So the jury is still out here.

As the cheese orginates in France and we are merely translating their term "fromage du chevre" we must look to them for the answer. My French is fairly basic but I assume "chevre" is singular.

I note that goat's cheese is usually made in small rounds. It could be that the small size is that which would have traditionally been produced by a single goat. As it is a country cheese very few people would have had more than one nanny goat so perhaps that is why the singular is used. The people who made the cheese would also know the name of the goat (to reply to a previous point).