Thanks for your views.

Blue and AnnaStrophic, like you, I think goat cheese is best. ( Or was Anna Strophic just unable to resist cruelly pinning me with her comparison?) Looking at some similar terms seems to help.

I fail to see why goat cheese should be treated any differently from other made food items like apple pie or chicken liver pate or oyster soup.

Whether actual bits and pieces like lambs fry or pigs trotters or straight extracts like cows milk should take an apostrophe is another question perhaps best left for another day. I'll just say that I think that apostrophes (unlike anastrophes) aren't always particularly useful and though we're stuck with them as long as they're used in educated written English, we shouldn't miss any opportunity we get to dispense with them. Not a universal view I know.

JMH, as you say there is no consistency of approach among food writers. I'm not sure that we must look to the French but even if we do, I think that while fromage de chevre is the most common term used, chevre fromage is used not infrequently and, occasionally, even fromage-chevre. Anyway, even though I don't agree with your conclusions I must say that I found the picture you drew of the cottager and his companion goat producing little pats of cheese a most charming one.