Yes we are quite sticklers for most rules of grammar. In "Fromage de chèvre" we are saying cheese FROM goat. If I said "de la chèvre" I would be talking about one goat in particular. It also somewhat implies possession of said chèvre. I hope I have the English term correct...DE (feminine) as well as DU (masculine) and DES (both M & F) are prepositions introducing a location, origin (the goat in this case), time, possession, cause etc.

LA (feminine), LE (masculine) are the articles. As I am sure you learned, everything is either masculine of feminine in French. La chaise (chair), le bureau (desk), LES is used to indicate more than one item for both feminine and masculine items.

You are right chèvre fromage is very grating and it makes no difference if it is plant/animal etc. I find we are not too keen on removing excess articles, prepositions et al. That is why the French text on everything is twice as long as the English text. To answer your questions, always include the article & preposition and you will never go wrong.

Geez, this is the first time I have ever talked French grammar for fun. Usually I am correcting label text and arguing about it (verb tenses, of which we have, say, a bazillion, are usually the culprit.) Maybe I'm getting fuddy-duddy without knowing it. I think I'll go wrestle a bear now. Salut!