I think the best suffix as an answer to your question is -phobe itself. While it may derive from a literal fear of something, in common use it also means simply an aversion or a dislike. Whether the user means to imply that the aversion is the result of an actual phobia -- the politically-charged term "homophobia" comes to mind -- is up to the user I guess. Applying -phobe to everyday objects seems a bit like overkill in any case. Is our current POTUS a broccoliphobe or does he simply not care for broccoli. As for myself, I am definitely an okraphobe. Indeed, I fear the okra.