I wouldn't have called my speech slovenly, but I certainly pronounce the unstressed form of "have" after a modal verb and the unstressed form of "of" very similarly, and I suspect most people do. Try it yourself by saying these in ordinary speech, rather than with the artificially precise pronunciation my mother would call a telephone voice: I should have loved him more. and I dreamed of love. The "have" and "of" do sound very similar. If, however, you say You should have and What did you dream of?, the difference becomes more obvious.

Bingley

Bingley