It was for more than just the offensiveness of the smell that the nosegays were carried. Right up until mid C19, the majority opinion among the medical profession was that whatever it was that carried disease was actually in the smell itself. This is known as "the miasmic theory of disease." So the herbs, etc in the nosegay were actually being used as a form of disinfectant, rather than as an aid to olfactory aesthetics.
From the 1850s, the belief that disease was carried by minute organisms began to overtake the miasmic theory, and was conclusively proven by Robert Koch in 1896 (or '98 - can't remember!) But there were still some reactionary doctors who held to the miasmic theory well into the C20.