pronounce herd and herbal as 'erb and 'erbal

Do you pronounce the H in herb? I can remember being scolded as a child because I said herb like it was spelled. Then when I met a man named Herb and pronounced his name Erb everyone looked at me as if I were insane. It was quite a challenge for little 6 year old me to figure out why it was ok to call pronounce a man's name different than the plant's name with the same spelling. I don't know why it is pronounced differently on opposing sides of the pond, but my dictionary says

herb \'erb. US also & Brit usu 'herb\

so obviously Websters' acknowleges the distinction between a British herb and an American erb...