When and why did hats fall out of fashion as an everyday item of a man's attire?

Alas, it appears that the ubiquitous baseball cap has usurped the proper hat's place, and the custom of men removing their headgear indoors has also vanished. I still do it, but I seem to be the only one.