Charles Funk, in A Hog on Ice, says that the phrase seems to have originated in some specific operation which would have required the removal of successive layers until the brass tacks were exposed, but that the references only contain the figurative use and are too recent to afford any clue to the purpose of the first brass tacks. He recounts the suggestion that the saying originated from the brass upholstery tacks place on counters in drapers' shops for use in measuring lengths of cloth, but says that explanation seems fanciful to him because the practice is not old and the tacks of comparatively modern manufacture.

OTOH, David Feldman in Who Put the Butter in Butterfly? and Jordan Almond in Dictionary of Word Origins both say that the expression does come from early English drapers' shops.