My opinion of the phrase is that it is a second rate knockoff on the much older one "poor as a churchmouse". Church mice were desperately poor in the sense that in the early churches when the phrase was coined, there was no food stored nor any eating (except communion wafers).