Speaking of the alteration of expressions, around here, the phrase is usually "turn up like a bad penny."

I always wondered at the origin of the expression, and now I'm speculating that it is an ironical alteration of the black penny reference.