"starting from scratch" is indeed derived from early sports , amongst other things it applies to early prizefighting, where two lines would be drawn (roughly 18 inches apart) from which each boxer would begin. when a fighter got knocked down, he would have 30 seconds to stand up and put his foot back on the line for action to recommence hence "toe the line"

the Duncster

the Duncster