It seems certain the phrase "by heart" has to be very old. Before the circulation of the blood was understood, quite a few mental processes were ascribed to the heart."Put your heart into it" - meaning try very hard. "Where is fancy bred, in the heart or in the head?" Insults may be "taken to heart". "Good hearted" - meaning generous, kind.