The "patty cake" connection you spoke of rang a bell with this ex-coed-schoolperson. We used to hear the girls chant and clap:

Pat a cake, pat a cake baker's man
Bake me a cake as fast as you can
Pat it and prick it and......there my memeory fails me, but perhaps it will jog someone else's.

Don't know why it was Baker's man rather than Baker man, but I feel sure it was.

dxb