It's a line from a song - a very old song.

I wonder do we distinquish the state of being grammatically incorrect from that of being ungrammatical.