Well, he was a tormented and certainly not a happy-go-lucky man.
But he was not insane or a lunatic and that he found comfort in the company of children does not make him more insane then he wasn't in the first place. But I'm sure this is no point to make a real point of. The quote surprised me too.(pleasantly) At least something to give some lightness in his life.