To me too this seems absolutely normal! We used to call it 'grasshopper logic'.

I concluded long ago that the people on the Board are very non-average. However, the "average" person listening to our family's conversation does indeed get confused. I think it's because we are perfectly comfortable saying the off-topic thing which we are thinking, and letting the conversation continue in that direction if it wishes, whereas most people may have those trains of thought but don't say them out loud. In fact, we often let the second thread continue and have two conversations at once, which is a bit tricky with only two people!