Quite

It's called 'denying the antecedent.' It's a logical fallacy. That's part of the reason the joke's so funny.