Honestly I think it is verbal laziness that makes people drop the n't from the expression.