While I've known the phrase 'pull out by the roots' I've never thought about actually grabbing the roots and pulling. I know that you have to go down as low as possible to pull something out by the roots but I've always just thought of the phrase as meaning that you pull it out roots and all.

There's also two potential meanings of 'by the roots'. It could be an instrumental thing, 'by means of the roots' or it could be more of a locative thing, 'by, i.e., in the vicinity of, the roots.'