"You listen to yourself too much" just seems to miss something implied in Napoleon's words.

Literal translations can be misleading. Perhaps it was meant to imply that he should have listened to others a little more.