And why do I have no problem using "without" in Jackie's example if I leave out the "you"?

Self-confidence comes from within. Self-confidence comes from without.