you could just turn it around

Exactly. Proper use of parallelism does not require you to repeat the by unless not doing so would create ambiguity. When sweeping clothes becomes something one might be expected to do *then "sweeping and washing clothes" might become ambiguous, but, *if you're worried that someone would get all tied in knots imagining poor Mary sweeping the clothes, simply reversing the order and saying "washing clothes and sweeping" should be sufficient.