Just a speculative theory. But back in a more rural, small-town era ladies' "bake-offs" were quite the annual event. Women became famous in town for making the best pies, cakes, cookies, etc. And brownies were high on the list of importance (along with pumpkin pie and a few others). So, perhaps, the expression as applied to "getting on the good side of a lady" with compliments, charm, or mannerful gestures relates to that, especially when awarding her brownies points as an easy way of getting on her good side...but then you have to earn them back with proof of your sincerity. Or maybe it was simply taken from complimenting a woman on her culinary expertise.