The more that research is done on "harems" within the animal world the more it is found that this is unclear at best and wishful thinking at worst.
It is the dominant mare of a herd of wild horses that decides where they will graze etc. The impressive looking male of a pride of lions is dependant on the females for successful hunting and often to help him chase off encroaching males. If they withdraw their support he doesn't last long.
A documentary of one type of primates (macacs???) showed that they lived in a large herd of smaller family groups consisting of related females and their young and an unrelated "dominant male". At one point the females of one family suddenly turned on their male and very aggressively drove him off even biting him. The cause seemed to be some injury to an infant, possibly accidental. Eventually he was allowed back and spent a great deal of time grooming the females and cuddling babies to regain his position.
With many primates the infants are very important and a male with adultory on his mind may make friends with a baby first to gain entry to a group since no one will attack when he is holding an infant.
It seems that many of the so called harems out there are actually matriarchies with a kept stud.