Then the more powerful partner (be it man or woman) would likely impose his/her will on the weaker one, and/or on the children.

Both during the relationship, and when ending it.

Don't fool yourself: humans are power-hungry. Given the chance, they would abuse their power. We need society and rules to temper that impulse.

Anarchy simply means that might makes right.