> why I tolerated his behavior for so long (the kids)

According to most therapists, when we submit and tolerate awful behavior from others, it stems mainly from something inside ourselves: trauma, emotional wounds, fragile identity, etc. Usually it's due to something bad we experienced in childhood, that left a mark on our psyche.

Of course there could be other reasons (e.g. the kids), but it's usually misleading thinking those are the only ones. We better look inside as well; if we don't, we might do the same mistakes again.