Well yes, it could be worse--he could kill you and he just might put you in the hospital if you continue to stay with him. Hitting you AT ALL is not acceptable, neither is the name calling and him seeking females. I don't know why he let things go on all these years and then all of a sudden is being mean to you, but he did not forgive your affair (if that is in fact why he's acting this way now). If he did forgive you, then he wouldn't be mean to you. Yes, what you did was wrong, but you were also young, naive and probably not ready for marriage. So if he felt that he couldn't forgive you, he should have just left you at that time--it's not right to stay with someone only for the purpose of abusing them. I am glad you are going to counseling to get some perspective, but you should not stay with this man if he continues to act this way.
If you were not in counselling or counselling fails, I would recommend leaving him. Hitting, even if it does not cause actual physical harm, is dangerous. Maybe the next time, there will be severe damage to your body. Your husband would have been justified to leave you for cheating on him, but not justified to hit you, especially after he supposedly forgave you.
Reading this post, and your replies to the others.....makes me very sad. You did what a lot of young girls do......you had an unhappy home life, so you married the first man who came along.....not because you were in love with him.....but because you saw him as a way out. You then looked at him like your savior, but at the same time, you were totally dependent on him for attention, and he was still a kid, and wanted to play video games. His having other interests upset you, and you did the same thing.....you looked for someone else to "comfort" you! You were lonely! When he started travelling you travelled with him and you were happy. Then you got tired of traveling, so you stayed home and he was gone, and you were lonely again, you had another affair. You didn't "fall in love" with a coworker.......you wanted "comfort", and he was willing. When you're not happy with yourself, you don't like yourself, and you need attention from others. If you don't get that attention, you look elsewhere for it. It all comes back to the verbal abuse you grew up with......you were taught that you were nothing, you were worthless, and you married and had affairs because you thought that validated you. You thought that if a man gave you attention, that meant you were worth something. It was only for that moment, and then it was gone again. Your husband was/is a damaged man too. He wasn't able to accept his father's death without you by his side. And because he was angry with GOD for taking his father away, he took out his anger on YOU because he couldn't beat God up, and you were handy. A mentally healthy man does NOT put his hands on a woman in any way, EVER! No, it's not right. And you have children who witnessed the abuse!
When you were married, you were both too young, and you were both emotionally unhealthy. You're both still young, and have time to heal yourselves and move on and have a happy life, but you cannot do that without continuing your counseling. Be sure you tell the therapist about your emotional abuse as a child.....I'm sure it will come up. You are only in your mid-twenties, and you have a lifetime ahead of you......make sure for your sake and more importantly for your children's sake that you work on yourself and your promlems before you even THINK of being in another relationship. Your husband has moved on, he's looking for other women, and you don't owe him anything, he didn't rescue you at 17, he just took you into another unhealthy relationship. You need to learn to love yourself before you will ever be able to truly love anther person. For your childrens sake, keep going to the counseling to fix yourself, don't worry about your marriage, it was never good, and it probably never will BE good.