I second what Nicole says in terms of making sure he gives you some you time. Your age gap is very close it's understandable you feel the way you do ... also I wonder do you feel you have a safe space to be vulnerable with your partner? Aggression tends to come out when a person has bottled up feelings. So I wonder what's under the anger, is it fear, sadness, stress. I wonder could he do a counselling session to help this maybe once a week from home. So then he's in a better place mentally to mind you all rather than just relying on the gym which isn't actually addressing his issues really. And he could continue the gym when you're in a better place recovery wise. It took me two months of adjustment period (actually all of us!) when baby number 2 arrived. Ideally you shouldn't have to be worried about managing his aggression but it's obviously an issue so I wonder if you had a chat about it to deal with it in a way that actually benefits you all. Xx
Maybe suggest he go every other day? Or for 30 min instead of an hour. And maybe have him take care of the baby while you do something you want to do? Or have him go while you’re both taking a nap instead of just the baby. It’s important both of you take care of your mental and physical health.