Court

Depends what he's taking you to court for? More time with baby, overnights, PR, CAO...?
Re abuse, it won't necessarily be relevant as you already allowed contact and even unsupervised. Do you deal directly with your ex when he collects baby? If so, whatever he did in the past to you will definitely not be relevant to child arrangements

More access with baby and for his family/friends and name change.

If you kind of go along with what he wants, they'll take that as mutual agreement. If you don't want something or it makes you feel uncomfortable, you have to make it clear. Name change seems a bit of a piss take. He wants to change baby's name to his name? I'll knock him out wth

He’s been horrendous since the moment I’ve given birth so demanding and trying to revolve it around his family and friends!! So I’m going to stand firm, and I’ve said he can visit our baby at my house. I think he thinks our baby is a toy. Our baby has my surname so he wants to add his, he was told no when we registered him but he didn’t listen

Yeah mine threw a tantrum when I told him I wouldn't give the baby his surname if he wasn't willing to put the time in to see him.
Just make sure you stand firm and what your conditions are

He will most likely get more contact as you agreeing to contact already means there's no concerns. Babys age and being EBF means that contact would be often and short periods of time, ie: 1-2hs every other day is common.

If he's been involved since birth + kept his weekend time, he's prob already bonded so alone time with baby may not be a priority. Baby is still young though so it will honestly depend on how you present your reasons for denying contact with his family/friends.

He can't remove your surname to add his but courts will likely ask to double barrel as they see it as part of baby's right to identity