Little boy cries when certain ppl hold him

My little boy used to cry at his grandma all of the time until he was around 8 months old. It is very common for children to cry at the people that almost feeling awkward/uncomfortable when holding babies.

Equally my neice is 18 months old and still cries at my husband sometimes

Sometimes its a case of familiarity. Are you his primary parent? Do the men have deeper voices, facial hair, ect?

Often women have a bit more of a fuss/appear more enthusiastic and happy (big smiles, higher voice, more comfortable and confident in holding) and thus baby feels often happier/more comfortable, especially when mother is primary parent, because they recognise the behaviour

Honestly, most babies in my family/friends were like this too, I think it's the energy. Mother's hold a different energy compared to most dad's. Women just give off a different kind of energy (I hope this makes sense to you)