I just don't want to hurt her or harm her by holding her up in the sitting position too early?
If you support her and all the weight is not on her back she should be fine 🥰 they soon will start to sit up anyway, around 6 months, so just 2 months left to wait 😁
I let mine sit in my lap as she enjoys sitting up. It’s not recommended to use containers like bumbos, jumperoos, walkers etc for more than 20 minutes at a time though. I think you can gently encourage but using containers too much can stop them from developing naturally.
My girl likes to lay half way up on feeding pillow or bouncing chair, sometimes I hold her up in sitting position. But I don't think there is anything much you can do, just wait until they are strong enough 🙂