Would this be a red flag for a future/potential employer?

Could be, but definitely go for an interview and feel it out. You never know what people have going on( their reasoning for late responses)

What type of job are you applying for?
Some context on that might help determine if this is something that would effect your day to day at work.

Depends on the job you applied for. If the job requires them to work late/not a Monday-Friday 9-5, you could’ve caught them on their Friday and then the response wasn’t for a few days because it was their weekend. When you go for the interview make sure to ask them questions to see if you find any red flags. Remember you may be interviewing to get a job there but you should also see it as an interview from you to them to make sure it’s the right fit on both sides.

Sounds like a sign, not a red flag. If this is a value you don't want to shift on that totally fair. Go with what aligns with you and feels right.

Also, the hiring manager isn't necessarily who you will be working with/for. (Unless you've already confirmed they are) If you're worried about being contacted outside working hours when you actually have the job I'd ask them questions in the interview about whether those values are aligned.

Personally, I just don't bring my work emails home. They're not on my phone so I can't be contacted outside work. They can email me any time of night but I wouldn't be expected to read it or respond until morning. Right now is probably annoying because they only have your personal email

It really depends on the job type and working hours that go with that, not just your available hours etc. In some fields it would be perfectly normal for the hours theyre replying to be part of their working hours or potentially the end of a working day and depending on the field they may not get much free time when working to handle paperwork and do so when they can within their working day. If that doesn't align with what you want then that's fine of course but whether it's a red flag needs more context

I think that’s fairly normal. I’ve had similar experiences. People are busy and interviews aren’t always their top priority. Go for the interview and see how you feel!

Is the hiring manager for sure located in your same time zone?

Being emailed late at night is better than not being emailed at all, at night may be the only time they are able to get this admin side of things done
When I was hiring for a retail role sometimes I would have to correspond via email at 8/9pm at night otherwise I never would be able to