Am I The Asshole

So my man and I got into an argument on Wednesday and we just resolved it last night. He got angry at me because I didn't have time or money to go visit him. We're in a LDR. I leave my teenagers,.my animals, my comfort zone to go visit him and I feel taken for granted. But he felt upset because he thought he was begging me to go see him. I usually end up staying almost a whole week when I do visit. So on Thursday I got mad that he was upset with me over such a petty issue. Now there's all this tension and I'm not sure what to do. He was drunk Wednesday and Thursday night so he was saying things that really hurt me, of course the next morning he denied meaning anything by it. I'm just stressed and depressed about it because I thought he was my safe place. I'm feeling hurt, alone and defeated.