I cannot say what I would LOVE to say to your "husband" right now, as I try my best to leave all negative statements unsaid.
So instead I will only say what I want to say to YOU right now and that is this.
You are not harming your family in ANY way by taking this trip. You are only probably inconveniencing your husband because he might actually have to do something for himself and so he is trying to make you feel guilty. Forget that! You deserve this (and honestly) a WHOLE LOT MORE.
Your kids will still eat, that is definitely no question. Your family will not suffer, in fact they will all probably be better off for it when their mommy returns happy and refreshed. So go, have yourself a care-free and great time, you deserve it. About the money thing. If he gives you issues you tell him that you are going one way or another so if he chooses to send the mother of his kids on a trip with no money to support herself then what kind of person is he and what is HE doing to his family?
That is a shame.
But I know that this trip will happen for you, you need it, you deserve it, so it will be! Keep the faith.