Here is a hot take: think of it like leaving a house you have been renting for 10 years. Our relationship wasn’t with the landlord as much as it was the neighbors and community.
Now, our landlord turned to a money-grubbing asshat spaz spez. You better believe we are telling everyone in our community, old and new, not to rent from him. Additionally, we are going to be venting the hurt of being evicted. We have all been forced to find a new neighborhood and learn everything new that comes with it. Our habits have been reset and it sucks.
A new marriage combined with kids makes transition difficult, and it sounds like you both have had a lot of transition in the last 3 years.
I know this isn’t a relationship advice thread, it is AITA. This situation is so much more complicated than yes or no. Leaving your wife to take on all the responsibility of being a parent and vacationing without her is an asshole move.
At the same time, he seems to be hurting from the little you shared. If he is saying that the move ruined his life, he is likely struggling from loneliness. You didn’t describe any connection between the two of you at all. You guys may just need to have some sincere conversations about all of the change in the last 3 years and what you each need.