It would, but less than the density difference, since you’ve removed weight from the balloon thus gravity has less of a pull on the balloon. My wife (a PE in thermodynamics) was the one that verified that comment before I posted it, hence why I didn’t say it would increase lift by the difference in density.