Is the mass of the universe constant?
The energy of the universe is constant. This doesn’t imply that the mass of the universe is constant. The make up of the universe can consist of matter, which has mass, and non-matter, like light, which exists and has a certain energy but no mass. So if you look at the equation just mentioned E=mc^2 as applied to a nuclear reaction you will find that the energy in this reaction is the same before the reaction and after the reaction, but some of the mass dissapears. This is because some of the mass was converted into energy, such as emission of light. So the most exact answer to your question is that the mass of the universe is not constant but the universe has a limited makeup consisting of matter and energy, and the sum of these two never changes.