We study the interaction between gravitational waves and quantum matter such as Bose-Einstein condensates, superfluid helium, or ultracold solids, explicitly taking into account the changes of the trapping potential induced by the gravitational wave. As a possible observable, we consider the change of energy due to the gravitational wave, for which we derive rigorous bounds in terms of kinetic energy and particle number. Finally, we discuss implications for possible experimental tests.