We present a fully quantum dynamical treatment of a quantum heat engine and its baths based on the Hierarchy of Pure States (HOPS), an exact and general method for open quantum system dynamics. We show how the change of the bath energy and the interaction energy can be determined within HOPS for arbitrary coupling strength and smooth time dependence of the modulation protocol. The dynamics of all energetic contributions during the operation can be carefully examined both in its initial transient phase and, also later, in its periodic steady state. A quantum Otto engine with a qubit as an inherently nonlinear work medium is studied in a regime where the energy associated with the interaction Hamiltonian plays an important role for the global energy balance and, thus, must not be neglected when calculating its power and efficiency. We confirm that the work required to drive the coupling with the baths sensitively depends on the speed of the modulation protocol. Remarkably, departing from the conventional scheme of well-separated phases by allowing for temporal overlap, we discover that one can even gain energy from the modulation of bath interactions. We visualize these various work contributions using the analog of state change diagrams of thermodynamic cycles. We offer a concise, full presentation of HOPS with its extension to bath observables, as it serves as a universal tool for the numerically exact description of general quantum dynamical (thermodynamic) scenarios far from the weak-coupling limit.