The hafnium oxide (HfO₂) material system offers a unique combination of outstanding physical properties, that enable a manifold of novel integrated ferroelectric, piezoelectric, and pyroelectric applications. Long-term stability is an essential concern for nonvolatile memory devices, sensors, and nanoelectromechanical systems. Herein, the aging effects of the pyroelectric response in polycrystalline Si-doped HfO₂ thin films in the field-free case are reported. It is observed that aging effects are accelerated by high temperatures, lower film thicknesses, and higher dopant concentration. The decay of the pyroelectric coefficients and the dielectric permittivity exhibits a logarithmic time dependence. The full pyroelectric response is restored by repeated electric field cycling (i.e., deaging). After the aging process, a significant internal bias field is observed. It is concluded that the migration of positively charged oxygen vacancies in the films is responsible for this aging process.