We show that the general and numerically exact Hierarchy of Pure States method (HOPS) is very well applicable to calculate the reduced dynamics of an open quantum system. In particular, we focus on environments with a sub-Ohmic spectral density (SD) resulting in an algebraic decay of the bath correlation function (BCF). The universal applicability of HOPS, reaching from weak to strong coupling for zero and nonzero temperature, is demonstrated by solving the spin-boson model for which we find perfect agreement with other methods, each one suitable for a special regime of parameters. The challenges arising in the strong coupling regime are not only reflected in the computational effort needed for the HOPS method to converge but also in the necessity for an importance sampling mechanism, accounted for by the nonlinear variant of HOPS. In order to include nonzero-temperature effects in the strong coupling regime we found that it is highly favorable for the HOPS method to use the zero-temperature BCF and include temperature via a stochastic Hermitian contribution to the system Hamiltonian.