The spectral properties of a quantum system are essential when probing theoretical predictions against experimental data. For an open quantum system strongly interacting with its environment, spectral features are challenging to calculate. Here we demonstrate that the stochastic Hierarchy of Pure States (HOPS) approach is well suited to calculate the response of an open quantum system to a, possibly strong, coherent probe driving. For weak driving, where Kubo's linear response theory is applicable, it turns out that the HOPS method is highly efficient since fluctuations inherent to the stochastic dynamics cancel for the response function and, thus, allow us to obtain the susceptibility easily. Our results are in agreement with experimental data for a strongly damped spin system showing that the transition from oscillatory to overdamped motion is also reflected by the transmission spectrum. As a further application we demonstrate that the susceptibility, quantifying the amplitude of the response, as a function of temperature exhibits a maximum which is the hallmark of stochastic resonance. Beyond the linear regime, the exact open system dynamics shows the asymptotic Floquet state. We use the topic of probe driving and response to present the HOPS approach in a novel and self-contained way. This includes the importance sampling scheme which yields the nonlinear HOPS as well as the stochastic treatment of a thermal initial environmental state within the zero temperature formalism. Special attention is given to the exponential representation of the algebraic Ohmic bath correlation function and the truncation condition for the hierarchy.