The exact stochastic decomposition of non-Markovian dissipative quantum dynamics is combined with the time-dependent semiclassical initial value formalism. It is shown that even in the challenging regime of moderate friction and low temperatures, where non-Markovian effects are substantial, this approach allows for the accurate description of dissipative dynamics in anharmonic potentials over many oscillation periods until thermalization is reached. The problem of convergence of the stochastic average at long times, which plagues full quantum mechanical implementations, is avoided through a joint sampling of the stochastic noise and the semiclassical phase-space distribution.