Within the realm of atomically thin semiconductors, CrSBr stands out uniquely as it harbors quasi-one-dimensional magnetic excitons in both its bulk and monolayer forms. Despite its fundamental significance, the precise exciton formation, relaxation and decay dynamics remained elusive. While terahertz polarization probing offers a direct avenue for tracking excitons, the resulting far-field signal can be affected by signals originating beyond the lateral dimensions of the sample. Here, we combine terahertz polarization spectroscopy with near-field microscopy to unveil the sub-picosecond decay dynamics of paramagnetic excitons in a CrSBr monolayer, which is faster by a factor of 30 than its bulk counterpart. Our analysis also unravels low-energy fingerprints of bound and unbound electron-hole pairs in bulk and gives access to the non-equilibrium dielectric function of the monolayer. These findings provide direct insight into the ultrafast dielectric response of quasi-one-dimensional excitons in CrSBr, with potential applications in quantum devices leveraging ultrathin van der Waals magnets.