The chronic intake of 3,4-methylenedioxymethamphetamine (MDMA, “ecstasy”) bears a strong risk for sustained declarative memory impairments. Although such memory deficits have been repeatedly reported, their neurofunctional origin remains elusive. Therefore, we here investigate the neuronal basis of altered declarative memory in recurrent MDMA users at the level of brain connectivity. We examined a group of 44 chronic MDMA users and 41 demographically matched controls. Declarative memory performance was assessed by the Rey Auditory Verbal Learning Test and a visual associative learning test. To uncover alterations in the whole brain connectome between groups, we employed a data-driven multi-voxel pattern analysis (MVPA) approach on participants' resting-state functional magnetic resonance imaging data. Recent MDMA use was confirmed by hair analyses. MDMA users showed lower performance in delayed recall across tasks compared to well-matched controls with moderate-to-strong effect sizes. MVPA revealed a large cluster located in the left postcentral gyrus of global connectivity differences between groups. Post hoc seed-based connectivity analyses with this cluster unraveled hypoconnectivity to temporal areas belonging to the auditory network and hyperconnectivity to dorsal parietal regions belonging to the dorsal attention network in MDMA users. Seed-based connectivity strength was associated with verbal memory performance in the whole sample as well as with MDMA intake patterns in the user group. Our findings suggest that functional underpinnings of MDMA-related memory impairments encompass altered patterns of multimodal sensory integration within auditory processing regions to a functional heteromodal connector hub, the left postcentral gyrus. In addition, hyperconnectivity in regions of a cognitive control network might indicate compensation for degraded sensory processing.