Recognizing others’ affective states is essential for successful social interactions. Alexithymia, characterized by difficulties in identifying and describing one’s own emotions, has been linked to deficits in recognizing emotions and mental states in others. To investigate how neural correlates of affective state recognition are affected by different facets of alexithymia, we conducted a functional magnetic resonance imaging study with 53 healthy participants (aged 19–36 years, 51% female) using the Reading the Mind in the Eyes Test (RMET) and three different measures of alexithymia [Toronto Structured Interview for Alexithymia (TSIA), Toronto Alexithymia Scale (TAS-20), and Bermond–Vorst Alexithymia Questionnaire]. In addition, we examined brain activity during the RMET and replicated previous findings with task-related brain activation in the inferior frontal and temporal gyri, as well as the insula. No association was found between alexithymia and behavioral performance in the RMET, possibly due to the low number of participants with high alexithymia levels. Region of interest based analyses revealed no associations between alexithymia and amygdala or insula activity during the RMET. At the whole-brain level, both a composite alexithymia score and the unique variance of the alexithymia interview (TSIA) were associated with greater activity in visual processing areas during the RMET. This may indicate that affective state recognition performance in alexithymia relies on a higher compensatory activation in visual areas.