Aggressive, violent behaviour is a major burden and challenge for society. It has been linked to deficits in social understanding, but the evidence is inconsistent and the specifics of such deficits are unclear. Here, we investigated affective (empathy) and cognitive (Theory of Mind) routes to understanding other people in aggressive individuals. Twenty-nine men with a history of legally relevant aggressive behaviour (i.e. serious assault) and 32 control participants were tested using a social video task (EmpaToM) that differentiates empathy and Theory of Mind and completed questionnaires on aggression and alexithymia. Aggressive participants showed reduced empathic responses to emotional videos of others’ suffering, which correlated with aggression severity. Theory of Mind performance, in contrast, was intact. A mediation analysis revealed that reduced empathy in aggressive men was mediated by alexithymia. These findings stress the importance of distinguishing between socio-affective and socio-cognitive deficits for understanding aggressive behaviour and thereby contribute to the development of more efficient treatments.