The formation of social first impressions from voices is a central component of everyday social interactions. While past research has primarily investigated the effect of bottom-up voice acoustic on social voice judgements, here we widen the perspective and investigate how bottom-up acoustic and top-down interpersonal similarity interactively influence social voice judgements. We quantified acoustic distance to an average voice as a measure of acoustic typicality and combined this with interpersonal similarity between each listener-speaker pair derived from Big Five personality traits. 110 listeners rated 30 words spoken by female speakers on attractiveness, likability, trustworthiness, and dominance, and data were analysed using Generalised Additive Mixed Models to capture nonlinear and interactive effects. Across social dimensions, acoustic typicality emerged as the strongest predictor of social judgements, with more typical voices receiving the most positive ratings. However, social judgements were not determined by acoustic cues alone. For attractiveness, likability and trustworthiness, similar personalities between speaker and listener enhanced social evaluations, particularly for acoustically atypical voices. In contrast, dominance judgements were exclusively explained by acoustic typicality without significant modulation by interpersonal similarity. Together, these results show that most social voice judgements arises from a dyadic, context-sensitive integration of acoustic information and interpersonal similarity.