For decades, person perception research has grappled with the distinction between the targets’ actual characteristics (“substance”) and how positively or negatively those characteristics are viewed by perceivers (“evaluation”); however, lack of an overarching theoretical framework makes it difficult to establish connections between related lines of research. We review the relevant literature, and present and test an algebraic model that incorporates the major insights from that literature. The model posits that all person judgments reflect substance and evaluation to different extents. The evaluation component reflects an interaction between the item’s evaluative tone and the perceiver’s evaluative attitude regarding the target person. The model may function as an integrative framework that helps improve conceptual clarity and cumulative progress in person perception research.