In industry, production is often organized in the form of a hybrid flow shop, and there is great interest in methods and algorithms for optimizing such production processes. While, thus far, methods have focused mostly on optimizing a single selected objective, it is increasingly important to address several objectives simultaneously in order to move from extreme to balanced solutions that consider diverse operational requirements. Following this, we classify and characterize the literature dealing with multi-objective hybrid flow shop scheduling problems (HFSP). We identify those features in metaheuristics that require particular attention during the process of finding Pareto solutions for HFSP (especially coding and decoding schemes, Pareto archives, and Pareto dominance concepts). To promote the evaluation of the suitability of algorithms for solving multi-criteria HFSP, we provide an overview of the test instances used in the literature and propose a systematization of performance criteria for the evaluation of Pareto fronts in order to create clear and consistent conceptual and semantic understanding. Based on this, recommendations are derived that can also be helpful for various multi-objective optimization problems and other application contexts for assessing solution quality as accurately and comparably as possible. Finally, current challenges and possible future research directions are highlighted.