The clinch joining process is simulated for 22 different tool- and material-combinations, using a modular axisymmetric finite element simulation model. Two ductile metals are considered for the sheets, namely the dual-phase steel HCT590X and the aluminium alloy EN AW-6014 T4. A finite elasto-plastic material model is utilised to capture the inherent large plastic strains. Moreover, it is coupled to stress-state-dependent ductile damage and failure to successfully predict possible fracture during the clinch joining process. For all 22 clinch combinations a good agreement is obtained between simulations and experiments, regarding the geometry of the clinch joint, the process force and the occurrence of material failure. This represents a significant advance in the development and comprehension of a versatile process chain resulting from joint research efforts. The validated process simulations are then applied to study the influence of the tool geometries, sheet pre-stretch, and friction. Failure is herein always observed by neck fracture. Nevertheless, detailed analyses of the stress state evolution during the joining process for various locations reveal that the material is exposed to distinctly non-proportional loading paths demanding suitable stress-state-dependent evolution laws. Moreover, even for valid joints, process-induced damage is distributed throughout the joint. Incorporating the damage-induced softening causes an accelerated failure evolution, but has less influence on the global behaviour.