It is essential to assess the rolling contact fatigue (RCF) of turnouts and maintain them in advance. It saves a lot of money while protecting the safety of railway operations. In Germany, the damage on rails, especially crossing noses, mainly depends on the subjective judgment of experts. There are no objective and comprehensive evaluation criteria. This paper presents the application of image processing and supervised machine learning algorithms to crossing nose fatigue judgment. The fatigue characteristics of the crossing nose rolling contact surface along the life cycle of the crossing nose are analyzed. The study used crack information from magnetic particle inspection (MPI) images of crossing nose surfaces. It uses basic image processing methods to collect physical information about features of fatigue cracks in images. Existing feature selection methods are used to exclude irrelevant features and retain valuable features. And we select the best feature selection method through the regression results. Statistically significant crack features and combinations that depict the surface fatigue state are found. In this paper, by comparing several usually machine learning regression algorithms, it is found that the supervised learning of support vector machine regression (SVR) has achieved the best results in the regression fitting of the crack feature data in this paper. The regression results form a simple system to evaluate the life cycle of crossing nose. The system finds the location of cracks that can create dangerous defects in the crossing nose surface. The research result consists of the early prediction of rail contact fatigue.