When dealing with modelling and simulation of surface cleaning processes knowledge of the specific cleaning mechanism at hand is necessary to apply the correct cleaning model. This paper presents an approach for the identification of cleaning mechanism based on fully convolutional networks. An efficient labelling strategy based on gradient filters is developed, allowing fast and efficient labelling of cleaning experiment video footage while achieving pixel-perfect labels. First, datasets are generated for model soils (starch 12410, ketchup, petroleum jelly) as well as standard soils (eggyolk, vanilla pudding, gelatine, starch 12616). A subset of the model soil dataset is used for training machine learning algorithms based on fully convolutional networks. The remaining data is subsequently employed as unseen data to assess the performance of the models. Independent testing on the model soils data achieves 93% accuracy and 82% Intersection over Union. With these values, selection of the correct cleaning model is possible. Application of the models to standard soils shows the ability of the models to generalize towards more realistic soils without further training or adaption. Finally, the models are applied to 89 experiments with ketchup under various operating conditions. The results were combined to construct a regime map for cleaning mechanisms constituting a new way to efficiently display the effect of different operating conditions on the cleaning behaviour of soils for the practitioner.