Measuring runoff formation on soil surfaces by rainfall simulators predominantly provide lumped values without spatiotemporal information in regard to water dynamics (e.g., water ponding timing and connectivity). Understating spatial and temporal variations of water storage on soil surface is key to assess hydrological connectivity and runoff generation. Furthermore, it is very relevant for erosion studies. Computer vision and deep learning has presented state-of-the-art results in environmental sciences, for instance to segment water using cameras as gauges or performing flood mapping with remote sensing images. However, automatic mapping of water forming on soil surfaces due to rainfall is very challenging because the water area is considerably smaller and water ponds present complex shapes and similar color characteristics to the soil itself, which is a challenge for deep learning models. The aim of this study is to assess the potential of computer vision and deep learning to estimate water ponding timing, connectivity and runoff formation behavior during rainfall simulations, with emphasis on data imbalance and label uncertainty.We conducted rainfall simulations at three different soil erosion plots with different soil and tillage caracteristics. Runoff was measured at the plot outlet. We collected time lapse images from the plot surface. And ground control points for model scaling were measured with a total station. To train the deep learning models, we manually labeled a selected set of images from all the plot images to derive binary masks (i.e., water and background). We trained three different convolution neural networks (CNN) and further considered techniques that take class imbalance and label uncertainty into account. Eventually, we assess the performance of ensemble models. We applied the best model on the whole set of time lapse images and measured the water pixel area and pond connectivity in terms of connected components. Our findings suggest that considering class imbalance and label uncertainty is key to reach satisfactory segmentation performance, being more important than the model architecture. Furthermore, ensemble models result in better performance when compared to single models. By comparing the measured discharge and the water area derived from the best deep learning model, we can observe different characteristics of the runoff formation related to distinct ponding and intensity of ponding and connectivity. Our approach presents an innovative visual and automatic observation option to quantify the water pond formation and its spatial temporal development. It is a step towards a better understanding of the runoff generation.