Defect detection in primary composite lightweight structures is a major ongoing challenge in terms of reliability, rapidity, and accuracy for a secure operational life-cycle. Shearography is a full-field and material independent non-destructive testing method. Despite its major suitability for large composite components, this method still requires specialists to reliably identify the defect patterns. Modern algorithms in terms of machine learning have gained huge popularity and provide the ability to outperform conventional algorithms, especially in image analysis. Hence, an object detection model based on convolutional neural networks has been implemented and applied to shearographic images of different composite specimens. Concluding, the model performs with a considerable high accuracy considering the medium sized and manually labeled dataset.