The accurate detection and assessment of wood surface defects in historic timber structures, particularly knots, is essential for effective conservation and strengthening planning. However, the application of automated visual grading methods to aged timber remains underexplored due to the irregular texture, weathering and lack of relevant datasets. In this study, we propose an end-to-end deep learning-based pipeline that integrates wood surface segmentation, perspective correction, and knot detection to estimate structural grading factors. A dedicated raw data collection of over 10,000 high-resolution images of historic timber surfaces was compiled using both DSLR cameras and mobile devices, resulting in multiple datasets with approximately 3,000 annotated samples. Three model families, YOLO, Detectron2 and DeepLabV3, were evaluated under different experimental setups. Beyond model benchmarking, we further compared the AI-derived results with expert manual measurements. The model for segmentation of timber surface achieved a mean IoU of over 0.85 and the model for detection of historical wood knots reached F1 scores of up to 0.9. The structural assessment factors estimated by the AI pipeline achieved a Pearson correlation coefficient of 0.641 compared to manual measurements, indicating a moderate level of consistency in knot factor estimation. This research highlights the potential of vision-based AI systems in supporting structural diagnosis and conservation of heritage timber elements.