In-process monitoring in milling, specifically tool condition monitoring (TCM), is an important technology for improving productivity and workpiece quality. However, industrial implementation of in-process TCMs remains a difficult task, since progressing tool wear is indicated by small changes of various physical parameters. Therefore, a sensitive monitoring system is needed to provide a reliable base of information while having minimal impact on the machine tool system and processes. Recent advancements in deep learning (DL) techniques are frequently applied on monitoring data for tool wear prediction as they can process and analyse raw data without prior feature engineering. This paper presents a suitable monitoring approach based on a recently developed sensory tool holder, which measures cutting forces and vibrations in direct proximity to the process zone. The system is equipped with wireless data transmission and a novel energy harvesting technology for energy supply. Two milling experiments with focus on increasing tool wear were conducted and the collected data processed. A DL based model, comprising three convolutional neural network (CNN) layers, one long short-term memory (LSTM) layer, and a multi-layer perceptron (MLP), was trained on the raw sensor signals to make predictions on the tool wear state. The model was evaluated using previously unseen test data and achieved a high prediction accuracy of at least 97,3% for all sensor signals, with the highest accuracy of 99,9% achieved when using bending moment signals.