The classification of brain tissue plays a key role in brain tumor diagnosis and treatment. It revolves around post-surgical histochemical staining, which is often time consuming and delays follow up treatment. Identifying tumor boarders during tumor resection is essential for an efficient therapy minimizing removed healthy tissue and maximizing removed tumor tissue. The different approaches in use are either expensive and time consuming or limited to certain tumor types. We propose a real-time in vivo label free classification approach, applicable for both demands. Based on autofluorescence properties, a label-free differentiation between tissue types is possible. Therefore, a multicore fiber (MCF) based endoscope is designed to fit into biopsy needles used during diagnosis and to be used as a handheld probe during tumor resection. It allows illuminating and imaging through the same MCF, minimizing the endoscope to a submillimeter diameter. Currently, autofluorescence images are not used in pathology. Thus, medical doctors cannot interpret them. We use a neural network for diagnosis, bridging this gap. One problem with neural networks in medical applications is data availability for training. Different techniques are investigated to maximize the classification performance with a limited training dataset. Cascaded neural networks in combination with digital twins improve the results while lowering the needed training dataset size. The preliminary data indicates that our technology might lead to a paradigm shift in brain tumor diagnosis and therapy due to the accurate result, the versatile design, and being low-cost.