The delineation of malignant brain tumors from surrounding tissue is crucial for maximal safe resection. Existing tools for intraoperative tumor visualization lack the possibility of real-time tissue classification. We therefore built a coherent fiber bundle (CFB) endoscope for autofluorescence imaging and trained a convolutional neural network (CNN) to distinguish tumor tissue (T) and non-tumor tissue (nT) in an ex vivo proof of concept study. The T-cohort included frozen tissue sections of 35 glioblastoma and 30 brain metastasis patients (10 breast cancer, 10 melanoma, 10 lung cancer). The nT-cohort comprised frozen tissue sections of hippocampi and temporal lobes from 23 patients undergoing surgery for epilepsy treatment. For image acquisition, a laser with excitation wavelength of 473 nm was coupled into a CFB and tissue autofluorescence between 500-550 nm was collected using a bandpass filter. The acquisition time of the camera was set to 700 ms. A CNN (VGG-19) was trained to differentiate the labels T from nT wherein the ground truth was defined by histopathological diagnosis. Training, validation and test sets were randomly generated across all patients in 66.6% / 16.6% / 16.6% proportion. Set generation was repeated 5 times and reported results for the independent test sets are mean values of all iterations. In total, 3259 CFB images were acquired (T:2408, nT:851), with usually 40 CFB images per sample. Removal of CFB background was not improving classification accuracy and thus, was not performed. The classification accuracy of the CNN with a threshold probability of 0.5 for T-label assignment was 0.986 (sensitivity: 0.988, specificity: 0.982) for analysis of images and 0.98 (sensitivity: 0.971, specificity: 1) when calculating the result for each patient. To simulate clinical application where correct identification of healthy tissue is essential, we created a specificity-tuned CNN with adjusted threshold probability of 0.9 for T-label assignment. This led to a decreased accuracy of 0.945 but increased specificity on the image evaluation approach to 1 (sensitivity: 0.92). Autofluorescence imaging through a CFB is capable of ex vivo recognition of brain tumor tissue with a CNN. The simplicity of the setup and minimal need for data pre-processing allows for intraoperative implementation and potentially enables real-time in vivo brain tumor delineation. To realize intraoperative transition, analysis of ex vivo bulk tissue will be upcoming to capture the signal complexity caused by tissue depth.