Quantitative phase imaging (QPI) through multi-core fibers (MCFs) has been an emerging in vivo label-free endoscopic imaging modality with minimal invasiveness. However, the computational demands of conventional iterative phase retrieval algorithms have limited their real-time imaging potential. We demonstrate a learning-based MCF phase imaging method that significantly reduced the phase reconstruction time to 5.5 ms, enabling video-rate imaging at 181 fps. Moreover, we introduce an innovative optical system that automatically generated the first, to the best of our knowledge, open-source dataset tailored for MCF phase imaging, comprising 50,176 paired speckles and phase images. Our trained deep neural network (DNN) demonstrates a robust phase reconstruction performance in experiments with a mean fidelity of up to 99.8%. Such an efficient fiber phase imaging approach can broaden the applications of QPI in hard-to-reach areas.