The performance and behaviour of critical infrastructure in case of disruptions is an important topic and we are still lacking of insights. Due to disruptions, infrastructure becomes unavailable and may force the trains and passengers to adapt. In this paper, we introduce a problem of railway network vulnerability from the perspective of passenger flows and train operations. We propose a new Railway Network Vulnerability Model (RNVM) to assess the vulnerability of the system by finding the critical combination of links, which cause the most adverse consequences to passengers and trains. To solve this challenging problem, we present a RNVM framework, which combines two heuristics based on column and row generation with mixed integer linear programming, to efficiently model alternative passenger flows and infrastructure constraints. The developed framework provides the critical combination of links, the corresponding passenger flows, train routes and timetables. We demonstrate the performance of the RNVM framework on the real-world instance of a part of the Dutch railway network. The results show that the RNVM framework can efficiently reassign passenger flows and reroute trains during disruptions. The results also reveal that the critical links are highly demand dependent rather than a static feature of the networks topology. Finally, the computation times remain small when increasing the number of disrupted links as well as the size of the passenger demand, which allows fast and efficient network vulnerability assessment.