The freshwater resource in karst is subjected to both sea level rise and an increasing pressure caused by the high-water demand. Therefore, developing an understanding of the hydrogeological dynamics of the karst aquifer can be a useful tool for improving protection and management actions. Vora Bosco cave (Apulia, Southern Italy) was instrumented with a multi-parameter probe for groundwater level measurements, aimed at exploring the system behavior within the cave recharge area. To characterize and quantify the natural recharge and discharge behavior of the system, a simple reservoir model was developed, initially also with the intention of predicting groundwater dynamics. Based on the original time-series of water level observations, different calibration datasets were established using different split-sample and bootstrapping approaches, and a regional sensitivity analysis was executed. Furthermore, in addition to the original observation time-series, a 3-month extension was used as a model testing period. Using these analyses, the parameters identifiability and the predictions robustness for the model testing period were evaluated. Results reveal that while the calibration on the whole dataset, as well as the bootstrapping approaches, lead to better performances in the calibration and validation period of the original time-series, and to a higher model precision with smaller uncertainty ranges. their performance in the model testing period becomes very poor and the observed water level data no longer plots within the uncertainty bands. Based on this extensive analysis, the model is finally rejected. Our study therefore also confirms the importance of model validation, especially when only a short time-series of observations are available.