The use of ensemble forecast for extreme weather events prediction, or for hydrologic forecasting, requires quantifying the degree of performance and reliability through evaluation. The inherent uncertainty evident in numeric weather predictions imposes a problem in terms of the degree of trust in the results of the ensemble modelling, which makes verification an action of extreme importance as it provides a deeper understanding of the overall predictive skill of the ensemble forecast. To our knowledge, evaluating the usage of ensemble forecasts as a tool for early warning in small catchments with fast hydrological processes was limited. For this reason, this research is particularly focusing on investigating the performance of the ensemble weather forecast provided by the Deutscher Wetterdienst (DWD) in small catchments, taking ten catchments distributed among three regions in Saxony, Germany as test areas.
For the purpose of this research, a computational tool designated for the structure and format of the forecast products was developed and utilized for the analysis of both rainfall and runoff ensemble forecasts. The tool encompasses a set of different evaluation metrics including contingency table-based metrics, that examine the forecast from extreme event point of view (e.g. false and true alarm rates, accuracy, area under the ROC curve), in addition to metrics that treat the entire forecast ensemble as a probability distribution and measure to what degree it conforms with the ground truth (e.g. CRPS). The carried-out evaluation exercise was based on comparing numerically modelled forecasts to actual measured observations. The evaluation was done once for the meteorological forecasts, ICOND2EPS/COSMOD2EPS against RADOLAN-RW radar observations, and for the hydrological forecasts, modelled against gauged catchment runoff measurements.
The results show, after examining the performance of the ensemble predictions from different perspectives, e.g. accuracy and reliability, that using ensemble forecasting in small catchments is considered to perform to a good degree of satisfaction. Although the performance in the central region of Saxony was noticed to be slightly better than the two other test regions. The results also showed the tendency of the ensemble average to overestimate the observations for both the rainfall and runoff forecasts, however, that is more related to the mathematical approach to the analysis. And finally, the collective outcomes of this research provide an insight on how ensemble forecasts could be utilized for decision making in each catchment on its own.