Quantifying Uncertainty in Runoff Simulation According to Multiple Evaluation Metrics and Varying Calibration Data Length
Therefore, this study quantified the uncertainty inherent in the use of different evaluation metrics and different calibration data lengths. The evaluation metrics used in this study are NSE, KGE, Pbias, NRMSE and Jensen–Shannon divergence, and the data lengths for calibration are 1, 2, …, 11 years. Using all calibrated parameter sets, the uncertainty was quantified using the simulated and observed runoff data for the validation period. The Soil and Water Assessment Tool (SWAT), QSWAT3 v1.6.5 was used and the parameters were calibrated using R-SWAT.
3.1. Model Performance over the Calibration Period
The IQR, which represents the model uncertainty in the calibration period, was the largest for P1 before and after calibration, at 0.122 and 0.100, and the smallest for P11, at 0.030 and 0.024. The shorter the calibration data length, the higher the model performance but the higher the inherent uncertainty of each independent period. The average IQR for the NSE, including all calibration data lengths, decreased from 0.085 to 0.069 after calibration, and showed a decrease in uncertainty.
3.2. Evaluation of Performance over Validation Period
The average value of the absolute Pbias was the lowest in P6 (2.84) and the highest in P8 (4.36), with the highest uncertainty in P8. The IQR of Pbias was the highest in P1 (4.03) and the lowest in P2 and P5 (2.00). NRMSE was the lowest for P5 (51.08), and the IQR was the highest for P1 (3.73). Considering the average, the uncertainty in the NRMSE was the largest for P1 (52.52).
Overall, the uncertainty based on the average value of each evaluation metric varied depending on the evaluation metric. NSE and NRMSE have higher uncertainties for shorter calibration data lengths, while KGE and Pbias have higher uncertainties for longer calibration data lengths. Consistently, P5 to P7 have lower uncertainties. The uncertainty based on IQR values was found to be the highest for P1 for all evaluation metrics, while P7 was evaluated as having relatively low uncertainty.
3.3. Uncertainty Index
3.4. Evaluation of the Extreme Runoff
3.5. Overall Uncertainty Assessment
This study has immediate applications in policy decisions and water management practices. Water resource managers and policymakers can employ the insights gained to optimize calibration lengths and evaluation metrics, thus enhancing model reliability. The methodological approach of using multiple evaluation metrics to quantify uncertainty represents a significant advancement in hydrological studies. Moreover, the results are particularly useful for locations where data may be scarce or incomplete, as demonstrated by the model’s performance despite missing data for 2011. However, there are some limitations that should be highlighted. While this study provides specific insights into the Yeongsan River basin, the methodology and findings offer broader implications for hydrological modeling. The approach to determining the optimal calibration data length, based on a balance between reducing uncertainty and the practicality of data availability, can be applied to other river basins. However, it is important to note that the specific optimal calibration period may vary depending on several factors, including the hydrological characteristics of the basin, the variability of meteorological conditions, and the quality and quantity of available data. Therefore, while our study findings suggest a general approach to identifying an optimal calibration data length, this study recommends that hydrologists and modelers conduct similar analyses tailored to their specific river basins. Such analyses should consider local hydrological dynamics and data characteristics to determine the most appropriate calibration period for their models.
The uncertainty of runoff simulations using climate data and a hydrological model in the Yeongsan River Basin located in southwest South Korea was quantified. The uncertainty of the runoff simulations was considered based on the calibration data length and the selection of the evaluation metrics. To quantify the uncertainty of the runoff simulation, and the extreme runoff (95th percentile flow), the difference in performance according to the calibration data length, and the difference in performance according to the validation period were quantified. Extreme runoff was evaluated using JS-D to determine the difference in the distribution from the observed data, and NSE, KGE, Pbias, and NRMSE were applied as the evaluation metrics. Based on the results, the following conclusions can be drawn:
Different evaluation metrics all showed different levels of uncertainty, which means it is necessary to consider multiple evaluation metrics rather than relying on any one single metric;
Runoff simulations using a hydrological model had the least uncertainty owing to the calibration data length when using a parameter set of seven years, and the uncertainty increased for calibration data lengths longer than seven years;
Parameter sets with the same calibration length showed period-dependent uncertainty, which led to uncertainty differences within the same length;
For extreme runoff simulations, employing long calibration data lengths (of more than seven years) achieved lower uncertainty than shorter calibration data lengths.
In the end, this study contributes to the broader knowledge base by providing a framework for assessing the optimal calibration data length in hydrological modeling. This framework can be adapted and applied to other river basins, with the understanding that local conditions and data availability will influence the specific outcomes.
Disasters Expo USA, is proud to be supported by Inergency for their next upcoming edition on March 6th & 7th 2024!
The leading event mitigating the world’s most costly disasters is returning to the Miami Beach
And in case you missed it, here is our ultimate road trip playlist is the perfect mix of podcasts, and hidden gems that will keep you energized for the entire journey-