- The meteoblue precipitation forecast is more accurate than the post-processed reanalysis model ERA5 for normal intensities.
- In the provider comparison of 2021, the meteoblue precipitation forecast had the highest HSS (Heidke Skill Score).
- Superior model performance is due to post-processing methods used for the meteoblue precipitation forecast, such as multimodel-mixing.
- Finer horizontal model resolution does not improve the model performance: NEMSGLOBAL (30 km) vs. with NEMS12 (12 km) and NEMS4 (4 km).
- The error metrics of the validation studies conducted in different years are very similar when comparing models with measurement data (compare verification study 2017 and verification study 2019).
- Error metrics could vary significantly for different continents or smaller regions.
- We recommend the use of the meteoblue MultiModel for the operational forecast, because daily precipitation events as well as annual precipitation sums are best reproduced.
- Results between different studies might vary due to a different number of stations, different time periods analysed or different measurement data providers.
Forecast | Daily precipitation > 1mm |
Daily precipitation > 10mm |
Daily precipitation > 50mm |
---|---|---|---|
meteoblue MultiModel | 0.47 | 0.36 | 0.14 |
ERA5 | 0.45 | 0.35 | 0.14 |
NEMS | 0.42 | 0.30 | 0.13 |
GFS | 0.42 | 0.30 | 0.12 |
CHIRPS2 | 0.30 | 0.31 | 0.19 |
Simulations of precipitation event and volume as well as actual precipitation volumes are very important variables in meteorology, that simultaneously show a highly variable occurrence over small distances, due to their complex physical nature, and are therefore amongst the most difficult variables to simulate with high accuracy.
In this section, we give an overview of the performance of our precipitation forecast not only compared to several numerical weather prediction models, but also compared with other weather providers. Furthermore, we want to ensure that our users are aware of some of the most important accuracy considerations related to precipitation forecasts, to be able to interpret the data and potential shortcomings correctly.
Verification Studies
The forecast of precipitation in numerical weather forecast models is increasingly gaining economic importance for various applications such as irrigation, agriculture, hydropower generation or flood warning. These numerical weather forecast models have been continuously improved in the last decades.
Within our verification studies, we analyze the model performance of the precipitation by using several raw numerical weather forecast models and the meteoblue MultiModel forecast. Model results were compared with precipitation measurements from up to 10’000 meteorological stations from all over the world. Additionally, we conducted a competition analysis, in which we compared our forecast accuracy with the forecast accuracy of other weather providers. meteoblue conducts these unique and extensive long-term verification studies to understand the quality of the precipitation data, produced by our own models as well as by various third-party operators.
Throughout the studies, we focus not only on the precipitation sums, but also on the model performance of predicting single precipitation events (e.g. daily precipitation > 1 mm) which is often more important than the correct absolute values of the annual precipitation (e.g. in the agricultural sector).
Study | Number of stations | Model domain | Time period | Measurements | Model | HSS Daily Precipitation > 1mm | MAE [mm] | MBE [mm] |
---|---|---|---|---|---|---|---|---|
Verification study 2017 | 6505 | worldwide | 2017 | METAR | ERA5 | 0.45 | 175 | - 7 |
NEMSGLOBAL | 0.42 | 228 | - 100 | |||||
meteoblue forecast | 0.47 | 161 | - 8 | |||||
GFS | 0.42 | 235 | + 62 | |||||
CHIRPS2 | 0.30 | 120 | + 33 | |||||
Verification study 2019 | 8112 | worldwide | 2019 | METAR | ERA5 | 0.45 | 234 | - 6 |
GFS | 0.43 | 270 | 84 | |||||
NEMSGLOBAL | 0.41 | 296 | - 144 | |||||
ICON | 0.46 | 253 | - 51 | |||||
Competition analysis | 100 | North America | Jan-Jul 2021 | METAR | meteoblue forecast | 0.61 | 42 | + 33 |
500 | worldwide | GSOD | meteoblue forecast | 0.44 | 135 | + 7 | ||
NEMS verification | 1605 | Europe | Jan-Jul 2021 | GSOD | NEMS4 | 0.41 | 119 | - 64 |
NEMS12 | 0.41 | 85 | - 36 |
In all verification studies, plausibility tests for the measurements have been conducted ensuring the quality of the measurements. Measurement data with more than 30% gaps were excluded from the analysis to ensure robust results. Additionally, a quality control for the measurements was applied to exclude erroneous measurements.
To learn more about the accuracy of our precipitation forecast and its limitations, choose one of the following links.
Find out more about our verification studies based on precipitation events.
Find out more about our verification studies based on precipitation amount.
Accuracy Considerations
As we are continuously working on improving our weather forecast simulations, it is also important to explain the most important accuracy issues related to precipitation forecasts, in order to make sure our users are able to correctly interpret the data and potential shortcomings.
Precipitation is one of the most important parameters of a weather forecast, but also one of the most difficult to predict. Verification typically measures the skill of a forecast system in predicting events, occurring when a certain volume of precipitation exceeds a threshold in a specified time period such as 24 hours. Furthermore, accumulated volumes of precipitation are compared to observations, which is typically done for longer time periods, e.g., one year.
In general, it is important to differentiate between precipitation volume accuracy and precipitation event accuracy, with sometimes diverging results. When accuracy measures are evaluated, please think about your priorities first: is a precipitation event the most important criteria (e.g., is it important whether tomorrow is totally dry or sees some rainfall) or the precipitation volume (e.g., is it important whether it rains 10 or 30 mm in the next 2 days). To get further information about accuracy limitations, please check the following links:
Technical characteristics of weather models and observation devices can cause limitations for the forecast.
Models can perform better or worse in certain regions of the world, where weather conditions can vary strongly.
Sometimes inaccuracies through display or data aggregation approaches cannot be avoided.