Measurement quality assurance

Measurement quality assurance prepares the raw data for further analysis. Data filtering, gap filling, and homogenisation add significant value to raw measurement data.

meteoblue provides measurement quality assurance based on the measurements available via the API. meteoblue distinguishes between three different use cases: single station, multiple stations and city network. The quality assurance is provided for the following weather variables:

  • Air temperature
  • Precipitation
  • Wind speed
  • Wind direction
  • Solar radiation
  • Relative humidity

The measurement quality assurance allows for the tracking of data transmission through real-time alerts in case the stations are not sending information. Moreover, we create monthly reports and meteorological statistics for climate indices and station coverage. In addition, the system provides data normalisation in equally distributed timestamps, and offers functions such as station quality control, gap filling, and data homogenisation. The availability of these functions depends on the use case and the meteorological variable.

Equally distributed timestamp

Different station providers can record the measurements with different temporal resolution. To compare the measurements of different station providers, it is necessary to compare the same timestamp. Additionally, the stations typically do not record exactly during the full or half hour. Therefore, the timestamping tool interpolates towards the full and half hours. The user can decide on the temporal resolution of the dataset. If the selected temporal resolution is smaller than the temporal resolution of the station provider, the values in between the regular temporal resolution are filled with NaN (Not a Number).

Station quality control

Quality control is conducted as a three-step process:

Screening

Metadata of the stations are checked for correctness. If the latitude or longitude of the station is located outside the natural coordinates of the Earth, the station data is submitted to a correction process. Additionally, the data coverage is calculated, based on the ratio of the relevant values (i.e., those which are not equal to NaN).

Static quality control for each location separately

Several different filters are applied to the time series of the individual station. These filters are typically different for different variables.

QC1 filter for temperature Description
NaN check Detect NaNs
Logical check Detect illogical values that are too high or too low
Static check Detect outliers outside of thresholds
Flatliner check Detect flatliners where consecutive values are the same
Blip check Detect sudden temperature changes above threshold
Spatial check Detect spacially illogical values

Dynamic quality control

Dynamic quality control is used for complex comparisons between different stations (only applies for the use case of city network, see above). The spatial consistency of the stations is verified by applying neighbouring checks, and comparing stations within a specified radius around the station.

Gap filling

Three different approaches are utilised for the gap filling.

  1. Linear interpolation
    • If the gap is only one timestamp
  2. Use another station
    • If the gap is larger than one timestep
    • Find the station with the best correlation based on historical measurements
    • Use the best correlated station and apply bias correction (to account for a potential bias between the two stations)
  3. Use model data
    • If the gap is larger than one timestep
    • Use model data from the meteoblue model chain which best fits the station (selection of up to 20 different numerical weather prediction (NWP) models depending on the location)

Data homogenisation

The meteoblue data homogenisation tool contains several functionalities for a variety of use cases.

For temperature measurements, two different approaches (bias correction and radiation correction) complement each other and are available based on different assumptions. In case a low-budget city sensor network is used, a bias correction can be applied in the operational and historical use case if at least one low-budget sensor is installed close to a standard WMO station (e.g., METAR, MeteoSwiss, etc.). The bias correction calculates in real-time the difference between the low-budget sensor and the WMO station, and corrects this difference for all the sensors in the city network with an update frequency of the WMO measurement. This bias correction is applied to all variables available in the city network. This method accounts for systemic discrepancies between the WMO station and the city network, e.g., errors due to heat accumulation or systematic errors occurring during nights.

For the use case “historical city network” a scientific radiation correction is applied to ensure the best possible quality of historical weather data. Note that due to the complexity of this radiation correction, it is not possible to provide this method in operational usage.

Therefore, results in operation differ from results in the historical data package, where typically the highest quality is reached.

The scientific radiation correction is based on our special routines published in a scientific research paper, and includes separate corrections for shaded and sunny conditions. Direct shortwave radiation, diffuse shortwave radiation, air temperature, wind speed, and the solar elevation angle are used as input variables for the radiation correction.

Comparison of the air temperature of three different sensors 
(Metos LoRain, Sensirion, WMO reference) at Schimmelstrasse, Zurich in June 2020. Comparison of the air temperature between the WMO reference, 
the uncorrected Metos LoRain and the corrected Metos LoRain sensor.