A New Tool for Automated Quality Control of Environmental Data in Open Web Services

EasyChair Preprint 1325

Abstract

We report on the development of a new software tool (auto-qc) for automated quality control (QC) of environmental timeseries data. Novel features of this tool include a flexible Python software architecture, which makes it easy for users to configure the sequence of tests as well as their statistical parameters, and a statistical concept to assign each value a probability of being a correct value. There are many occasions when it is necessary to inspect the quality of environmental datasets, from first quality checks during real-time sampling and data transmission to assessing the quality of long-term monitoring data from measurement stations. Erroneous data can have a substantial impact on the statistical data analysis and, for example, lead to wrong estimates of trends. Existing QC workflows largely rely on individual investigator knowledge and have often been constructed from practical considerations alone. Our tool aims to complement traditional data quality analyses and adds some insights into the nature of the individual tests that are being applied.

Keyphrases: Auto-qc tool, TOAR database, air quality, automated quality control, autoqc4env tool, quality control, software framework, time series

@booklet{EasyChair:1325,
  author    = {Najmeh Kaffashzadeh and Felix Kleinert and Martin G Schultz},
  title     = {A New Tool for Automated Quality Control of Environmental Data in Open Web Services},
  howpublished = {EasyChair Preprint 1325},
  year      = {EasyChair, 2019}}