WMO Global Cryosphere Watch (GCW) is depending on a number of observing stations (CryoNet stations) for feeding the GCW value chain with observations. GCW has a requirement for both real time and archived data. In the period 2015-2017, GCW has been working with WSL/SLF to set up interoperability between the WSL/SLF data centre being responsible for one of the CryoNet stations. WSL/SLF has kindly agreed to make the software stack they have developed available for a wider community.
The following is a brief description of several software tool used at the WSL Institute for Snow and Avalanche Research (SLF) to processes and manage data at various stages of the “datacycle” from sensors to published dataset. All projects are available under open source licenses.
The software package is designed to support the value chain provided below.
The GCW/SLF software package consists of several modules. The purpose of these modules is listed below and how the components work together to support the purpose of the GCW portal is illustrated below.
The core element in the software package is the data preprocessor MeteoIO that takes data from the sensor, through a quality control procedure into standardised NetCDF/CF files which can be published.
MeteoIO was originally developed to provide robust meteorological forcing data to an operational model that forms part of the avalanche forecast at the SLF. However, it also happens to be very good at reading diverse data sources and producing a standardised output. It has a modular architecture which makes it flexible and fast to develop new use cases. It can handle both gridded and time series data and has various functions for cleaning/ processing data to various quality standards and produces QA reports. MeteoIO is C++ library.
MeteoIO git: https://models.slf.ch/p/meteoio/
In order to publish discovery metadata for the data prepared through MeteoIO, ENVIDAT is used. EnviDat is the WSL/SLF main CKAN based dataportal and metadata repository. Core CKAN has been extended to cover specific requirements of research data management. These include an OAI-PMH server, DOI publishing and supporting metadata standards. The advantage of CKAN is that it provides a robust and intuitive UI for structured metadata submission. This enables large parts of the data management process to be decentralised to the submitter.
CKAN project: https://ckan.org/
EnviDat extensions are here: https://github.com/EnviDat
The OPeNDAP component providing external access to the data preprocessed through MeteoIO and announced through ENVIDAT is currently under development.