Implementation of a workflow for streaming sensor data for a large-scale hydrologic monitoring network
Location
Eccles Conference Center
Event Website
http://water.usu.edu
Start Date
4-1-2014 6:20 PM
End Date
4-1-2014 6:40 PM
Description
Hydrologic monitoring with in situ environmental sensors presents many challenges for data management, particularly for large-scale networks consisting of multiple sites, sensors, and personnel. The high frequency, extended duration, and spatial distribution of data collection efforts require cyberinfrastructure to support and facilitate research. Researchers and practitioners need tools for data import and storage as well as data access and management. In addition to addressing the challenges presented by the sheer quantity of data, monitoring networks need practices to ensure high data quality, including procedures and tools for post processing. Data quality is further enhanced if networks are able to track physical infrastructure such as equipment, deployments, calibrations, and other events related to site maintenance and associate these details with observational data. We will present a case study of a workflow for streaming sensor data for the iUTAH (innovative Urban Transitions and Aridregion Hydrosustainability) ecohydrologic observatory. The iUTAH monitoring network consists of aquatic and climate sensors deployed in three Utah watersheds to monitor Gradients Along Mountain to Urban Transitions (GAMUT). The variety of environmental sensors and the multi-watershed, multi-institutional nature of the network necessitate a well-planned and efficient workflow for acquiring, managing, and sharing sensor data. We will present the overall workflow that we have developed for GAMUT data management, the software tools that we have developed and deployed, and aspects of the data quality assurance and quality control plan that have been implemented. Features of the workflow include a data model and web interface for managing sensor infrastructure, Python-based tools for performing quality control post-processing, and web-based applications providing access and visualization of the data. The tools presented will be useful for similar large-scale and long term monitoring networks.
Implementation of a workflow for streaming sensor data for a large-scale hydrologic monitoring network
Eccles Conference Center
Hydrologic monitoring with in situ environmental sensors presents many challenges for data management, particularly for large-scale networks consisting of multiple sites, sensors, and personnel. The high frequency, extended duration, and spatial distribution of data collection efforts require cyberinfrastructure to support and facilitate research. Researchers and practitioners need tools for data import and storage as well as data access and management. In addition to addressing the challenges presented by the sheer quantity of data, monitoring networks need practices to ensure high data quality, including procedures and tools for post processing. Data quality is further enhanced if networks are able to track physical infrastructure such as equipment, deployments, calibrations, and other events related to site maintenance and associate these details with observational data. We will present a case study of a workflow for streaming sensor data for the iUTAH (innovative Urban Transitions and Aridregion Hydrosustainability) ecohydrologic observatory. The iUTAH monitoring network consists of aquatic and climate sensors deployed in three Utah watersheds to monitor Gradients Along Mountain to Urban Transitions (GAMUT). The variety of environmental sensors and the multi-watershed, multi-institutional nature of the network necessitate a well-planned and efficient workflow for acquiring, managing, and sharing sensor data. We will present the overall workflow that we have developed for GAMUT data management, the software tools that we have developed and deployed, and aspects of the data quality assurance and quality control plan that have been implemented. Features of the workflow include a data model and web interface for managing sensor infrastructure, Python-based tools for performing quality control post-processing, and web-based applications providing access and visualization of the data. The tools presented will be useful for similar large-scale and long term monitoring networks.
https://digitalcommons.usu.edu/runoff/2014/2014Posters/38