OUR COMMUNITY PROJECTS

ESS Community Data Curator

Kim Ely, LBNL

The ESS Community Data Curator project and role will assist ESS-DIVE data contributors in curating high quality datasets using ESS-DIVE reporting formats, improve data curation workflows, advise, and provide training for ESS-DIVE data contributors. To request ESS Community Data Curator support, use the ESS-DIVE Contact Us form and “Request data curation assistance” by selecting the Other option, and provide details for what you need help with.

A Workflow and Reporting Format for Processing Environmental Sensor Data and
Automated Generation of ESS-DIVE Compliant Metadata

Stephanie Pennington and Ben Bond-Lamberty, PNNL

This project will develop an environmental sensor data reporting format, which will include proposing standard guidelines for versioning and processing levels of sensor data, and an automated processing pipeline to handle sensor data and generate ESS-DIVE compliant sensor data and metadata. Reporting format development will involve surveying other major ESS projects using data loggers and environmental sensors to synthesize best practices and commonly used formats. This project will also update the existing Soil Respiration Reporting Format to interact with these sensor data processing new tools.

Harmonization and Increased Usability of ESS-DIVE’s Hydrologic Monitoring and Soil, Sediment, and Water Chemistry Reporting Formats

Amy Goldman and Brieanne Forbes, PNNL

This project will update and harmonize the ESS-DIVE Hydrologic Monitoring and Water-Soil-Sediment Chemistry reporting formats to improve machine readability and interoperability with ESS-DIVE’s features, and develop programmatic tools for data contributors and users to aid in use of the two reporting formats.

Improving Advanced Terrestrial Simulator (ATS) model data managing and archiving standards

Ethan Coon, ORNL and Zhi Li, PNNL

This project will prototype standardized procedures for managing and archiving model data for the Advanced Terrestrial Simulator (ATS) model. This project will enhance, optimize, and standardize the existing Model Data Archiving Guidelines reporting format; and improve ATS’s ability to generate standardized Model Data Archives (MDA), including implementing CSV-standard compliant ATS observation files and developing scripts for setting up, automating, and submitting MDAs to ESS-DIVE including the semi-automated generation of File Level Metadata and Data Dictionary files.

Unoccupied Aerial System (UAS) data and metadata

Kim Ely and Shawn Serbin, BNL

This project developed a reporting format that provides metadata requirements and formatting guidelines for uploading data from small (≤ 25 kg) Unoccupied Aerial Systems (UASs) to the ESS-DIVE archive. This includes basic recording of core flight data (e.g. telemetry), when available, and primary flight metadata (e.g. flight location, flight paths, altitude, date, start/end time), basic UAS payload and instrumentation information (e.g. camera make and model), and the general data product and file metadata needed for interpreting and accessing derived UAS datasets (e.g. pixel resolution, spatial reference system). The objective was to develop a community-standard basic metadata format that works for any UAS platform and instrument payload while providing some guidance on how to expand on this with advanced metadata levels and suggested data storage formats that are optimized for cloud access.

Unoccupied Aerial System (UAS) Reporting Format Documentation

Generic standard for file-level metadata and comma-separated data files

Ranjeet Devarakonda, Susan Heinz, and Terri Velliquette, ORNL

This project developed two standards for data types important to the ESS community: 1) a generic standard for the structure of data stored in comma-separated values (CSV) text format files, and 2) a metadata schema to describe the contents, scope, and structure of each data file within the ESS-DIVE repository. File-level metadata is the description of individual files that are part of a larger collection. This metadata is fully consistent with, and augmented, the metadata collected to describe each data collection. Providing file-level metadata helps to provide granular information that will enable data users to search for and locate files that contain specific measured parameters or variables, to understand and compare the contents of each data file within an overarching data collection.

File-level metadata Reporting Format and Comma-seperated Values (CSV) Reporting Format Documentation

Soil respiration data and metadata

Ben Bond-Lamberty and Stephanie Pennington, PNNL

This project designed standards and tools for a soil respiration database. It built upon a prototype continuous soil respiration database (COSORE, see https://github.com/bpbond/cosore), in which individual contributed datasets are mapped to standardized data and metadata fields. This research project aimed to (i) identify and develop a standard format for soil respiration data important to the earth system science community, (ii) define the cross-walk between this standard and other soil respiration standards in use in the community, and (iii) test QA/QC and ingestion tools.

Soil Respiration Reporting Format Documentation

Hydrological Monitoring Data

Amy Goldman and Huiying Ren, PNNL

The project was build upon existing standards and/or well-established best practices for hydrologic monitoring data including single-point-in-time and logged measurements. Variables included are pressure, river stage, temperature, and conductivity (electrical and specific conductance). The scope was expanded to include other variables such as those measured by aquatic sondes (e.g., dissolved oxygen, pH, oxidation reduction potential).

Hydrological Monitoring Reporting Format Documentation

Storage of leaf level gas exchange data and metadata files

Alistair Rogers and Kim Ely, BNL

This research project identified and developed a standard format for storage of leaf level gas exchange data and metadata files. The aim of this work was to develop standardized formats, vocabulary, and metadata requirements for uploading data from leaf-level gas exchange to the ESS-DIVE archive. This included all measurements made with a portable gas exchange system such as survey style measurements of photosynthesis, respiration and stomatal conductance; response curves e.g. light response curves, CO2 response curves, and temperature response curves; curves used to determine stomatal slope and fitting of derived parameters e.g. Vc, max from photosynthetic CO2 response curves. It also included other derivations of key parameters e.g. derivations of Vc max from proxies such as “one-point” measurements or spectroscopy.

Leaf-level Gas Exchange Reporting Format Documentation

Water quality and soil/sediment samples data and metadata files

Kristin Boye, SLAC

The aim of this work was to develop standardized formats, vocabulary, and metadata requirements for uploading data from water, soil, and sediment samples to the ESS-DIVE archive. It built upon existing standards and/or well-established best practices for water sample data (e.g. EPA’s WQX) and geological/soil sample data (e.g. National Cooperative Soil Survey and other USDA/USGS recommended/employed formats) to develop ESS-specific standards and templates for data and associated metadata uploads to ESS-DIVE.

Water-soil-sediment Chemistry Reporting Format Documentation

16S amplicon data products standardization and KBase integration

Pamela Weisenhorn, ANL

The aim of this work was to develop standardized formats, vocabulary, and metadata requirements for uploading 16S amplicon data products (specifically, operational taxonomic unit (OTU) and environmental sequence variant (ESV) tables) to the ESS-DIVE archive. It buildt upon existing standards and/or well-established best practices for this data product, which results from bioinformatic processing of 16S data. Multiple pipelines exist for the generation of this data product and appropriate capture of the metadata associated with the analysis is critical in determining the ability to combine or compare across data sets generated by different researchers for meta-analysis or modeling purposes. This work involved determining best practices for the preservation of data provenance and metadata on bioinformatic processing to enable end-users to determine the appropriateness of the data products for reuse. In addition to consulting broadly with the ESS funded university partners, this data standard was developed in consultation with the JGI, the National Microbiome Data Collaborative (NMDC), KBase, MG RAST, and other relevant stakeholders.

Amplicon Data Reporting Format Documentation