ESS-DIVE

Deep Insight for Earth Science Data

ESS-DIVE Training Sessions at the 2022 U.S. DOE ESS PI Meeting

by

Join ESS-DIVE at the annual Environmental System Science (ESS) Program Principal Investigators Meeting that begins on Tuesday, May 24, 2022, and concludes on Thursday, May 26, 2022. The ESS-DIVE team will present at multiple sessions at this virtual meeting hosted by the U.S. Department of Energy. 

On Tuesday, May 24, 2022 the ESS-DIVE team will present the following hands-on data management trainings from 1:30 pm – 3:30 pm ET that will help you better understand and use various ESS-DIVE features for managing project data. Here are more details:


New user bootcamp
Presenter: Emily Robles
Breakout room A

Learn how to upload and publish data with ESS-DIVE as a new contributor in this hands-on training session. We will go over the process for publishing data on ESS-DIVE, from important considerations for organizing data and accompanying metadata, to requesting publication and following the dataset review process. We will also discuss how to utilize the ESS-DIVE main portal search features and download data.


Project data management
Presenters: Charuleka Varadharajan and Madison Burrus
Breakout room B

This session is designed for PI’s and data managers who want to learn how to manage project data on ESS-DIVE. We will first provide an overview of how ESS-DIVE can support a project’s data management needs. We will then describe how projects can use ESS-DIVE features to curate and manage data including: (1) a package Service API for bulk programmatic dataset creation and editing (2) sharing datasets with project members for collaborative editing and review. (3) linking data on ESS-DIVE to related datasets on other public data repositories (4) portals to showcase data collections, and (5) an upcoming feature to manage project members and data. 


ESS-DIVE data formats
Presenters: Robert Crystal-Ornelas and Dylan O’Ryan
Breakout room C 

In this session, we will discuss ways that researchers can make research data more reusable using ESS-DIVE’s 11 community data and metadata formats. These data formatting templates apply to many data types relevant to earth and environmental science research. They include general formats that apply to many data types (CSV files, file-level metadata, locations, samples, model data), and formats for specific data types (water and sediment chemistry, sensor-based hydrological monitoring, leaf-level gas exchange, soil respiration, and amplicon abundance). In this session, participants will apply the reporting formats to tabular data, and hear from researchers who are currently using reporting formats on their own environmental data and metadata.

 

There will be plenty of time to share your questions, comments, and other thoughts with the ESS-DIVE team throughout each session. Other ESS-DIVE activities at the event include an overview poster and a plenary talk by Shreyas Cholia on Thursday, May 26 from 11:55 am-12:05 pm ET. 

For more information, contact ESS-DIVE support at  ess-dive-support@lbl.gov.

March Webinar on Terrestrial Model Data Archiving

by

March Community Webinar

Monday, March 21 | 11:00-12:00 PT / 14:00-15:00 ET

View Webinar Video / Link to Webinar Slides

Are you looking to publish your model data on ESS-DIVE? Learn about our guidelines for archiving terrestrial simulation data developed in partnership with the ESS modeling community and outlined in a recent publication (Simmonds et al., 2020).

This will be a short webinar, with ample time for questions and feedback. We will cover the following topics related to archiving model data using FAIR principles: 

  • What is terrestrial model data and what model data components should be archived?
  • What are the considerations when publishing terrestrial model data?
  • What are the benefits of archiving model data?
  • Provide feedback on additional needs for archiving terrestrial or other model data

Please encourage anyone from your project who may be interested to attend.

Webinar will be presented by Charuleka Varadharajan ESS-DIVE Deputy Lead, Lead: Community Engagement

Charu is the Deputy Lead for ESS-DIVE. She also leads the Community Engagement task, working with the ESS community to build use cases for ESS data needs and deploy data standards, and leading outreach efforts. Charu is a biogeochemist and with expertise in groundwater quality and biogeochemical cycling of methane. She is working on synthesis and analysis of environmental datasets, and has developed software tools for the Watershed Function SFA and NGEE Tropics ESS projects. She is a co-chair of the ESS Cyberinfrastructure Data Management Working Group.

Webinar on Expanding the Discoverability of your ESS-DIVE Data through External Linking

by

February Community Webinar

Monday, February 7 | 11:00-12:00 PT / 14:00-15:00 ET

View Webinar Video / Link to Webinar Slides

Enhance the discoverability of your datasets by cross-linking relevant data and associated information across repositories. Learn about ESS-DIVE’s new approach and capability to link your ESS-DIVE data to other online repositories and data systems.

This will be a short webinar, with ample time for questions and feedback. We will cover the following use cases for linking your data and metadata:

  • Link to individual data files or copies of your data stored elsewhere
  • Link to the original publication of a dataset (e.g. your project’s data archive) where metadata and data can be found
  • Provide feedback on additional needs for linking to external data, methods, samples, and publications.

Please encourage anyone from your project who may be interested to attend.

Webinar presented by Joan Damerow Community Engagement Lead Scientist

Joan is an environmental scientist with a diverse background in geoscience sampling, freshwater ecology, and biodiversity informatics. She runs engagement activities for ESS-DIVE, including ESS-DIVE webinars, our community data workshop, and is active in relevant conferences and data working groups (e.g. ESIP, RDA, AGU). Joan is interested in interdisciplinary data management and tracking, and works with our community of data contributors to identify, develop, and implement practical data standards in ESS-DIVE that support FAIR principles.

Standardizing Water Quality Data with New ESS-DIVE Community Reporting Formats

by

Photo by Hans Reniers on Unsplash

Dylan O’Ryan is a Student Assistant with the Environmental System Science Data Infrastructure for a Virtual Ecosystem (ESS-DIVE) data repository. He writes here about his experience with ESS-DIVE’s development and implementation of data reporting formats [community data standards] in collaboration with six teams of scientists in the US DOE Environmental System Science (ESS) community. Dylan first began working with ESS-DIVE as part of a Community College Internship (CCI), where he standardized existing water quality data using ESS-DIVE’s community data reporting formats.

Some of ESS-DIVE’s data reporting formats, such as that for soil and water quality, are specific to research domains. Other reporting formats are generalized to a wide range of data such as Comma Separated Value (CSV) files and sample collections. These standard reporting formats are designed to make data more Findable, Accessible, Interoperable, Reusable (FAIR) from the perspective of our ESS community scientists. Data reporting formats standardize data to enable creation of better tools that allow advanced search, integration, and visualization of data within and across multiple datasets. 

Kristin Boye, a Staff Scientist at SLAC National Accelerator Laboratory, developed a water/soil/sediment chemistry reporting format as part of her collaboration with ESS-DIVE. She developed this reporting format by synthesizing recommendations from other generalized reporting formats, such as CSV and FLMD (File-level Metadata), and incorporating community feedback on how to format water/soil/sediment chemistry data. 

Storm Drain Detectives is a community-based water quality monitoring program in Lodi, California. For the past seven years, I’ve helped the program measure water quality parameters such as DO, pH, temperature, and bacteria. This experience with water quality testing enabled me to better understand the datasets that I was converting for ESS-DIVE. I used the community water/soil/sediment chemistry data reporting format to convert existing datasets within the Lawrence Berkeley National Laboratory (LBNL) Watershed Function Scientific Focus Area (WFSFA) project. I converted water quality datasets where their metadata information was already published on ESS-DIVE, including ICP-MS, DIC/NPOC/TDN, Ammonia-N, Anion, and Isotope data. 

Here is the step-by-step process that details how I converted existing datasets to the water/soil/sediment chemistry reporting format [See Image 1 for workflow diagram]:

  • Retrieve the water quality data file and locate the associated metadata published on ESS-DIVE. 
  • Populate the methods file template. The methods file is where you store information on the samples’ methods of collection, analysis, storage, etc. I entered the information supplied by the data provider from the associated dataset metadata describing their methods. See Image 2 for an example of converted methods information to the reporting format methods file.
  • Populate the data file template. This data file is similar to most data files where you input sample information and measurements; however, this reporting format data file is designed to include information needed for future interpretation and reuse, such as: unique sample names, methods information (collection/analysis procedures, detection limits, analysis precision) as well as the data. The data file template also allows for standardized variable names and units across the files. Standardized names and units can be included in the term list. See Image 3 for an example of a converted data file from a data provider to the reporting format data file template. 
    • Note: I first filled in the methods information and header rows before populating the sample data. 
  • As part of this reporting format, you can choose to fill out an optional terminology file. The terminology file can include all terms that would benefit from additional description and definition (e.g., data flags or other codes used throughout the data and method files). We note that the terminology file is different from the required data dictionary file that is part of ESS-DIVE’s file-level metadata reporting format. In the data dictionary, you provide definitions of column or row names, and their units. The terminology file is specifically designed for terms that are not captured in the column or row names. See Image 4 for an example of a terminology file.

The water/soil/sediment chemistry reporting format was straightforward. I was able to catch on to using the template and requirements of the reporting format. Transferring datasets is easy once you understand the general structure. While converting these data files, I became faster with converting where it became quick to create a methods file and data file with over 200 samples within 30 minutes. 

Here are a few more tips and tricks related to converting a multitude of datasets:

  • You generally only need to create one methods file for a particular measurement (e.g., ICP-MS), where you would only need to adjust the data file to include the samples you tested. 
  • Similarly, the data file headers and associated terms can be repeated if there are no collection or analysis procedure changes. 

I found that utilizing ESS-DIVE’s reporting formats was straightforward and made the data easier to find, understand, and use in new ways. The converted datasets include unique sample names, contextual information describing the data (metadata), standardized formatting of missing values, and many more qualities that increase the usability of the data. The examples of converted water quality datasets are now being utilized by some WFSFA data providers in order to standardize their data and metadata.

Some other reporting formats that may help you standardize your data and metadata are CSV, File Level Metadata (FLMD), Sample Identifiers and Metadata, and Model Data Archiving Guidelines, which are high-level reporting formats that apply across multiple domains. The Leaf Gas Exchange reporting format, which is intended for leaf-level gas exchange data. The Soil Respiration reporting format, which is intended for soil respiration data and metadata. Hydrological Monitoring reporting format, which is designed for water parameters measured by in situ meters/probes. There are a couple of reporting formats in development: 16S Amplicon Sequencing and Locations Metadata. See Image 5 for a workflow for use of ESS-DIVE’s reporting formats. 

The ESS-DIVE team is available for questions and help for those who want to use the reporting formats. Please email ess-dive-support@lbl.gov or you can use the “Contact US” feature on the ESS-DIVE website.

Image 1: Workflow for Conversion of Reporting Format

 

Image 2: Example conversion of methods information to methods file

 

Image 3: Example conversion of datafile to reporting format data file template

 

Image 4: Example of terminology file

 

Image 5: Workflow for use of reporting formats

ESS-DIVE at AGU 2021

by


The ESS-DIVE team is excited to present their work and connect with the Earth and Environmental Systems Science (ESS) community at the upcoming American Geophysical Union (AGU) Fall Meeting 2021. The event will take place in New Orleans, LA and online everywhere 13-17 December 2021. Several ESS-DIVE team members are presenting on relevant topics, ranging from best practices for data curation and publication to approaches to support metadata synthesis. ESS-DIVE will be involved with a total of 6 oral and eLightening presentations.

#AGU21 is the leading forum for advancing Earth and space science and leveraging this research toward solutions for societal challenges. The Earth and space science community is gathering both in person and virtually for this annual meeting to learn and collaborate around the theme of “Science is Society.” With more than 25,000 individuals from 100+ countries expected to attend representing the global Earth and space sciences community, the event will consist of inspiring plenary talks, cutting-edge science presentations and more.Most sessions will be recorded and available to this global community of researchers, scientists, educators, students, policymakers, partners, science enthusiasts, journalists, and communicators. With in-person and worldwide online participation, attendees will have numerous opportunities to network with government regulators, scientific visionaries, and industry thought-leaders. 

 

ESS-DIVE is enthusiastic about the opportunity to engage in this collaborative and interdisciplinary event. The interactive nature of this event will serve as a platform to share research findings, discuss use cases, and more. The team looks forward to not only sharing their knowledge, but also gaining new insights and experiences. 

ESS-DIVE is funded by the Data Management program within the Earth and Environmental Systems Science Division under the DOE’s Office of Science Biological and Environmental Research program and is maintained by the Lawrence Berkeley National Laboratory.