1. Earth Science Data Systems (ESDS) Program
  2. Open Science
  3. Open Source Science for the Earth System Observatory Mission Data Processing Study Workshops

Open Source Science for the Earth System Observatory Mission Data Processing Study Workshops

Diagram explaining components of the Earth System Observatory

NASA is formulating the Earth System Observatory (ESO), a set of Earth-focused missions to provide key information to guide efforts related to monitoring climate change, mitigating disasters, fighting forest fires, and improving real-time agricultural processes. With this integrated approach, Kevin Murphy (Chief Science Data Officer, Science Mission Directorate, and Program Manager, Earth Science Data Systems Program) has set forth a challenge to the mission processing community to:

Identify and assess potential architectures that can meet the ESO mission science processing objectives, enable data system efficiencies, promote open science principles, and seek opportunities that support Earth system science.

A mission data processing system is the set of algorithms, software, compute infrastructure, operational procedures, documentation, and teams that automatically process raw instrument data through to science-quality data products. This includes the software tools that support the development of the processing algorithms and the validation and analysis of the processed data.

Graphic representing open science as layers, with open science at the center, open science focus areas in the middle, and data program-specific strategies that enable open science in the outer layer.
The open science concept, with open science at the center, open science focus areas in the middle, and data program-specific strategies in the outer layer. Image from Ramachandran, et al. From Open Data to Open Science, Earth and Space Science.

Open science is a foundational objective of NASA’s Science Mission Directorate (SMD) and is defined as “a collaborative culture enabled by technology that empowers the open sharing of data, information, and knowledge within the scientific community and the wider public to accelerate scientific research and understanding” (Ramachandran, R., Bugbee, K. & Murphy, K.J. From Open Data to Open Science. Earth and Space Science, 8(5), doi:10.1029/2020EA001562). On October 14, SMD will report out on open science and activities at NASA's Open Source Science for Data Processing and Archives Workshop.

The Open Source Science for ESO Mission Processing Study has a Steering Committee and System Architecture Working Group (SAWG) to address the challenge laid out above. The Study will consist of three workshops and a trade study designed to solicit input, perform systematic trade, and make a recommendation.

  • Workshop #1: received input from NASA Program Officers and ESO Missions regarding requirements, constraints, recommendations, and opportunities for science data processing.
  • Workshop #2: will understand the state-of-the-art in Big Data processing systems by soliciting inputs from governmental agencies, industry, and academia.
  • The SAWG will perform an architecture trade study between March and July, 2022. The team will use the findings from Workshop #1 and #2 as well as reach-back into the community as input to their study.
  • Workshop #3: will provide a technical and programmatic recommendation to NASA Headquarters.

Workshop #1, October 19-20, 2021

Workshop #1, October 19-20, 2021

We began collecting information with our first virtual two-day workshop on Tuesday, October 19, 2021 (programmatic presentations) and Wednesday, October 20, 2021 (flight project presentations).

The input provided, along with information collected in the second workshop (scheduled for February 2022) on the state-of-the-art in science processing and open science, will be used by the ESO Open Source Processing Study Architecture Working Group to develop and assess potential architectures.

Agenda

Open science is defined as a collaborative culture enabled by technology that empowers the open sharing of data, information and knowledge within the scientific community and the wider public to accelerate scientific research and understanding. Under the guidelines set forth in the 2017 Decadal survey, NASA's new set of Earth-focused missions that form the ESO will adhere to open data, software, algorithm, and publication policies where missions will develop algorithms (in Levels 1–4), software, and documentation in open systems from inception. In order to generate community input and support, this first workshop solicited input from NASA programmatic officials and flight project teams.

Recordings of the workshop are available: ESO Mission: Data Processing Study Workshop

Day 1: Tuesday October 19th, 2021 (1 - 5 PM EST)
Duration EST PST Topic Speaker
0:10 1:00 PM 10:00 AM Welcome and Purpose Andrew Mitchell
0:15 1:10 PM 10:10 AM Earth Science Division (ESD) Opening Remarks Kevin Murphy
0:15 1:25 PM 10:25 AM Science Mission Directorate (SMD) Scientific Information Policy Steven Crawford
0:15 1:40 PM 10:40 AM ESDS Perspective of Open Science Katie Baynes
0:15 1:55 PM 10:55 AM Overview of the Mission Processing Study

Andrew Bingham

0:15 2:10 PM 11:10 AM Q&A with the System Architecture Working Group
0:10 2:25 PM 11:25 AM Break
0:15 2:35 PM 11:35 AM Flight Projects Program Perspective Charles Webb, Kathleen Boggs
0:15 2:50 PM 11:50 AM Earth Science Technology Office (ESTO) Perspective

Pam Millar, Jacqueline Lemoigne-Stewart, Ben Smith

0:15 3:05 PM 12:05 PM R&A Program Perspective
Jack Kaye
0:15 3:20 PM 12:20 PM Q&A with the System Architecture Working Group
0:10 3:35 PM 12:35 PM Break

0:15 3:45 PM 12:45 PM Applied Science Program Perspective Nancy Searby
0:15 4:00 PM 1:00 PM

High Performance Computing (HPC) Perspective

Tsengdar Lee
0:10 4:15 PM 1:15 PM Q&A with the System Architecture Working Group
0:30 4:25 PM 1:25 PM Open Discussion
4:55 PM 1:55 PM END

 

Day 2: Wednesday October 20th, 2021 (12:20 - 5 PM EST)
Duration EST PST Topic Speaker
0:10 12:20 PM 9:20 AM Recap and Agenda
0:15 12:30 PM 9:30 AM NISAR Programmatic Perspective Gerald Bawden
0:15 12:45 PM 9:45 AM NISAR Project Science Paul Rosen
0:15 1:00 PM 10:00 AM NISAR Project Data Processing Systems Hook Hua
0:10 1:15 PM 10:15 AM Q&A with the System Architecture Working Group
0:05 1:25 PM 10:25 AM Break
0:15 1:30 PM 10:30 AM Atmosphere Observing System (AOS) Programmatic Perspective Hal Maring
0:15 1:45 PM 10:45 AM AOS Project Science Scott Braun
0:15 2:00 PM 11:00 AM AOS Project Data Processing Systems Robert Wolfe
0:10 2:15 PM 11:15 AM Q&A with the System Architecture Working Group
0:05 2:25 PM 11:25 AM Break
0:15 2:30 PM 11:30 AM Mass Change (MC) Programmatic Perspective Lucia Tsaoussi
0:15 2:45 PM 11:45 AM MC Project Science David Wiese 
0:15 3:00 PM 12:00 PM MC Project Data Processing Systems David Wiese
0:10 3:15 PM 12:15 PM Q&A with the System Architecture Working Group
0:05 3:25 PM 12:25 PM Break
0:20 3:30 PM 12:30 PM Surface Biology and Geology (SBG) Overview and Project Science Dave Schimel
0:20 3:50 PM 12:50 PM SBG Project Data Processing Systems Jeff Pon
0:05 4:10 PM 1:10 PM SBG Programmatic Perspective Ben Phillips
0:10 4:15 PM 1:15 PM Q&A with the System Architecture Working Group
0:30 4:25 PM 1:25 PM Open Discussion
4:55 PM 1:55 PM END

Study Team

Study Team

Dr. Andrew Bingham has more than 20 years of experience managing and leading teams responsible for delivering science processing, archive, distribution, and analysis systems for Earth and planetary missions. He is the manager for the Science Data and Instrument Operations System Section at NASA's Jet Propulsion Laboratory.

Luke Dahl has more than 20 years of experience as an architect and systems engineer supporting diverse activities ranging from enterprise infrastructure and applications, multi-mission instrument operations, and Earth and planetary science data systems.

Dr. Chelle Gentemann is a senior scientist at the Farallon Institute leading research on open science, cloud computing, remote sensing, and physical oceanography. For more than 20 years, she has worked on every aspect of passive microwave satellite missions, both domestically and internationally, from launch through decommission, including calibration, algorithm development, validation, data distribution, and science applications.

Dr. Sara Lubkin has a broad background in Earth science. She has worked with NASA since 2015 and is a Distributed Active Archive Center (DAAC) Operations Engineer with NASA’s Earth Science Data and Information System (ESDIS) Project at NASA's Goddard Space Flight Center.

Andrew Mitchell has multiple years of experience executing the technical and financial management of the science systems of NASA's Earth Observing System Data and Information System (EOSDIS). He is responsible for managing the processing, archiving, and distribution of Earth science data while ensuring scientists and the public have access to these data to enable the study of Earth. He is the project manager for ESDIS at Goddard.

Karen Yuen has 20 years of technical and management experience in science applications, project formulation, system engineering, and communications for multiple NASA missions and programs. She is the Science Data Applications Lead for the Orbiting Carbon Observatory-2 (OCO-2) and OCO-3 missions.

Study Architecture Working Group Members

Study Architecture Working Group Members
Name Focus Area Organization Required Expertise
Elias Sayfi Co-Chair Jet Propulsion Laboratory
  • Flight mission data processing system architect.
  • Community builder
Natasha Stavros Co-Chair University of Colorado, Boulder
  • Science data processing system architect
  • Community builder
Hook Hua SBG & NISAR Science Processing Representative Jet Propulsion Laboratory
  • Science data processing systems
  • L3 requirements and mission objectives
Curt Tilmes AOS Science Processing Representative Goddard Space Flight Center/Ames Research Center
  • Science data processing systems
  • L3 requirements and mission objectives
Bernie Bienstock MC Science Processing Representative Jet Propulsion Laboratory
  • Science data processing systems
  • L3 requirements and mission objectives
Wenying Su Applied Science Algorithm Developer Langley Research Center
  • Expertise developing L3+ algorithms for science applications
  • Experience integrating algorithms into a pipeline.
  • Experience working in a collaborative development environment.
Andy Michaelis Data Processing Workflow Community-focused Developer Ames Research Center
  • Expertise in modern open-source development practices, cloud-based orchestration, GitHub, Continuous Integration.
  • Experience with big data applications (science preferred, but not necessary)
Lesley Ott Geophysical Numerical Modeler Goddard Space Flight Center
  • Expertise with HPC
  • Expertise with big data assimilation and operations
Qing Yue Instrument Algorithm Developer Jet Propulsion Laboratory
  • L2 algorithm development
  • Science data processing systems
Evelyn Ho Science Processing Architect from a comparable big-data science-based organization or project Earth Science Data Information System (ESDIS) Science Investigator-led Processing Systems (SIPS)  
Chris Engebretson Science Processing Architect from a comparable big-data science-based organization or project United States Geological Survey (USGS)  
Adrian Parker Science Processing Architect from a comparable big-data science-based organization or project National Oceanic and Atmospheric Administration (NOAA)  
Sean Harkin Science Processing Architect from a comparable big-data science-based organization or project Marshall Space Flight Center  

Page Last Updated: Nov 19, 2021 at 8:49 AM EST