Utilizing NASA and ESA Earth Observations to Monitor Water Quality Conditions in the San Francisco-Bay Delta

The Bay Delta Live Water Quality and Operations Data Portal is a collaborative effort to integrate NASA Earth Science Data with environmental data and decision support tools for water operations. This project’s major objective is to make NASA Earth science data available to water resource managers and decision makers in an easy to use web-based application. The project utilizes multi-spectral imagery from the Sentinel-2 and Landsat-8 satellites to estimate water quality conditions including chlorophyll, turbidity and water surface temperature. These products have the potential to significantly increase the extent of in situ monitoring networks or supplement such programs by orders of magnitude.

Project Highlights
  • Utilize Sentinel-2 and Landsat-8 Imagery to monitor water quality conditions over large spatial extent
  • Recalibrate and validate water quality algorithms using automated Data Matchup Pipeline
  • Map, analyze and graph water quality conditions in pre-configured dashboards
  • Export graphs, maps and data for key management operations

Map representing Turbidity product derived from Sentinel-2 imagery for the San Francisco Bay on February 5, 2016.

Turbidity product derived from Sentinel-2 imagery for the San Francisco Bay on February 5, 2016.

BDL Explore Data Interface to access the Data Matchup Pipeline results.

BDL Explore Data Interface to access the Data Matchup Pipeline results.

Data
  • Sentinel-2 Multispectral Imagery
  • Landsat-8 Multispectral Imagery
  • California Data Exchange Center
  • National Water Information System

The multi-spectral imagery is initially processed by Nick Tufillaro and his team at Oregon State University to produce preliminary single band rasters stored as GeoTIFFs. 34 North retrieves the raster imagery from the OSU remote servers and uploads it to the Bay Delta Live server. The GeoTIFFs are then imported into PostgreSQL tables that enable web based rendering and advanced pixel queries. Once stored in the BDL database, the imagery is run through the custom Data Matchup Pipeline that compares the estimated water quality values with in-situ observations from the California Data Exchange Center (CDEC) water quality monitoring network. The matchup data is displayed on a set of interactive dashboards that enable advanced visualization, regression analysis and outlier identification. These data are used to re-calibrate the algorithms, and ultimately provides a validation of the accuracy. This pipeline greatly increases the speed and efficiency of the calibration and validation process, automating a workflow which had previously been done manually.

Preconfigured Cal/Val Dashboard with turbidity products and Data Matchup Pipeline results.

Preconfigured Cal/Val Dashboard with turbidity products and Data Matchup Pipeline results.

Interactive graphing panel showing Data Matchup Pipeline results at two CDEC Stations (Prisoners Point and Sacramento River At Decker Island). Graphs are configured as scatter plots with linear regression lines.

Interactive graphing panel showing Data Matchup Pipeline results at two CDEC Stations (Prisoners Point and Sacramento River At Decker Island). Graphs are configured as scatter plots with linear regression lines.

The Data Matchup Pipeline is written in Python, and utilizes a series of existing OpenNRM data utilities that retrieve data from the CDEC web service. The first step in the pipeline is to extract the date and time of each image capture, which is stored in the filename. Next, the pipeline loops through each CDEC station from a user supplied list. The user also supplies a time parameter that specifies the buffer around image capture time to pull sensor observations from. The temporally specified sensor observations are retrieved from the CDEC web service, and the average, standard deviation, coefficient of variation and count of these data are recorded. Next, the raster imagery is analyzed around each station. The user provides a neighborhood parameter that determines the size of the pixel neighborhood centered on each station. All non- land masked pixels with in this neighborhood are retrieved from the imagery stored on the Bay Delta Live servers using a custom web service. The average, standard deviation, coefficient of variation and count of these pixel values are recorded. The resulting matchup data is imported into the Bay Delta Live Explore Data inferface, allowing for custom queries, visualization and download.