Summary of the Project

• Developed a reproducible workflow for SWOT (https://swot.jpl.nasa.gov) data preprocessing and calibration against in situ measurements for river hydrometry, in the broader context of the Generative AI for Earth System Science project.
• Included in situ data download, quality filtering, and self-contained outlier detection.
• Established a stronger foundation for transfer across locations and for subsequent model development and evaluation.

Project Description

This work focused on strengthening the preparation of SWOT observations for downstream AI and hydrological applications through implementation-oriented research and development. The main effort addressed the preprocessing and calibration stage, with particular attention to how SWOT measurements should be quality-controlled, filtered, and compared against in situ references before being used in later modelling steps.

A substantial part of the contribution concerned the joint handling of SWOT and in situ data. This included developing code to download and prepare in situ gauge observations, aligning them with SWOT-derived data, and using these references to guide the calibration of preprocessing choices. Within this context, the Filter Calibrator (Fig. 1) was developed as a structured framework to support systematic evaluation of filtering strategies and parameter settings against gauge measurements.

An additional methodological contribution was the introduction of a self-contained outlier detection logic based on internal consistency checks. The aim was to identify anomalous measurements without relying primarily on external assumptions, and to make the preprocessing stage more robust across varying conditions and locations.

Alongside this implementation work, a parallel contribution addressed the broader AI problem definition within ongoing projects, particularly the question of how hydrological observations should be represented and ingested by AI-based methods. Together, these efforts helped establish a stronger basis for subsequent model development, evaluation, and potential future publication.

Research and Development tools

The development workflow used Python scientific and geospatial tooling for data preparation, calibration, and diagnostics. Collaboration and versioning managed via GitHub. Core libraries included NumPy, pandas, xarray, and h5netcdf for time-series and NetCDF processing, GeoPandas for spatial operations, and Matplotlib for quality-control plots and inspection. Multiprocessing was used to accelerate heavy preprocessing stages.

Development Outputs

Current outputs

• Enhancements to the SWOT preprocessing pipeline to support AI modelling.
• CLI Filter Calibrator framework for repeatable calibration against in situ measurements.
• Self-contained outlier detection logic into the preprocessing and calibration workflow.
• Internal documentation of preprocessing and calibration procedures, code description and user manuals.

Planned upcoming outputs

• After ongoing testing and calibrations, a preset of filters and optimal values for SWOT data.
• A curated dataset.
• No formal publication was completed within the current collaboration window. However,  contributions focused on the preprocessed dataset and filtering methodology are under preparation.