This project was developed by the CIN Researcher Evgenios Tsigkanos

Project Summary

• Foundation models
• Time-critical applications
• Dataset preparation
• Verification and Validation of ML models for on-board

Development Tools

• Tensorflow
• Pytorch
• Python
• Rasterio
• Zarr

Development Outputs

• Upcoming paper
• ΦsatNet: A Deployable Foundation Model for Onboard AI on Φsat-2 (unpublished)

Project Description

The present project first addresses a pivotal challenge in Earth observation (EO) research and applications: the consolidation and harmonization of multi-modal remote sensing datasets, specifically for time-critical scenarios, relevant for on-board use and specifically relevant for the Φ-sat-2 satellite. These include, but are not limited to, methane detection, vessel detection, fire detection, and flood detection. The overarching objective is to streamline heterogeneous data sources and tailor them for on-board usage, enabling high-throughput, real-time analyses in spaceborne missions. In particular, the project focuses on harmonizing disparate datasets into standardized, efficient data structures that facilitate both immediate operational utility and long-term reusability. By ensuring consistency in file formats and metadata, the effort significantly reduces the cognitive and computational overhead typically associated with cross-domain data integration.

A core component of this initiative involves developing advanced preprocessing pipelines to accurately simulate the radiometric and geometric characteristics of the forthcoming Φ-sat-2 satellite sensor. These pipelines encompass a wide spectrum of transformations: from emulating band misalignment and synthesizing panchromatic bands to simulating reflectances, radiances, and signal degradation via signal-to-noise ratio (SNR) and modulation transfer function (MTF) models. Spatial resampling and patch-based chip creation further refine the datasets to match the sensor’s specific spatial resolution and coverage parameters. By aligning these transformations with known instrument performance and operational constraints, the project ensures that the preprocessed data realistically mirrors the conditions anticipated in orbit.

Subsequent to preprocessing, a suite of dataloaders is being engineered to seamlessly interface with modern AI frameworks. These dataloaders are optimized for diverse model architectures, with an emphasis on foundation models that offer broad representational capabilities applicable to multiple downstream tasks. Through fine-tuning these large-scale models on the curated datasets, the project aims to develop robust classifiers, detectors, and segmenters, each specialized for critical applications in EO analytics. By leveraging transfer learning, the endeavor enhances model performance on tasks that require rapid inference and high reliability, aligning with the operational demands of real-time, on-board satellite processing.

Finally, the project places considerable emphasis on the rigorous verification and validation (V&V) of the resulting models. This includes both algorithmic assessments—quantifying robustness, interpretability, and domain adaptability—and operational testing under scenarios mimicking satellite telemetry constraints.