Project Summary

Over an almost two‑year fellowship as Internal Research Fellow (July 2022 – June 2024) at ESA’s Φ‑lab in Frascati, Casper Fibaek used AI and computer vision to advance Earth‑observation analytics, targeting global school‑mapping for UNICEF‑GIGA and the development of geospatial foundation models. The work yielded operational tools, open resources and strengthened Φ‑lab’s international partnerships. In particular, he:

• Built a four‑phase school‑mapping pipeline and a real‑time human‑in‑the‑loop toolbox; achieved 89.3 % (Phase I) and 83.2 % (Phase III) precision.
• Supervised two PhD interns and created an operational school mapping prototype.
• Co‑created PhilEO‑Bench and started the Foundation Model Suite for mixture‑of‑experts pre‑training; created a benchmark labelled 500 GB Sentinel‑1/2 corpus and secured paper acceptances at IGARSS 2024 and EGU 2024.
• Released PhilEO Downstream and VIIRS night‑lights trend datasets, and enhanced the Buteo, Sen2Sharp, and Spatial Label Smoothing open‑source packages.
• Initiated new partnerships with Leonardo Labs, VITO and NASA; supported the SDGSat workflow and ESRIN‑wide AI outreach. Contributor to the MIT/WEF report on “Charting the Future of Earth Observation”
• Presented findings at Teach w/ Space, taught at the Climate Change AI Summer School, ESSAI and EGU, engaging researchers, students, and educators.

Development Tools

Casper Fibaek combined deep learning frameworks with Earth-Observation (EO) toolchains throughout the research fellowship. PyTorch powered most development, model prototyping, and the human-in-the-loop retraining service. He initially used TensorFlow as he came from an industry background where that was originally the standard. Performance-critical routines, such as morphological filtering, were accelerated with Cython or Numba. For EO pre-processing he relied on ESA’s SNAP Toolbox (Sentinel-1/-2 calibration, speckle filtering, terrain correction) and the Orfeo Toolbox for large-scale segmentation and feature extraction. Casper Fibaek orchestrated all the components through the Buteo Toolbox, which he extended with automated co-registration and dataset alignment to guarantee reproducible pipelines across projects.

This software stack allowed me to conduct research from raw satellite imagery to deployable AI services, supporting both the GIGA school-mapping workflow and the foundation-model benchmarking while remaining portable between local GPUs and the precursor EOHPC cluster.

Core tools: PyTorch, TensorFlow, Cython & Numba, ESA SNAP Toolbox, Orfeo Toolbox, Buteo Toolbox.

Development Outputs

C. Fibaek, L. Camilleri, A. Luyts, N. Dionelis, and B. Le Saux, “PhilEO bench: Evaluating geo-spatial foundation models,” in Proc. IGARSS 2024 IEEE Int. Geosci. Remote Sens. Symp., 2024, pp. 2739–2744.

N. Dionelis, C. Fibaek, L. Camilleri, A. Luyts, J. Bosmans, and B. Le Saux, “Evaluating and benchmarking foundation models for earth observation and geospatial AI,” arXiv preprint arXiv:2406.18295, 2024.

C. Fibaek, A. Riley, K. Doerksen, and R. Schneider, “A multi‑tiered human‑in‑the‑loop approach for interactive school mapping using Earth observation and machine learning,” Int. J. Appl. Earth Obs. Geoinf., unpublished, 2025 (not yet submitted for publication).

S. J. Köhn, A. C. M. Fernandes, C. S. Fibæk, and K. Nielsen, “Mapping of small-scale surface water in Denmark—using deep learning, fusing high-resolution geo data and Sentinel-1 SAR,” Advances in Space Research., unpublished, 2025 (Submitted, under 2nd review).

B. Le Saux et al., “The PhilEO geospatial foundation model suite,” in Proc. EGU General Assembly Conf. Abstracts, 2024, p. 17934.

K. Doerksen, C. Fibaek, R. Schneider, D.-H. Kim, and I. Tingzon, “Predicting internet connectivity in schools: A feasibility study leveraging multi-modal data and location encoders in low-resource settings,” arXiv preprint arXiv:2412.12188, 2024.

R. Musto et al., “GeoDINO: A vision foundation model for earth observation leveraging DINO architecture and Sentinel-2 multi-spectral data,” unpublished, 2025.

N. Dionelis et al., “Generalist geospatial foundation model PhilEO on satellite Sentinel-2 MajorTOM multi-spectral data,” unpublished, 2025.

World Economic Forum, “Charting the Future of Earth Observation: Technology Innovation for Climate Intelligence,” white paper, Geneva, Switzerland, 16 Sep. 2024. [Online]. Available: https://www.weforum.org/publications/charting-the-future-of-earth-observation/. [Accessed: 1‑April‑2025].

Fibaek, Casper. (2024). Buteo: Geospatial Data Analysis Framework for AI/EO. Zenodo. https://doi.org/10.5281/zenodo.7936577

Fibaek, Casper. (2023), “Super‑sample Sentinel‑2,” GitHub repository, 2023. [Online]. Available: https://github.com/casperfibaek/super_res_s2. [Accessed: 1‑April‑2025].

Fibaek, Casper. (2023), “Spatial Label Smoothing,” GitHub repository, 2023. [Online]. Available: https://github.com/casperfibaek/spatial_label_smoothing. [Accessed: 1‑April‑2025].

Project Description

Over an almost two‑year fellowship (July 2022 – June 2024) at ESA’s Φ‑lab in Frascati, Casper Fibaek investigated how artificial intelligence for Earth‑observation (EO) analytics. His research centered on two flagship initiatives: A global school‑mapping pipeline for the UNICEF‑GIGA programme and the exploration of geospatial foundation models. Together, these two projects delivered operational tools, open datasets, scientific contributions and new partnerships that extend Φ‑lab’s impact beyond ESA.

UNICEF‑GIGA School‑Mapping Pipeline

The GIGA project seeks to locate every school worldwide, assess its connectivity, and inform equitable internet roll‑out. Working with UNICEF and ESA, Casper Fibaek co‑designed a four‑phase workflow:

• Phase I – Data curation (cleaning and gap analysis of existing school registries);
• Phase II – Sentinel‑based candidate discovery (ongoing);
• Phase III – Very‑high‑resolution (VHR) screening of candidates;
• Phase IV – Connectivity inference using VHR imagery and auxiliary data.

Casper Fibaek supervised two PhD interns, ensuring continuous progress while on‑boarding deep-learning best practices. By the project’s close, Phases I, III and IV had reached precisions of 89.3 % and 83.2 % respectively, demonstrating the feasibility of satellite‑first school detection at scale.

To maximise stakeholder buy‑in, they developed a human‑in‑the‑loop toolbox combining a Flask REST‑API, Leaflet front‑end and PyTorch backend. Users can retrain the model interactively, steer attention to unmapped regions and export results directly to for use in operational databases.

The interface of the Human-in-the-loop approach developed.

The collaboration yielded three manuscript submissions (ICLR‑ML4RS, IJCAI‑AI4SG, IJ‑AEO submission), expanded GIGA’s EO capacity and provided ESA with a reusable template for socially driven, infrastructure mapping AI projects. A journal article is being readied for submission.

Geospatial Foundation Models & PhilEO Suite

In year two, Φ‑lab and Leonardo Labs started working on the emerging field of EO foundation models (FMs). Confronted with the prohibitive cost of training monolithic networks, they proposed a mixture‑of‑experts (MoE) strategy in which multiple specialised backbones share routing layers. To test this idea, Casper Fibaek spearheaded PhilEO‑Bench — a reproducible evaluation harness that contrasts FM outputs against fully supervised baselines on a 500 GB Sentinel‑1/‑2 corpus—and co‑authored the PhilEO Foundation Model Suite.

Results were presented at EGU 2024 and at IGARSS 2024, positioning Φ‑lab at the forefront of FM research for remote sensing. Based on discussions with NASA at the EGU 2024, a collaboration was also set up on investigating joint development of future foundation models between ESA/NASA.

General layout of the proposed Mixture of Experts’ approach

The work started new partnerships with VITO and NASA’s IMPACT team, who are now actively collaborating with the PHILAB on informing the future development of geospatial foundation models.

Supplementary Contributions

• Open‑source software: continuous upgrades to Buteo, Sen2Sharp, and Spatial Label Smoothing, plus a Python re‑implementation of the GeFolki co‑registration algorithm used in the SDGSat night‑lights study.
• Datasets: the 500 GB PhilEO‑Downstream benchmark and a VIIRS night‑lights trend‑normalised archive for rural electrification research.
• Community outreach: Sentinel‑based flood‑mapping tutorials for the Climate Change AI Summer School, an ESA‑wide AI update, and presentations at Teach w/ Space, ESSAI, EGU and IGARSS. Contributor to the WEF/MIT white paper on the future of earth observation.

Outcomes of the Φ‑lab Collaboration

• Operational impact: UNICEF received a ready‑to‑deploy mapping service; Φ‑lab gained a modular AI pipeline that can be repurposed for other infrastructure mapping projects.
• Capacity building: Training of YGTs, PhD interns and external partners in advanced EO‑AI workflows.
• Strategic alliances: Formalised collaborations with Leonardo Labs, VITO and NASA, expanding ESA’s innovation ecosystem. They also investigated a formal partnership with ITU on infrastructure mapping based on the approaches learned.
• Sustainable tooling: Robust, open‑source packages and datasets that lower entry barriers for future Φ‑lab projects.

The fellowship demonstrated how targeted AI research, embedded within an agile innovation hub like Φ‑lab, can translate rapidly from satellite pixels to societal benefit.