Project Summary

• Semi-supervised learning enables the creation of robust models for hyperspectral image analysis, even with limited training data – our semi-supervised pipeline fine-tunes a deep residual model with just 10 labeled samples/class, providing high-quality multi-class classification.
• The deep architecture remains unaffected by semi-supervised training, making it possible to design resource-efficient AI models for on-board deployment.
• Feature pooling can enhance soil estimation, and our pipeline eliminates the need for manual feature engineering.
• We developed and validated an end-to-end, fully unsupervised pipeline for soil parameter estimation from hyperspectral images. It uses unsupervised clustering to remove noise and can be deployed in real-world environments without retraining. This is the first fully unsupervised pipeline for estimating soil parameters.
• Pseudolabels from our superpixel-powered processing pipeline can be used to train supervised models, thereby reducing the impact of noisy pixels.
• The superpixel-powered pipeline is versatile and can be applied to any Earth observation task and image modality.
• Band selection impacts the unsupervised analysis, with key spectrum parts improving soil parameter estimation. Proper band selection reduces data volume, accelerates processing, and is important for resource-constrained applications such as on-board satellites.
• Superpixel analysis provides more detailed soil information from spectral data, but further validation with in situ measurements is required.

Development Tools

IC–EO converts natural-language queries into executable Earth Observation (EO) workflows. The framework combines a modular API of selected EO tools with a LLM that plans and writes code. This approach ensures interpretability, reproducibility, and extensibility across tasks.

The experimental evaluation of IC–EO is designed to assess the full system protocol: how the assistant integrates EO models, helper tools, and a code-generating controller to produce valid, interpretable answers from natural-language queries.

Development Outputs

• Hugging Face: The artifacts related to our pseudolabels-powered processing pipeline: https://huggingface.co/KPLabs/WeaklyLabeledHYPERVIEW/tree/main

Project Description

Introduction

Current technological advancements offer exciting possibilities in hyperspectral image (HSI) analysis across various Earth observation (EO) apps. To extract insights from this highly dimensional data, a range of machine learning (ML) pipelines has been developed. Although real-world data lacks “labels” and obtaining ground truth (GT) data aligned with HSI is costly and difficult to scale, numerous supervised methods have been proposed for EO-related tasks. There are techniques to handle limited GT data, including transfer learning, semi- and self-supervised learning, and HSI data augmentation (at both training and inference-time).

In some cases, only coarse-grained GT is available, such as when airborne or satellite imagery has low spatial resolution or specific in situ measurements are missing or scarcely obtained. Soil parameter estimation serves as a key example here. In this task, soil samples are collected in the field, then mixed and sent to the laboratory for chemical analysis. GT data in this context comprises soil parameters of interest derived from samples collected in each field. This weakly-labeled, image-level GT is linked to the HSI of a field, which may include hundreds or thousands of pixels. Since sampling points are limited and spread across wide areas, image-label GT tuples ultimately relate to diverse (and likely noisy) pixels, not all of which contribute equally to lab-validated parameters.

In this project, we aimed to address the challenges associated with building ML models using weakly-labeled datasets for both regression and multi-label classification downstream tasks. Our primary goal was to develop new algorithms for this purpose and thoroughly validate them with our real-life soil analysis dataset. We anticipate that this project will be an important step toward creating AI models capable of generalizing well from image-level GT labels, which are more feasible and economical to obtain in real EO scenarios.

Overall, the main project objectives (POs) of this activity were as follows:

PO.1: To develop classic and deep machine learning algorithms for high-quality soil analysis in the weakly-labeled scenario (at least one in both classic and deep machine learning approaches).

PO.2: To develop training data selection, augmentation & enhancement approaches for weakly-labeled hyperspectral image data to improve the generalization capabilities of machine learning models.

PO.3: To thoroughly validate the approaches developed within this project using the real-life soil analysis (HYPERVIEW: https://ieeexplore.ieee.org/document/10526314) dataset.

PO.4: To widely disseminate the results obtained in this project.

Project Background

Machine learning on multi- and hyperspectral images provides rapid access to critical information with fewer resources than traditional in situ sampling, which can be costly and time-consuming. AI methods are widely used to detect visible objects (such as infrastructure, ships, and so forth) and invisible substances (gases, pollution) in images, with visible objects generally easier to identify. Detecting non-homogeneous, irregular targets like gases or pollution is more complex due to environmental factors.

Machine learning models can classify, segment, or perform regression on spectral images, but developing accurate models requires high-quality data and robust validation. Supervised learning relies on GT data, which is challenging to obtain for invisible objects and often includes labeling errors due to variability within an area. Weakly supervised learning (WSL) addresses these challenges by using weakly-labeled datasets (e.g., image-level labels) and leveraging data properties to improve predictions. WSL is especially valuable when labeling is costly or incomplete, as it enables semi-automatic labeling. Interest in WSL for object detection using spectral data has grown, with recent taxonomies classifying WSL approaches for limited-access HSI data, including sample generation, limited-sample training, unsupervised algorithms, and spectral-spatial analysis.

Soil composition analysis has seen limited use of WSL, which is mainly applied to detect visible objects. However, the HYPERVIEW dataset (introduced in our other activities and used in the HYPERVIEW Challenge organized with ESA), which includes image-level labels for irregular, invisible objects, suggests that unsupervised or semi-supervised methods could assess soil parameters influenced by environmental factors. This activity focused on developing methods and tools for building well-generalizing models from image-level labels. Of note, there are a variety of practical challenges directly related to how ground-truth data is generated across tasks – we tackled such challenges in this project as well, hypothesizing that some parts of the ground truth might be noisy and that, e.g., pruning them could help improve the models' capabilities.

Methodology

Semi-supervised Learning for Soil Analysis

In this research, we extended the MSMatch pipeline (from the FixMatch family) to hyperspectral images for soil analysis. This is the first application of a semi-supervised algorithm based on FixMatch to process hyperspectral data for identifying soil parameters. The semi-supervised training used pseudo-labeling and consistency regularization: pseudo-labels were generated for unlabeled data, and the model was trained to produce consistent outputs for similar inputs. We used a ResNet-18 model, pre-trained on ImageNet, with slight adjustments for the soil dataset.

Weakly Supervised Learning for Soil Analysis

We explored weakly supervised learning using the WELDON framework for automated soil analysis from hyperspectral images. This involved designing, implementing, and validating several deep learning models. The architecture included data processing with UNet++, followed by WELDON pooling, and a linear layer for regression. Various hyperparameters were tested, with only one run per configuration to estimate performance. We further examined WELDON pooling with these variations:

• UNet++ outputting one feature map per pixel instead of 150 (where 150 is the number of bands available in the HYPERVIEW dataset).
• WELDON pooling outputting min and max values instead of summing them.
• Resizing patches to 32x32 during pre-processing.

Superpixel-Powered Analysis for Soil Analysis

Soil parameter data is available only at the image level, limiting spatial detail within plots due to mixed soil samples. To explore spatial variations, we developed a superpixel-based analysis pipeline using unsupervised clustering to examine the consistency and distribution of superpixels across images. This approach aimed to identify superpixel clusters in both the training and test subsets of the HYPERVIEW dataset. In this pipeline, we can perform fully unsupervised extraction of soil parameters from hyperspectral images, elaborate pseudolabels based on the superpixel analysis (such pseudolabels may be useful to prune noisy pixels from the hyperspectral images that may not contribute to correct predictions, or may even deteriorate the performance of machine learning algorithms) or perform additional exploratory analysis to determine groups of heterogeneous superpixels/areas within the captured image data. To our knowledge, our pipeline is the first in the literature to build upon such unsupervised techniques, which can be used seamlessly in an end-to-end fashion to extract soil parameters and may also be conveniently deployed to enhance supervised machine learning models. All approaches developed in this activity have been thoroughly verified through quantitative, statistical, and qualitative analyses. In Figure 1, we present the most important steps of the developed end-to-end pipeline.

Figure 1: Generating pseudolabels (PLs) for hyperspectral images can help improve downstream analysis in both unsupervised and supervised schemes. We present an (A) RGB image, the bare soil mask, and an in situ sampling strategy to extract GT (B, C, D). Superpixels (E) obtained for the full spectrum (or its subpart F) are used to retrieve PLs (G), later used in (un)supervised pipelines (I) to extract soil parameters (J) from HSI (H). In the I panel, we render an optional step of building separate machine learning models for each soil parameter (alternatively, it is possible to design multitarget regression models). For each pivotal step in the pipeline, we indicate the corresponding section of our paper (https://ieeexplore.ieee.org/abstract/document/11370490) that discusses it in detail.

Conclusion

This activity tackled practical challenges directly related to the availability of ground-truth data in Earth observation. It highlighted the potential of semi-supervised learning to develop robust hyperspectral image analysis models, even with limited training data. Our semi-supervised pipeline demonstrated the ability to fine-tune deep models with just 10 labeled samples per class, offering high-quality multi-class classification and enabling resource-efficient AI solutions for on-board deployment. By employing appropriate feature pooling, our approach enhanced soil estimation, eliminating the need for manual feature engineering. Additionally, we introduced the first fully unsupervised pipeline for soil parameter estimation, using unsupervised clustering (superpixels) to reduce noise and enable more flexible deployment across various environments. Pseudolabels generated by this pipeline could be used to train supervised models, improving performance while reducing the impact of noisy pixels. The superpixel-powered pipeline was versatile, applicable to any Earth observation task, and enabled efficient band selection, thereby optimizing data processing for hardware-constrained platforms. While the analysis provided detailed insights into soil parameters, further validation through in situ measurements was necessary to achieve more accurate results.

We believe that the algorithms, technology, and tools developed in this activity will be an important step toward addressing limited ground-truth datasets in the Earth observation field. The approaches developed within this activity may accelerate the adoption of AI for such tasks, for both on-ground and on-board processing, with the latter aimed at reducing the amount of data to be transferred from, e.g., an imaging satellite, by extracting the most important and actionable insights for end users.

Publications

• A. M. Wijata, L. Tulczyjew, P. Naylor, B. Le Saux, N. Longépé and J. Nalepa, "Getting the Most Out of the Image-Level Labels: (Un)Supervised Learning for Extracting Soil Parameters From Hyperspectral Images," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 19, pp. 7401-7418, 2026, doi: 10.1109/JSTARS.2026.3660363, https://ieeexplore.ieee.org/abstract/document/11370490
• B. Grabowski, A. M. Wijata, L. Tulczyjew, B. Le Saux and J. Nalepa, "Soil Analysis with Very Few Labels Using Semi-Supervised Hyperspectral Image Classification," IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium, Athens, Greece, 2024, pp. 407-411, doi: 10.1109/IGARSS53475.2024.10642465, https://ieeexplore.ieee.org/document/10642465
• A. M. Wijata, P. Naylor, and J. Nalepa, " Learning from Image-Level Labels: Where Unsupervised and Supervised AI Algorithms Come Together," Living Planet Symposium 2025, Vienna, Austria, https://lps25.esa.int/programme/programme-session/?id=5345408F-6932-4CBD-8587-1F8BB229ADD6&presentationId=0C312308-4D8D-454B-9ABF-2585620E1D45