Intellibee Visualization

📆 Project Period	November, 2023
📍 GitHub	ESA Portal

Intellibee – Finding ideal bee forage areas using EO

The recent bee mass deaths in the Međimurje, Varaždin, and Bjelovar areas, as well as taken confiscation actions against illegal pesticides, indicate the need of informing beekeepers about the ideal locations for setting up bee colonies. Besides mentioned, it is necessary to implement stricter controls and punish offenders. Can new technologies, using advanced sensor systems and a combination of machine learning, IOT, and satellite image data, provide additional security for beekeepers?

We will train AI to recognize specific plant covers in order to increase the project’s scope and to do something useful for the beekeeping community, based on the satellite image and field data. After the artificial intelligence recognizes them, it compares them with the parameters important for an ideal bee pasture. Besides the bee grazing diversity, we also work on a system of assessing human activity (mostly through the distance from inhabited places, factories, and agricultural areas of the application of insecticides and pesticides, then through the presence of unpolluted water and the position’s insolation, humidity and fog, windiness and likewise). Specific GIS parameters are also considered, such as topography, land ownership, and land accessibility through existing infrastructure.

Recognition and mapping pasture for bees project was also supported by the European Space Agency, whose satellites, and Earth Observation systems we use as part of this project. This is a pilot project in Croatia through which we can gather enough knowledge and experience that will facilitate the project’s expansion on the entire European Union territory.

The final result will be an interactive map or application to support beekeepers that will facilitate grazing planning, as well as management of hives and yields. All with the aim of beekeepers ultimately developing a strategy for sustainable honey production by combining satellite and sensor data, supported by relevant data from the locations in real-time. Mentioned will happen in the spring of 2023, according to our estimates.

Getting started

To use the code and dataset locally, you'll need a few things:

The code from the repository.
The dataset.
The python environment set up.
Jupyterlab to run the notebook.

Cloning the repository

git clone https://gitlab.esa.int/ai4eo/esa-philab-group/intellibee.git

Installing the environment

conda create -n env_name
conda activate env_name
conda install ipykernel
python -m ipykernel install --user --name=env_name

Importing the required libraries

conda install pip
pip install -r requirements.txt

Running the notebook locally

jupyter lab
(select the kernel to match the environment setup for the project)

Managing the data

xgboost_model = joblib.load("../data/xgboost/xgboost_model_20230623095028.joblib")
gnn_model = load_model("../data/gnn/gnn_final_model_2023-06-06_00-55-40.h5")
image_Krog = rasterio.open("../data/sentinel_images/Krog - 7/2018-06-19/response.tiff")

Code

import pandas as pd
import numpy as np
import joblib
from keras.models import load_model
import rasterio
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import classification_report, accuracy_score, confusion_matrix, cohen_kappa_score
import math
import matplotlib.patches as mpatches

xgboost_model = joblib.load("../data/xgboost_model_20230623095028.joblib")
gnn_model = load_model("../data/gnn_final_model_2023-06-06_00-55-40.h5")

df = pd.read_csv("../data/updated_training_dataset.csv")
label_encoder = LabelEncoder()
df["y_true"] = label_encoder.fit_transform(df["Class"])
classes_mapping = {c: m for c, m in zip(label_encoder.transform(label_encoder.classes_), label_encoder.classes_)}

df_Krog = df.loc[(df["name"].str.startswith("Krog - 7")) & (df["timestamp"] == "2018-06-19")]
df_Krog.reset_index(inplace=True, drop=True)

features = ["ndvi", "ndmi", "temperature", "humidity"]

def get_predicted_values(shape, pred_values, pred_df):
    result = []
    for i in range(shape[0] * shape[1]):
        result.append(pred_values[pred_df.loc[pred_df["pixel_id"] == i].index[0]] if not pred_df.loc[pred_df["pixel_id"] == i].empty else math.nan)
    result = np.array(result).reshape(shape[0], shape[1])
    return result

def plot_image(pred, title):
    image = pred.copy()
    image[np.isnan(image)] = 3
    image = [[int(j) for j in i] for i in image]
    # palette = np.array([[255,0,0], [0,255,0], [0,0,255], [255,255,255]])
    palette = np.array([[113,198,113], [255,236,139], [255,69,0], [255,255,255]])
    image = palette[image]
    im = plt.imshow(image)
    values = np.unique(pred[~np.isnan(pred)])
    # colors = [[1,0,0], [0,1,0], [0,0,1]]
    colors = [[113/255, 198/255, 113/255], [255/255, 236/255, 139/255], [255/255, 69/255, 0/255]]
    patches = [ mpatches.Patch(color=colors[int(i)], label="Class {}".format(classes_mapping[int(i)]) ) for i in values ]
    plt.legend(handles=patches, bbox_to_anchor=(1.01, 1), loc=2, borderaxespad=0. )
    plt.title(title)
    plt.show()

Feature Importance

plt.bar(range(len(xgboost_model.feature_importances_)), xgboost_model.feature_importances_)
plt.xticks(range(len(xgboost_model.feature_importances_)), ["ndvi", "ndmi", "temperature", "humidity"])
plt.title("XGBoost feature importance")
plt.show()

importance_scores = np.abs(gnn_model.layers[1].get_weights()[0]).sum(axis=1)

feature_names = ["ndvi", "ndmi", "temperature", "humidity"]
importance_df = pd.DataFrame({"Feature": feature_names, "Importance": importance_scores})

# importance_df = importance_df.sort_values("Importance", ascending=False)

# plt.figure(figsize=(10, 6))
plt.bar(importance_df["Feature"], importance_df["Importance"])
plt.xlabel("Feature")
plt.ylabel("Importance")
plt.title("GNN feature Importance")
plt.show()

Krog - 7, 19.6.2018.

image_Krog = rasterio.open("../data/sentinel_images/Krog - 7/response.tiff")

plt.subplots(1,2, figsize=(10,4))
plt.subplot(1,2,1)
plt.imshow(image_Krog.read(4))
plt.colorbar()
plt.title("NDVI")
plt.subplot(1,2,2)
plt.imshow(image_Krog.read(5))
plt.colorbar()
plt.title("NDMI")
plt.show()

print("Average temperature: {}".format(df_Krog["temperature"][0]))
print("Average humidity: {}".format(df_Krog["humidity"][0]))

#Average temperature: 22.5625
#Average humidity: 58.75

y_pred_xgboost = xgboost_model.predict(df_Krog[features])

features_krog = df_Krog[features]
features_mean = [0.68878916, 0.22068822, 21.38213684, 64.49685305]
features_std = [0.21463931, 0.16673503, 5.26209807, 9.97340007]
features_normalized_krog = (features_krog - features_mean) / features_std

y_pred_gnn = gnn_model.predict(features_normalized_krog)
y_pred_gnn = np.argmax(y_pred_gnn, axis=1)

pred_xgboost = get_predicted_values(image_Krog.shape, y_pred_xgboost, df_Krog)
pred_gnn = get_predicted_values(image_Krog.shape, y_pred_gnn, df_Krog)

fig, ax = plt.subplots(1,2, figsize=(10,4))
plt.subplot(1,2,1)
plot_image(pred_xgboost, "XGBoost, Krog - 7, 19.6.2018.")
plt.subplot(1,2,2)
plot_image(pred_gnn, "GNN, Krog - 7, 19.6.2018.")

Project Description

[put here]