Skip to content

rabotnev/DsLab26S_infrared.city-2

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

45 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Urban Heat Hotspot Mapping via Landsat LST Downscaling

Statistical downscaling of Land Surface Temperature (LST) from the 30 m resolution of Landsat 8/9 down to 10 m, using Sentinel-2 spectral data, OpenStreetMap ground composition and ERA5-Land climate variables. The models are trained on 30 m grids where both the predictors and the LST target are known, and then applied to 10 m predictors to estimate LST at a resolution no satellite currently measures directly.

Developed for Data Science Lab 4806 at WU Vienna in collaboration with the project partner infrared.city. Three cities are covered to span different climates: Vienna (temperate oceanic, Köppen Cfb), Barcelona (hot-summer Mediterranean, Csa) and Hong Kong (humid subtropical, Cwa).

Team: Laurids Aichmayr, Luca Boesso, Daniil Rabotnev, Sahar Saffi.

Website.Flask.App.mov

Data

All sources use the warm-season window 2023-06-01 to 2023-09-30 for every city, so satellite and climate data are temporally aligned.

Source Resolution Role
Landsat 8/9 LST (ST_B10, Collection 2 L2) 30 m Target variable and master grid
Sentinel-2 SR (B2–B4, B8, B11, B12 + NDVI/NDBI/NDWI) 10 m Spectral predictors
OpenStreetMap (buildings, land use, roads, natural, leisure) vector Fractional ground cover per cell
ERA5-Land (2 m temperature, relative humidity, 10 m wind) ~11 km Seasonal climate context

Landsat and Sentinel-2 are cloud/shadow-masked and reduced to a warm-season median composite. Sentinel-2 is aggregated to the 30 m grid before the spectral indices are computed. OSM vectors are rasterized as fractional coverage (a 3×3 sub-pixel average per cell); missing OSM classes are treated as absence (0), not missing data. ERA5-Land is reduced to seasonal means and assigned to each grid cell by nearest neighbour. Area bounding boxes and per-city UTM projections are defined in CONFIG.py.

Requirements

Python 3.10 or newer and a Google Earth Engine account for the acquisition stage.

python -m venv .venv && source .venv/bin/activate
pip install earthengine-api geemap osmnx geopandas rasterio numpy pandas \
            scikit-learn xgboost joblib jupyter nbconvert matplotlib
earthengine authenticate   # once per machine

Model training runs on CPU across all cores (scikit-learn / XGBoost n_jobs); no distributed framework is required.

Running the pipeline

python 00_run_pipeline.py                      # default: processing -> reporting (04 -> 99)
python 00_run_pipeline.py --stage acquisition  # 01 -> 03 (then download the S2 exports)
python 00_run_pipeline.py --from 09 --to 13    # an arbitrary range of steps
python 00_run_pipeline.py --list               # print the step table and exit

The acquisition stage is separated on purpose: step 01 submits the Sentinel-2 exports as Google Drive tasks, which have to be downloaded into data/raw/s2_30m/ and data/raw/s2_10m/ before processing can continue.

Stage Steps What it does
Acquisition 01–03 Landsat + Sentinel-2 (GEE), OSM (osmnx), ERA5-Land climate
Processing 04–08 Master grid, OSM rasterization, climate features, training table, filtering
Training 09–11 Linear Regression, XGBoost, Random Forest
Validation 12–13 Spatial block CV, cross-city leave-one-out
Downscaling 14–18 10 m grid, features, prediction with intervals, upscaling check
Reporting 19–20, 99 Degradation test, result figures, rule-based output checks

Models

Three algorithms are trained through a shared runner (ml_common.py). Each produces five variants from the same training table: a general model with city one-hot flags (the production model), a general model without those flags (to measure their contribution), and one local model per city. That is 3 × 5 = 15 fitted models. Hyperparameters live in CONFIG.py (XGBoost: 300 trees, depth 8, learning rate 0.05, 80% row/column subsampling, L2 = 1.0, histogram method; Random Forest: 120 trees, depth 12).

XGBoost is the strongest model and is used for the final 10 m predictions. The 10 m output also carries a split-conformal prediction interval calibrated from the spatial-block residuals, and a parent-mean correction that anchors each 30 m cell's nine predictions to the observed Landsat LST.

Validation

  • 80/20 split — literature-comparable baseline (optimistic).
  • Spatial block CV (step 12) — 50×50-cell (~1.5 km) blocks hashed into 5 scattered folds; every cell is predicted once out-of-sample. This is the honest within-city metric.
  • Cross-city leave-one-out (step 13) — train on two cities, test on the third using LST anomalies; includes a local-model transfer test.
  • Degradation test (step 19) — train at 60 m, predict 30 m, as a proxy for the 30 m→10 m gap.
  • Upscaling check (step 18) — average 10 m predictions back to 30 m and compare to Landsat.
  • Rule-based output validation (step 99) — range, duplicate and consistency checks.

Headline honest metric (spatial-block CV, XGBoost): RMSE ≈ 1.87 °C, R² ≈ 0.875. Full per-model, per-city metrics are written under ml/<model>/performance/ and visualised in Figures/.

Repository layout

00_run_pipeline.py            Orchestrates the whole pipeline
01..20, 99                    Numbered pipeline steps (.py scripts and .ipynb notebooks)
ml_common.py, osm_common.py   Shared training and OSM-rasterization logic
CONFIG.py                     Cities, CRS, paths, feature schema, hyperparameters
ml/                           Trained models and performance metrics
Figures/                      Result figures
data/                         Raw and processed data (mostly git-ignored)
flask_app/                    Case-study website presenting the results
archive/                      Superseded earlier scripts and notebooks, kept for reference
QGIS_Exports/                 OSM layer exports for QGIS

Website

flask_app/ contains a static case-study site that presents the method and results. See flask_app/README.md for build and deployment instructions.

Limitations

  • Cross-city transfer is weak. Models generalise well within a city but poorly to an unseen one; deploy the city-local models or retrain for a new city.
  • Climate features are spatially coarse. ERA5-Land contributes only a handful of grid points per city, so the climate variables act as a city-level baseline rather than a fine-scale predictor.
  • Cloud cover. Some Vienna cells have no valid warm-season LST and are excluded.
  • No 10 m ground truth exists, so the fine-scale output is validated indirectly rather than against observed 10 m temperatures.

License

Released under the MIT License. See LICENSE.

About

A statistical downscaling pipeline using machine learning (XGBoost, Random Forest) to predict fine-grained Land Surface Temperature (LST) on 10-meter grids. Integrates Landsat 8/9, Sentinel-2 imagery, OpenStreetMap features, and ERA5-Land climate data across Vienna, Barcelona, and Hong Kong to map urban heat islands.

Topics

Resources

License

Stars

1 star

Watchers

0 watching

Forks

Contributors

Languages