Chincha Norte — Baselines for Recovery (2020 → 2025)

Why this project?

This repo is a reproducible “value demo” to support island–ocean restoration work around Chincha Norte (Perú), part of the Guano Islands system. In June 2025, Chincha Norte joined the Island–Ocean Connection Challenge (IOCC)—a global initiative led in part by Island Conservation (IC) to restore island ecosystems “ridge-to-reef.” That makes Chincha Norte an ideal, concrete AOI for transparent pre-intervention baselines that can be reused as recovery unfolds. IOCC/IC news.

There's also urgent ecological context: Peru's coastal guano-bird populations dropped by >75% from ~4 M (2022) to ~0.5 M (2025), driven by avian flu (2022), El Niño (2023), and anchoveta pressure (2024). A clear, repeatable baseline helps target monitoring and future validation. Reuters.

Goal (MVP)

Deliver three practical baselines + context framing:

Guano-colony stain proxy (baseline + Δ 2020 → 2025)
Seasonal Sentinel-2 medians, NDVI/NDWI, and a conservative fresh/aged guano-candidate proxy (with a binary candidate mask). This is exploratory, rocky substrates are challenging, yet grounded in literature that shows distinct guano spectral signatures for dense surface-nesters. Remote Sensing of Environment paper.
Near-island vessel-activity index (neutral pressure layer)
Monthly vessel presence/activity counts within ~30 km of the island (2020 → 2025), with a time-series + coarse heatmap. This layer is descriptive (no claim about gear/type), to be interpreted with local knowledge. GFW API docs.
Terrain & sensor-siting map (field-planning win)
From Copernicus DEM GLO-30: slope/aspect/roughness + distance-to-coast, producing 10 candidate audio-sensor locations (GeoJSON) with a brief rationale table. Copernicus DEM on AWS.
Predator-removal recovery framing (context)
(1–3) are pre-intervention baselines relative to rat-removal work starting in Sep 2025 under IOCC, so later acoustic counts/imagery can be compared.

Data

Sentinel-2 L2A COGs (AWS) global, ~10 m, ideal for seasonal medians. AWS Registry.
STAC API (Earth Search) via pystac-client for filtered discovery. Docs.
Copernicus DEM GLO-30 for terrain layers. AWS Registry, OpenTopography.
Global Fishing Watch public global presence (API extraction over AOI window, daily rows aggregated to monthly). GFW API docs.

Data attribution

Global Fishing Watch (GFW): vessel-presence inputs come from GFW public datasets via API.
Copernicus DEM GLO-30: terrain layers are derived from Copernicus DEM tiles accessed via AWS Open Data.

Method (MVP)

A) Guano-colony stain proxy

AOI: data/aoi/aoi_chincha_norte.geojson (EPSG:4326) with 500 m buffer.
Query: pick comparable seasonal windows for 2020 vs 2025 (current default: Jan–Mar both years), clouds <20% via STAC.
Composites: lazy COG read with xarray/rioxarray; median per band B02/B03/B04/B08/B11.
Indices: NDVI, NDWI + a conservative fresh/aged guano-candidate proxy from visible bands (POC).
Δ-mask: compute 2020 → 2025 change on the proxy; percentile thresholds + light morph cleanup.
Outputs: PNGs (per-year + Δ), GeoTIFF Δ, and 5–8 “tiles” for quick review.

Rationale: seabird guano has a distinct spectral signature detectable in multi-spectral imagery; transferability to rocky deserts is uncertain → mark as POC. RSE/guano.

A.1) Guano proxy (POC) — implemented baseline (`src/indices.py`)

This repository currently implements a conservative two-family proxy (fresh-like + aged-like) and applies strict masking before candidate selection.

Formulas

Base indices:

$$ \mathrm{NDVI} = \frac{NIR - Red}{NIR + Red + \epsilon} $$

$$ \mathrm{NDWI} = \frac{Green - NIR}{Green + NIR + \epsilon} $$

Visible-band features:

$$ vis_mean = \frac{Blue + Green + Red}{3} $$

$$ vis_std = std(Blue, Green, Red), \quad chroma = \frac{vis_std}{vis_mean + \epsilon} $$

$$ red_blue = \frac{Red}{Blue + \epsilon}, \quad yellowish = \frac{Red + Green}{2 \cdot Blue + \epsilon} $$

Score definitions (with per-year z-score normalization):

$$ fresh_score = z(vis_mean) - z(chroma) - z(red_blue) $$

$$ aged_score = z(vis_mean) + z(yellowish) + z(red_blue) $$

Conservative mask logic:

land_filter = (NDVI < 0.20) AND (NDWI < 0.00)
fresh_mask = fresh_score >= q85(fresh_score within land_filter)
aged_mask = aged_score >= q85(aged_score within land_filter)
guano_candidate_mask = land_filter AND (fresh_mask OR aged_mask)

Outputs (current)

Per-year index rasters in data/exports/indices/:
- ndvi_{year}.tif, ndwi_{year}.tif
- guano_fresh_score_{year}.tif, guano_aged_score_{year}.tif
- guano_candidate_mask_{year}.tif (0/1)
Stats exports:
- indices_stats_2020.json, indices_stats_2025.json
- indices_stats_2020_2025.json
- indices_stats_2020_2025.table.csv
Quicklook PNG:
- figures/guano_proxy_indices_2020_2025_quicklook.png

Notebook references and purpose

Main MVP notebook (audience-facing): notebooks/chincha_norte_mvp.ipynb
- A) guano proxy baseline + delta
- B) terrain + sensor-siting
- C) vessel context
Technical smoke-test notebook (developer-facing): notebooks/cnc-2025-baseline-methods-smoke-test.ipynb contains expanded diagnostics and implementation checks across modules

Caveats

This is an exploratory proxy, not pixel-level ecological truth.
Chincha Norte is rocky/coastal-desert; bright rock/dust can produce false positives.
Fresh vs aged interpretation is based on guano appearance/aging guidance; confirm with field/VHR data before ecological claims. (IntechOpen)

A.2) Change detection (guano proxy) — implemented baseline (`src/change.py`)

Change layers are computed as:

$$ \Delta fresh = fresh\_score_{2025} - fresh\_score_{2020} $$

$$ \Delta aged = aged\_score_{2025} - aged\_score_{2020} $$

Thresholding + denoise:

percentile masks on each delta layer (q15 for losses, q85 for gains)
light morphological opening (default: 3x3) to reduce speckle
union masks:
- gain_union_mask
- loss_union_mask
- delta_signal_mask = gain OR loss

Candidate transition classes from binary masks (guano_candidate_mask_2020 vs 2025):

0: stable non-candidate (0 -> 0)
1: loss (1 -> 0)
2: gain (0 -> 1)
3: stable candidate (1 -> 1)

Current change outputs:

data/exports/change/delta_fresh_score_2025_minus_2020.tif
data/exports/change/delta_aged_score_2025_minus_2020.tif
data/exports/change/delta_gain_union_mask_2025_minus_2020.tif
data/exports/change/delta_loss_union_mask_2025_minus_2020.tif
data/exports/change/delta_signal_mask_2025_minus_2020.tif
data/exports/change/delta_transition_class_2020_to_2025.tif
data/exports/change/delta_candidate_mask_2025_minus_2020.tif
data/exports/change/delta_change_stats_2020_2025.json
figures/delta_guano_proxy_2020_2025_quicklook.png

Confidence / limitations (rocky desert substrate):

Deltas are proxy deltas, not direct ecological counts.
False gains/losses can occur from illumination effects, residual cloud/shadow, and bright rocky textures.
Morphological opening removes speckle but can also remove tiny real patches.
Use this as screening/prioritization for follow-up validation, not final ecological attribution.

B) Terrain & sensor-siting

DEM: fetch Copernicus GLO-30 tiles for AOI+buffer.
Derivatives: slope, aspect, roughness; compute distance-to-coast.
Ranking: propose 10 candidate audio-sensor points (GeoJSON) with a tiny rationale table (access, exposure).
Outputs: PNG terrain map + sensor_sites_top10.geojson.

Rationale: fast, actionable planning input for in-situ acoustics. GLO-30.

B.1) Terrain + sensor siting — implemented baseline (`src/terrain.py`)

Current implementation exports:

data/exports/terrain/rasters/terrain_dem_clip.tif
data/exports/terrain/rasters/terrain_slope_deg.tif
data/exports/terrain/rasters/terrain_aspect_deg.tif
data/exports/terrain/rasters/terrain_roughness_m.tif
data/exports/terrain/rasters/terrain_distance_to_coast_m.tif
data/exports/terrain/rasters/terrain_sensor_site_score.tif
data/exports/terrain/sensor_sites_top10.geojson
data/exports/terrain/sensor_sites_top10.csv
data/exports/terrain/terrain_stats.json
figures/terrain_sensor_siting_quicklook.png

C) Vessel-activity index (neutral)

Window: 30 km radius around AOI; months 2020 → 2025.
Counts: monthly activity counts; export CSV + time-series PNG.
Heat: coarse raster/PNG heatmap of cumulative detections.

Rationale: contextual pressure indicator; descriptive only (no claim about gear/type). GFW API docs.

C.1) Near-island vessel activity (GFW presence input) — implemented baseline (`src/vessels.py`)

Current implementation computes monthly detections inside a 30 km radius around Chincha Norte and exports:

data/exports/vessels/vessel_activity_monthly_counts_2020_2025.csv
figures/vessel_activity_monthly_series_2020_2025.png
data/exports/vessels/vessel_activity_cumulative_heat_2020_2025.tif
figures/vessel_activity_cumulative_heat_2020_2025.png
data/exports/vessels/vessel_activity_stats_2020_2025.json

This layer is intentionally neutral/descriptive: monthly vessel-activity context, not fishing-type attribution.

GFW presence caveats (important):

Coverage depends on broadcasting/compliance and reception; not all vessels are represented equally.
This is still a contextual pressure layer, not a direct cause attribution for colony outcomes.
Daytime vessel disturbance around colonies can affect seabirds; treat this as one context layer and combine with local observations. (Canada seabird disturbance guidance)

AOI (Chincha Norte)

File: data/aoi/aoi_chincha_norte.geojson
CRS: EPSG:4326 (WGS84 lon/lat; GeoJSON)
Buffer: 500 m (buffered in projected CRS → reprojected back)
Preview: figures/aoi_chincha_norte.png

Repo structure

chincha-norte-baseline-poc/
├─ notebooks/
│  ├─ chincha_norte_mvp.ipynb
│  ├─ cnc-discovery-stac-collection.ipynb
│  ├─ cnc-preprocess-smoke-test.ipynb
│  ├─ cnc-2025-baseline-methods-smoke-test.ipynb
│  └─ cnc-databricks_sanity.ipynb
├─ src/
│  ├─ stac_query.py
│  ├─ preprocess.py              # seasonal composites
│  ├─ indices.py                 # NDVI/NDWI + guano proxy
│  ├─ change.py                  # 2020 → 2025 delta layers + denoise
│  ├─ vessels.py                 # monthly vessel index + cumulative heatmap
│  ├─ terrain.py                 # DEM derivatives + sensor siting
│  └─ cnc/config.py
├─ scripts/
│  ├─ make_aoi.py
│  ├─ download_gfw_presence.py
│  └─ download_copdem_glo30.py
├─ data/
│  ├─ aoi/aoi_chincha_norte.geojson
│  └─ exports/
│     ├─ composites/
│     ├─ intermediate/
│     ├─ indices/
│     ├─ change/
│     ├─ vessels/
│     ├─ terrain/
│     └─ s2_l2a_items_summer_2020_2025.sample.json
├─ figures/
│  ├─ aoi_chincha_norte.png
│  ├─ seasonal_composite_red_2020_2025.png
│  ├─ guano_proxy_indices_2020_2025_quicklook.png
│  ├─ delta_guano_proxy_2020_2025_quicklook.png
│  ├─ vessel_activity_monthly_series_2020_2025.png
│  ├─ vessel_activity_cumulative_heat_2020_2025.png
│  ├─ terrain_sensor_siting_quicklook.png
│  ├─ mvp_validation_patch_coherence_2025.png
│  └─ mvp_validation_overlay_2025.png
├─ docs/
│  ├─ links.txt
│  ├─ Carrasco-Meza.pdf
│  └─ Conservción en Islas y Puntas Guaneras.pdf
├─ pyproject.toml
├─ Makefile
├─ uv.lock
└─ README.md

How to Reproduce

1) Environment setup

make setup

2) Prepare required local inputs

Required:

Sentinel-2 composites under data/exports/composites/
Copernicus DEM tiles under data/raw/terrain/copernicus_glo30/dem/

Optional:

GFW vessel CSV at data/raw/vessels/gfw_presence_2020_2025.csv

Use the repository scripts when you need to fetch raw inputs:

uv run python scripts/download_copdem_glo30.py
uv run python scripts/download_gfw_presence.py ...

3) Execute the main MVP notebook

make run

4) Export notebook HTML (optional)

make export

Make targets

make help
make setup   # uv sync
make run     # execute notebooks/chincha_norte_mvp.ipynb
make export  # export notebooks/chincha_norte_mvp.html

Run current pipeline components

# 1) STAC discovery (notebook)
# notebooks/cnc-discovery-stac-collection.ipynb

# 2) Seasonal composite preprocessing smoke test
# notebooks/cnc-preprocess-smoke-test.ipynb

# 3) MVP v0 notebook (A+B core, C optional)
# notebooks/chincha_norte_mvp.ipynb

# 4) Technical smoke-test notebook (full diagnostics)
# notebooks/cnc-2025-baseline-methods-smoke-test.ipynb

Published Artifacts

This POC publishes its current deliverables through the repository itself: the main MVP notebook, a small set of final PNGs, and one key GeoJSON output.

Primary artifact

Main notebook: notebooks/chincha_norte_mvp.ipynb
- This is the audience-facing walkthrough of the current MVP.
- It includes:
  - guano proxy baseline + delta
  - terrain + sensor-siting
  - optional vessel context
  - lightweight validation / bias notes

Final PNGs

figures/mvp_a_guano_delta_quicklook.png
- Main optical baseline summary (2020/2025 candidate context + transition view)
figures/mvp_b_terrain_siting_quicklook.png
- Terrain baseline + ranked sensor-siting candidates
figures/mvp_validation_patch_coherence_2025.png
- Validation check showing whether 2025 candidate pixels form coherent patches
figures/mvp_validation_overlay_2025.png
- 2025 RGB composite with candidate-mask overlay for qualitative review

Optional context figures:

figures/vessel_activity_monthly_series_2020_2025.png
figures/vessel_activity_cumulative_heat_2020_2025.png

Final vector output

data/exports/terrain/sensor_sites_top10.geojson
- Top-10 candidate sensor locations from the terrain heuristic
- This is the main portable vector deliverable in the current MVP

How to view

Notebook (.ipynb)
- Open locally in Jupyter/JupyterLab
- Or view directly in GitHub’s notebook renderer
PNGs
- Open locally in any image viewer
- Or preview directly in GitHub
GeoJSON
- Open in QGIS for full GIS use
- Or drag into geojson.io
- Or use GitHub’s GeoJSON preview when browsing the file in the repository

Naming convention

mvp_a_*: optical baseline + delta outputs
mvp_b_*: terrain + sensor-siting outputs
mvp_validation_*: lightweight validation outputs

This project intentionally keeps publishing lightweight: the repository notebook, final PNGs, and the GeoJSON output are the current shareable artifact set.

Companion write-up

A longer narrative article based on this POC was published separately here. This repository remains focused on reproducible code, notebook outputs, and shareable static artifacts.

Reproduction checklist

Install dependencies with make setup.
Ensure required local inputs exist:
- data/exports/composites/*.tif
- data/raw/terrain/copernicus_glo30/dem/*.tif
- optional: data/raw/vessels/gfw_presence_2020_2025.csv
Run the MVP notebook with make run.
Inspect:
- notebooks/chincha_norte_mvp.ipynb
- final PNGs under figures/
- data/exports/terrain/sensor_sites_top10.geojson

Download and prepare vessel input CSV (GFW)

This project expects a local CSV at data/raw/vessels/gfw_presence_2020_2025.csv with columns:

date
lat
lon
hours (estimated vessel-presence effort in each cell-day; used as default weight)
vesselIDs (distinct vessel IDs in each cell-day; optional secondary metric)

Create a GFW account and API token (see docs): GFW API introduction.
Export your token in shell:

export GFW_TOKEN="YOUR_TOKEN_HERE"

Download AOI-window data and write the CSV:

uv run python scripts/download_gfw_presence.py \
  --start 2020-01-01 \
  --end 2025-12-31 \
  --bbox -76.75 -13.95 -76.05 -13.25 \
  --out data/raw/vessels/gfw_presence_2020_2025.csv

The script makes monthly API calls and concatenates rows into one file. By default, src/vessels.py aggregates monthly activity using hours; you can switch to vesselIDs via VesselsConfig(weight_col="vesselIDs") when you want a unique-vessel view.

Download terrain input (Copernicus DEM GLO-30)

Use the repository script to fetch only AOI-overlapping GLO-30 tiles (reproducible and lightweight):

# Dry run: show detected tile prefixes/keys only
uv run python scripts/download_copdem_glo30.py --dry-run

# Download DEM tiles for current AOI
uv run python scripts/download_copdem_glo30.py

# Optional: also download EDM masks
uv run python scripts/download_copdem_glo30.py --include-edm

Defaults:

AOI: data/aoi/aoi_chincha_norte.geojson
Output dir: data/raw/terrain/copernicus_glo30/
- DEM rasters: data/raw/terrain/copernicus_glo30/dem/*.tif
- EDM rasters (optional): data/raw/terrain/copernicus_glo30/edm/*.tif
- Manifest: data/raw/terrain/copernicus_glo30/download_manifest.json

Run indices from Python (scriptable)

from src.indices import GuanoProxyConfig, run_indices_for_2020_2025

results = run_indices_for_2020_2025(
    composites_dir="data/exports/composites",
    out_dir="data/exports/indices",
    cfg=GuanoProxyConfig(quantile=0.85, ndvi_max=0.20, ndwi_max=0.00),
)

print(results[2020]["thresholds"])
print(results[2025]["thresholds"])

Run change layers from Python (scriptable)

from src.change import DeltaChangeConfig, run_delta_change

change = run_delta_change(
    indices_dir="data/exports/indices",
    out_dir="data/exports/change",
    figures_dir="figures",
    cfg=DeltaChangeConfig(low_quantile=0.15, high_quantile=0.85, morph_open_size=3),
)

print(change["stats_path"])
print(change["quicklook_png"])

Run vessels index from Python (scriptable)

from src.vessels import VesselsConfig, run_vessel_activity_index

vessels = run_vessel_activity_index(
    VesselsConfig(
        input_path="data/raw/vessels/gfw_presence_2020_2025.csv",
        # default GFW schema columns: date/lon/lat/hours
        time_col="date",
        lon_col="lon",
        lat_col="lat",
        weight_col="hours",
        output_dir="data/exports/vessels",
        figures_dir="figures",
        radius_km=30.0,
        grid_cell_m=2000.0,
        date_start="2020-01-01",
        date_end="2025-12-31",
    )
)

print(vessels["monthly_csv"])
print(vessels["heat_tif"])

Run terrain + sensor-siting from Python (scriptable)

from src.terrain import TerrainConfig, run_terrain_and_siting

terrain = run_terrain_and_siting(
    TerrainConfig(
        dem_dir="data/raw/terrain/copernicus_glo30/dem",
        out_dir="data/exports/terrain",
        figures_dir="figures",
        candidate_count=10,
        min_spacing_m=250.0,
    )
)

print(terrain["exports"]["candidates_geojson"])
print(terrain["preview_png"])

Re-run later for monitoring (months/years after intervention)

Use this project as a repeatable monitoring protocol by keeping preprocessing/index parameters fixed and only updating the target observation window.

Recommended monitoring checklist:

Keep baseline parameters unchanged:
- AOI geometry
- season window (e.g., Jan–Mar)
- cloud threshold
- index thresholds (ndvi_max, ndwi_max, quantiles)
- morphology size (morph_open_size)
Query and preprocess new scenes for the new year using the same settings.
Generate new per-year composites and index layers.
Compute change against the baseline and compare gain/loss transitions.
Track and store summary metrics over time (gain/loss/stable counts) for trend monitoring.

Current implementation note:

src/change.py currently compares the fixed pair 2020 vs 2025.
For ongoing campaigns, the intended next enhancement is parameterizing baseline and comparison years (e.g., 2025 vs 2026, 2025 vs 2027) while preserving the same thresholds/caveats.

STAC query (Sentinel-2 L2A)

Use src/stac_query.py to fetch low-cloud Sentinel‑2 L2A items for comparable windows (default: Jan–Mar 2020 vs Jan–Mar 2025) over the AOI.

Notes:

This project uses Earth Search STAC (sentinel-2-l2a collection).
Earth Search asset keys map to common Sentinel‑2 bands: blue=B02, green=B03, red=B04, nir=B08, swir16=B11.
A tiny sample export is written to data/exports/s2_l2a_items_summer_2020_2025.sample.json from the discovery notebook.

from pathlib import Path

from src.stac_query import query_s2_l2a_items_2020_2025_summer
from src.cnc.config import AOI_PATH

repo_root = Path(".")
out = query_s2_l2a_items_2020_2025_summer(
    aoi_path=repo_root / AOI_PATH,
    cloud_lt=20.0,
    max_items=20,
)

print({year: len(items) for year, items in out.items()})

Baseline Pipeline Overview

This workflow combines three implemented baseline branches: optical change proxy, vessel activity context, and terrain/sensor siting. Core modules:

src/preprocess.py (AOI-safe composite generation and exports)
src/indices.py (indices/scores/masks and index exports)
src/change.py (2025-2020 deltas, denoise, transition classes, change exports)
src/vessels.py (monthly vessel activity + cumulative heat exports)
src/terrain.py (DEM derivatives + candidate sensor-site ranking)

Pipeline Overview

flowchart LR
    A[STAC items + AOI] --> B[preprocess.py<br/>open_band_lazy -> build_year_stack -> seasonal_median_for_year]
    B --> C[data/exports/composites/*.tif<br/>+ intermediate/*.nc]
    C --> D[indices.py<br/>NDVI/NDWI + fresh/aged scores + candidate masks]
    D --> E[data/exports/indices/*.tif<br/>+ stats json/csv]
    E --> F[change.py<br/>delta scores + q15/q85 masks + 3x3 opening + transitions]
    F --> G[data/exports/change/*.tif<br/>+ delta stats json]
    E --> H[figures/guano_proxy_indices_2020_2025_quicklook.png]
    G --> I[figures/delta_guano_proxy_2020_2025_quicklook.png]
    J[GFW presence CSV] --> K[vessels.py<br/>monthly counts + cumulative heat]
    K --> L[data/exports/vessels/*.csv *.tif *.json]
    K --> M[figures/vessel_activity_*_2020_2025.png]
    N[Copernicus DEM tiles + AOI] --> O[terrain.py<br/>slope/aspect/roughness + coast distance + rank top10]
    O --> P[data/exports/terrain/rasters/*.tif<br/>sensor_sites_top10.geojson/csv + stats]
    O --> Q[figures/terrain_sensor_siting_quicklook.png]

Data Flow Summary

Stage	Function	Input → Output	Key Transformation
1. AOI Preparation	`load_aoi_geom` / AOI utilities	Vector file → normalized geometry	Normalize CRS and validate AOI
2. Composite Build	`open_band_lazy` + `build_year_stack` + `seasonal_median_for_year`	STAC scenes → seasonal composites	Lazy COG read, clip, align, median by year
3. Index Computation	`compute_base_indices` + `compute_guano_features` + `compute_guano_scores` + `build_candidate_masks`	Composite rasters → score/mask rasters	NDVI/NDWI + fresh/aged guano proxy + conservative q85 masking
4. Change Computation	`compute_delta_layers` + `save_change_rasters` + `save_change_quicklook`	2020/2025 indices → delta rasters	2025-2020 deltas, q15/q85 thresholding, light opening, transitions
5. Vessel Context	`run_vessel_activity_index`	GFW CSV → monthly series + cumulative heat	Radius filter (30 km), monthly aggregation, coarse heatmap
6. Terrain + Siting	`run_terrain_and_siting`	CopDEM tiles + AOI → derivatives + ranked sites	DEM mosaic/clip, slope/aspect/roughness/coast distance, top-10 spacing filter

Key Design Principles

Lazy by Default

No pixel data is read until absolutely necessary. Every function returns Dask-backed xarray objects that defer computation until .compute() or I/O. This enables:

Processing datasets larger than RAM
Efficient parallel execution
Interactive exploration without waiting

Enforced Alignment

The pipeline aggressively validates and forces geometric alignment:

open_band_lazy reprojects every scene to identical target_crs/target_res
build_year_stack uses join="exact" to fail if any scene differs
seasonal_median_for_year uses reproject_match to force all bands to identical grid

Misalignment = Error. No silent data corruption.

Cloud-Native Outputs

Final exports are professional-grade COGs matching Sentinel-2 distribution standards:

512x512 internal tiles (optimal for cloud range requests)
DEFLATE compression (smaller files, lossless)
Internal overviews (fast zooming)
One file per band (selective download)

Production Readiness

Comprehensive error handling with specific exception types
Input validation at every stage
Quality metrics embedded in output attributes (nan_ratio, scene_count)
Checkpoint system for expensive operations

Quality Assurance

Each composite band includes embedded diagnostics:

composite.red.attrs
{
    "composite_stat": "median",
    "source_scene_count": 15,
    "nan_ratio": 0.027,
    "valid_ratio": 0.973
}

Interpretation:

nan_ratio < 0.05 : Excellent
nan_ratio < 0.10 : Good
nan_ratio < 0.20 : Acceptable
nan_ratio >= 0.20 : Poor - reconsider cloud filtering or date range

Typical Usage Pattern

# 1. Load your area of interest
aoi_geom = load_aoi_geom("field_boundary.gpkg", aoi_crs=4326)

# 2. Fetch STAC items (external to this module)
items = stac_search(...)  # Filtered by date, cloud cover

# 3. Generate composite
composite = seasonal_median_for_year(
    items=items,
    bands=["blue", "red", "nir"],
    aoi_geom=aoi_geom,
    chunks={"x": 1024, "y": 1024},
    target_crs="EPSG:32610",
    target_res=10.0,
    year_label=2020
)

# 4. Save checkpoint (optional but recommended)
save_intermediates(composite, "data/exports/intermediate/summer_2020.nc")

# 5. Export production COGs
save_composite_cogs(
    ds=composite,
    out_dir="data/exports/composites",
    prefix="summer_2020"
)

Dependencies

rioxarray / rasterio: COG access, reprojection, CRS handling
xarray: Lazy array operations, dataset merging
dask: Chunked parallel computation
geopandas: Vector I/O and CRS transformation
pandas: Datetime handling
h5netcdf/h5py: Intermediate checkpoint backend for NetCDF writes

Databricks Free Edition

Run via Git integration; %pip install libs in-notebook. FE is serverless, small, and no GPU—prefer lazy reads, small AOIs. Databricks FE notes.

Environment: Databricks Free Edition (FE)

Last verified: Feb. 06, 2026
Repo sync: GitHub → Databricks Repos (manual pull / reset as needed)

Runtime / Compute

DBR (runtime): Serverless environment version 4
Python: 3.12.3
Compute type: Serverless
Node/driver: N/A

Dependency install (FE)

Method: %pip in-notebook
Packages: xarray, rioxarray, rasterio, geopandas

Smoke test notebook

Path: notebooks/cnc-databricks_sanity.ipynb
What it does:
- Installs deps
- Imports libs + prints versions
- Loads data/aoi/aoi_chincha_norte.geojson with GeoPandas
- Prints CRS + total bounds
- Minimal rasterio/rioxarray read/write sanity check

FE limitations / gotchas (observed)

Resource limits: Not yet encountered
GPU: No
Other constraints: N/A

Run confirmation

Check latest run in cnc-databricks-sanity notebook.

Validation (lightweight)

Guano proxy: validation in notebooks/chincha_norte_mvp.ipynb now emphasizes patch coherence and visual plausibility overlays for the 2025 candidate mask rather than relying mainly on simple coast-distance assumptions.
Updated ecological reading: reference materials saved under docs/ suggest Chincha-style seabird use can occur on inland rocky terraces / flat colony pads, not only immediate cliff edges; this makes a strict “nearer to coast = more plausible” rule too simplistic.
Current quantitative signal: the 2025 candidate mask still contains many tiny fragments, but 94.1% of candidate pixels fall inside patches of at least 10 pixels, which supports keeping it as a structured screening layer rather than random speckle.
Visual caveat: dense colonies may appear as dark (guanay) or white (seagull-like) bird aggregations, while long-term guano may appear as lighter staining; bright rock/dust can still mimic both classes in this small-sample Sentinel baseline.
Terrain: visual check of proposed sensor points (access/exposure heuristic).
Vessels: monthly series + quick seasonality check; note coverage caveats from source data.

Limitations

No ground-truth colony labels in MVP; guano proxy is exploratory.
Seasonality/atmosphere: medians reduce noise, won't remove all confounders.
Vessel datasets are partial observations; interpret with local expertise.

Roadmap (next steps)

Add coastline distance from a high-res shoreline; integrate DEM-derived hazards.
Add a composite QA/EDA stage before downstream indices: scene-count adequacy checks, cloud/SCL distribution, per-band NaN maps, temporal coverage gaps, and a confidence gate threshold.
Explore a tiny supervised model if label points become available.
Extend to nearby islands in the Guano Islands system.
Optional: incorporate future acoustic summaries to cross-validate recovery.

Pilot (2–4 weeks)

Automation Parameterize and operationalize the current baseline workflow so partners can rerun the same steps on a fixed cadence (same AOI, same thresholds, same artifact outputs) with minimal analyst intervention.
Validation Run partner-informed validation on the strongest candidate areas: patch review, field spot-checks, and threshold refinement for the guano proxy and sensor-siting outputs, with explicit bias tracking.
Simple dashboard / reporting Deliver a lightweight stakeholder-facing reporting surface (curated figures + concise status metrics, optionally a minimal static dashboard) so non-technical teams can review the baseline without reading code.

Acknowledgements & references

IOCC & Chincha Norte (IC news). (Island Conservation)
Peru guano-bird decline (Reuters). (Reuters)
Guano spectral signature (Remote Sensing of Environment). (ScienceDirect)
Guano composition/appearance and aging context (IntechOpen). (IntechOpen)
GFW API docs and examples for public global presence extraction. (GFW API docs)
Seabird colony disturbance guidance (boats/daytime context). (Canada seabird disturbance guidance)
Copernicus DEM GLO-30 (AWS) / OpenTopography. (AWS Open Data Registry)

Rights / reuse

Unless otherwise agreed in writing, the code and project materials in this repository are shared for portfolio review and collaboration evaluation only.

This repository is intentionally presented as a demonstration of approach, method, and delivery quality. Reuse, redistribution, or commercial use is not granted by default.

Data sources and derived outputs may carry their own attribution and usage requirements:

Sentinel-2 / STAC provider terms
Copernicus DEM attribution requirements
Global Fishing Watch attribution requirements

Contact

Author: Iván G. Pérez — Data Scientist (remote, contractor W-8BEN), Lima (UTC-5)
Interested partners: Island Conservation and conservation analytics providers.

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
data		data
docs		docs
figures		figures
notebooks		notebooks
scripts		scripts
src		src
.gitignore		.gitignore
.python-version		.python-version
Makefile		Makefile
README.md		README.md
main.py		main.py
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Folders and files

Latest commit

History

Repository files navigation