Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ jobs:
echo "primed $job: $(cat "$out")"
}
pids=()
for job in alerts temp windtex refc; do prime "$job" & pids+=($!); done
for job in alerts temp windtex refc cape; do prime "$job" & pids+=($!); done
fail=0
for pid in "${pids[@]}"; do wait "$pid" || fail=1; done
[ "$fail" -eq 0 ] || { echo "one or more prime jobs failed"; exit 1; }
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Live weather on a deck.gl map, served almost entirely from free tiers: the whole bill is thirteen dollars a year of vanity domain plus a few cents a month for Route 53 and SES email.

OpenStreetMap basemap tiles come from [martin](https://github.com/maplibre/martin) running **inside AWS Lambda**, reading [PMTiles](https://docs.protomaps.com/pmtiles/) extracts straight from a private S3 bucket. A scheduled Rust lambda ([cargo-lambda](https://www.cargo-lambda.info/)) snapshots US-wide NWS alerts and decodes NOAA GFS GRIB2 straight from NOAA's open-data S3 bucket into global 2 m temperatures (a planet-wide lattice when zoomed out, per-city point forecasts when zoomed in), animated global wind, and a global precipitation forecast (GFS composite reflectivity) — city list from GeoNames. Precipitation shows the live RainViewer global radar composite (IEM NEXRAD as fallback) while the timeline is parked at *now*, then swaps to the GFS forecast raster as you scrub forward, so the whole map moves on one timeline. The web app is React + deck.gl + MapLibre, served from the same CloudFront distribution as the tiles and weather: one origin, no CORS, hashed assets cached immutable at the edge. Map views mirror into the URL hash, so any view is a link.
OpenStreetMap basemap tiles come from [martin](https://github.com/maplibre/martin) running **inside AWS Lambda**, reading [PMTiles](https://docs.protomaps.com/pmtiles/) extracts straight from a private S3 bucket. A scheduled Rust lambda ([cargo-lambda](https://www.cargo-lambda.info/)) snapshots US-wide NWS alerts and decodes NOAA GFS GRIB2 straight from NOAA's open-data S3 bucket into global 2 m temperatures (a planet-wide lattice when zoomed out, per-city point forecasts when zoomed in), animated global wind, a global precipitation forecast (GFS composite reflectivity), and a global storm-potential overlay (GFS surface CAPE) — city list from GeoNames. Precipitation shows the live RainViewer global radar composite (IEM NEXRAD as fallback) while the timeline is parked at *now*, then swaps to the GFS forecast raster as you scrub forward, so the whole map moves on one timeline. Storm potential (off by default) shades where the atmosphere is primed for thunderstorms, alongside the live NWS alerts. The web app is React + deck.gl + MapLibre, served from the same CloudFront distribution as the tiles and weather: one origin, no CORS, hashed assets cached immutable at the edge. Map views mirror into the URL hash, so any view is a link.

> **Not an official weather source.** This is a hobby map on shoestring infrastructure: alerts refresh on a schedule, live radar lags by several minutes, the forecast precipitation is coarse 0.25° model output (it smooths over individual cells), zone-based NWS alerts (no polygon geometry) are not shown, and any piece can fail silently with no on-map indication. For decisions involving life or property, use [weather.gov](https://www.weather.gov/) and local emergency guidance.

Expand Down Expand Up @@ -35,7 +35,7 @@ flowchart LR
sched -->|"invokes"| ingest
nws --> ingest
gfs --> ingest
ingest -->|"PUT JSON + wind/precip PNGs"| bucket
ingest -->|"PUT JSON + wind/precip/cape PNGs"| bucket
```

## What it costs
Expand Down Expand Up @@ -135,19 +135,19 @@ One piece lives outside CloudFormation: the stormdeck.live certificate was reque
| Lattice ↔ city temp switch | `GRID_ZOOM_SPLIT` in `web/src/config.ts` | z6.5 |
| Map start view | `web/src/config.ts` (URL hash wins) | world, z0 |
| World context detail | `WORLD_MAXZOOM` env for `just tiles extract` | z0-6 |
| Schedules | `cdk/lib/stormdeck-stack.ts` | alerts 5 min, temp 6 h, windtex 6 h |
| Schedules | `cdk/lib/stormdeck-stack.ts` | alerts 5 min; temp / windtex / refc / cape 6 h |

The `bbox` sets the full-detail basemap region; outside it the map shows the coarse world tiles. Temperature, wind, and the precipitation forecast are global (GFS), so they ignore it.
The `bbox` sets the full-detail basemap region; outside it the map shows the coarse world tiles. Temperature, wind, the precipitation forecast, and storm potential are global (GFS), so they ignore it.

## Notes

- **martin-in-Lambda**: martin ≥ v0.14 detects `AWS_LAMBDA_RUNTIME_API` and serves Lambda events natively. The zip is just the upstream `aarch64-musl` binary plus a two-line `bootstrap`. The function URL is IAM-auth; only CloudFront (OAC SigV4) may invoke it.
- **No aws-sdk in the ingester**: it only PUTs a handful of small objects, JSON snapshots plus the GFS wind PNGs, so it signs the request itself (SigV4, ~80 lines, test vector included). As of June 2026 the SDK also doesn't compile (aws-runtime 1.7.4 vs aws-smithy-runtime-api 1.12.3 skew). Check back later if you need more S3 surface.
- **Zone-based NWS alerts** (no polygon geometry) are dropped; rendering them would mean shipping zone shapefiles. Counted in the lambda logs.
- **GFS straight from GRIB2**: temperature, wind, and the precipitation forecast all come from NOAA GFS with no per-point API metering: the ingester pulls 0.25° UGRD/VGRD/TMP/REFC fields from NOAA's public `noaa-gfs-bdp-pds` S3 bucket and decodes the GRIB2 itself, so one ~0.9 MB field covers the whole planet (1440×721) and any number of points sample for free. One pass writes a whole-planet `lattice.json` (the zoomed-out grid) plus per-city tiles (zoomed in), sampled from the same TMP fields, so the grid costs no extra fetches; the wind u/v PNGs (±40 m/s); and the precipitation PNGs (composite reflectivity, a grayscale dBZ texture over −20…75 dBZ — GFS floors clear sky at ~−20 dBZ, so the web renders that transparent). All carry the model run's snapshot so a new run refetches cleanly, and all share one forecast-hour axis so the timeline scrubs grid, cities, wind, and precipitation together.
- **GFS straight from GRIB2**: temperature, wind, the precipitation forecast, and storm potential all come from NOAA GFS with no per-point API metering: the ingester pulls 0.25° UGRD/VGRD/TMP/REFC/CAPE fields from NOAA's public `noaa-gfs-bdp-pds` S3 bucket and decodes the GRIB2 itself, so one ~0.9 MB field covers the whole planet (1440×721) and any number of points sample for free. One pass writes a whole-planet `lattice.json` (the zoomed-out grid) plus per-city tiles (zoomed in), sampled from the same TMP fields, so the grid costs no extra fetches; the wind u/v PNGs (±40 m/s); the precipitation PNGs (composite reflectivity, a grayscale dBZ texture over −20…75 dBZ — GFS floors clear sky at ~−20 dBZ, so the web renders that transparent); and the storm-potential PNGs (surface CAPE, a grayscale texture over 0…5000 J/kg, faded out below ~250 J/kg). All carry the model run's snapshot so a new run refetches cleanly, and all share one forecast-hour axis so the timeline scrubs grid, cities, wind, precipitation, and storm potential together.

## Attribution

Map data © [OpenStreetMap](https://openstreetmap.org/copyright) contributors, tiles via [Protomaps](https://protomaps.com) builds (ODbL). Radar: [RainViewer](https://www.rainviewer.com/) global composite (free tier, attribution required), falling back to NOAA NEXRAD via the [Iowa Environmental Mesonet](https://mesonet.agron.iastate.edu/). Alerts: [National Weather Service](https://www.weather.gov/) (public domain). Temperatures, wind, and the precipitation forecast (composite reflectivity): [NOAA GFS](https://registry.opendata.aws/noaa-gfs-bdp-pds/) via NOAA Open Data Dissemination (public domain). City list: [GeoNames](https://www.geonames.org/) (CC-BY 4.0).
Map data © [OpenStreetMap](https://openstreetmap.org/copyright) contributors, tiles via [Protomaps](https://protomaps.com) builds (ODbL). Radar: [RainViewer](https://www.rainviewer.com/) global composite (free tier, attribution required), falling back to NOAA NEXRAD via the [Iowa Environmental Mesonet](https://mesonet.agron.iastate.edu/). Alerts: [National Weather Service](https://www.weather.gov/) (public domain). Temperatures, wind, the precipitation forecast (composite reflectivity), and storm potential (surface CAPE): [NOAA GFS](https://registry.opendata.aws/noaa-gfs-bdp-pds/) via NOAA Open Data Dissemination (public domain). City list: [GeoNames](https://www.geonames.org/) (CC-BY 4.0).

MIT.
20 changes: 11 additions & 9 deletions cdk/lib/stormdeck-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,15 @@ export class StormdeckStack extends Stack {
enforceSSL: true,
removalPolicy: RemovalPolicy.DESTROY,
autoDeleteObjects: true,
// citytile + windtex + refctex each write a fresh {snapshot}/ tree per
// run; expire old snapshots so they don't accumulate. Their latest.json
// pointers are rewritten each run, so they stay under the age cutoff and
// never age out.
// citytile + windtex + refctex + capetex each write a fresh {snapshot}/
// tree per run; expire old snapshots so they don't accumulate. Their
// latest.json pointers are rewritten each run, so they stay under the age
// cutoff and never age out.
lifecycleRules: [
{ prefix: 'weather/citytile/', expiration: Duration.days(2) },
{ prefix: 'weather/windtex/', expiration: Duration.days(2) },
{ prefix: 'weather/refctex/', expiration: Duration.days(2) },
{ prefix: 'weather/capetex/', expiration: Duration.days(2) },
],
});

Expand Down Expand Up @@ -102,12 +103,12 @@ export class StormdeckStack extends Stack {
manifestPath: CRATES_MANIFEST,
binaryName: 'weather-ingest',
architecture: lambda.Architecture.ARM_64,
// temp + windtex + refc decode GFS GRIB fields (~4 MB grids, several in
// flight; windtex also holds a u/v pair while its PNG encodes); alerts is
// lighter.
// temp + windtex + refc + cape decode GFS GRIB fields (~4 MB grids,
// several in flight; windtex also holds a u/v pair while its PNG encodes);
// alerts is lighter.
memorySize: 512,
// temp samples ~57 GFS TMP fields (cities + lattice); windtex decodes u/v
// pairs and refc single REFC fields into PNGs — all well within 600s.
// pairs, refc/cape single REFC/CAPE fields into PNGs — all within 600s.
timeout: Duration.seconds(600),
environment: {
BUCKET: bucket.bucketName,
Expand Down Expand Up @@ -298,12 +299,13 @@ export class StormdeckStack extends Stack {
// EventBridge Scheduler triggers (14M invocations/month free tier). Every
// weather job is free GFS GRIB from NODD (no per-call limit) plus the public
// NWS alerts feed, so the cadence is bounded by freshness, not API quotas;
// temp + windtex + refc refresh once per GFS cycle window.
// temp + windtex + refc + cape refresh once per GFS cycle window.
const jobs: Array<[string, Duration]> = [
['alerts', Duration.minutes(5)],
['temp', Duration.hours(6)],
['windtex', Duration.hours(6)],
['refc', Duration.hours(6)],
['cape', Duration.hours(6)],
];
for (const [job, every] of jobs) {
new scheduler.Schedule(this, `Schedule-${job}`, {
Expand Down
28 changes: 27 additions & 1 deletion crates/weather-ingest/src/contract.rs
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,31 @@ pub struct RefcTexIndex {
pub dbz_max: f32,
}

/// The `capetex/latest.json` pointer: the current snapshot, its forecast-hour
/// axis, the equirectangular texture dimensions, and the CAPE (J/kg) bounds the
/// grayscale PNG was normalized over (so the web denormalizes identically). The
/// per-step textures live at `capetex/{snapshotMs}/{hour}.png` (immutable); the
/// web's storm-potential overlay loads whichever step the timeline is on. Same
/// axis as windtex/refctex/citytile, so the one timeline scrubs it with the rest.
#[derive(Serialize)]
#[cfg_attr(feature = "ts", derive(specta::Type))]
pub struct CapeTexIndex {
#[serde(rename = "snapshotMs")]
#[cfg_attr(feature = "ts", specta(type = specta_typescript::Number))]
pub snapshot_ms: u64,
pub hours: Vec<u32>,
pub width: u32,
pub height: u32,
// Always-present reals; the `Number` override avoids the `number | null`
// that a bare f32 exports (same trick as CityForecast::t).
#[serde(rename = "capeMin")]
#[cfg_attr(feature = "ts", specta(type = specta_typescript::Number))]
pub cape_min: f32,
#[serde(rename = "capeMax")]
#[cfg_attr(feature = "ts", specta(type = specta_typescript::Number))]
pub cape_max: f32,
}

#[cfg(all(test, feature = "ts"))]
mod export {
use specta::Types;
Expand All @@ -221,7 +246,8 @@ mod export {
.register::<super::CityForecast>()
.register::<super::CityTileIndex>()
.register::<super::WindTexIndex>()
.register::<super::RefcTexIndex>();
.register::<super::RefcTexIndex>()
.register::<super::CapeTexIndex>();
Typescript::default()
.header("// Generated from crates/weather-ingest/src/contract.rs by `just build types`. Do not edit.\n")
.export_to(
Expand Down
8 changes: 8 additions & 0 deletions crates/weather-ingest/src/gfs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ pub const WIND_MS_MAX: f32 = 40.0;
pub const REFC_DBZ_MIN: f32 = -20.0;
pub const REFC_DBZ_MAX: f32 = 75.0;

/// Surface CAPE (convective available potential energy, J/kg) normalization
/// range for the storm-potential texture, same scheme as [`REFC_DBZ_MIN`]. 0 is
/// a stable atmosphere; the web renders below its display threshold (~250 J/kg,
/// marginal instability) transparent. 5000 J/kg caps all but the most extreme
/// setups (observed global max ~4800), which clamp.
pub const CAPE_JKG_MIN: f32 = 0.0;
pub const CAPE_JKG_MAX: f32 = 5000.0;

/// A decoded 0.25° global field, row-major from (90N, 0E).
pub struct Field {
values: Vec<f32>,
Expand Down
Loading