WeatherSentinel

National Weather Intelligence, Verification & Analytics Platform

WeatherSentinel brings weather observations, data-quality checks, rule-based hazard detection, short-horizon temperature forecasts, geospatial views, and evidence-grounded conversational access into one platform. It is designed to turn heterogeneous weather data into information that is easier to inspect, compare, and act on.

Project status and interpretation: WeatherSentinel is a project/prototype, not an official meteorological warning service. Forecasts are model-generated estimates and should not be treated as verified forecasts or safety-critical advice. Always consult the relevant official weather and disaster-management authorities for decisions that affect safety.

Overview

Weather data is useful only when its source, timestamp, quality, and context are clear. WeatherSentinel combines a data-ingestion pipeline with API services, persistent storage, analytics dashboards, a machine-learning temperature prediction component, and a local conversational agent.

The system is designed to help users:

Inspect current and recent weather observations for registered locations.
See source timestamps and distinguish fresh, delayed, and stale data.
Review rule-triggered weather events and verification-related information.
Explore locations and analytics through dashboard visualizations and maps.
Request weather summaries and evidence-based briefings through an agent that accesses backend capabilities over MCP.

Key capabilities

Weather ingestion and processing

Ingests weather observations from configured providers, including the Open-Meteo path.
Uses Kafka to move observation messages through the processing pipeline.
Validates and normalizes incoming records before persistence.
Stores observations and related records in PostgreSQL, with PostGIS available for geospatial data.

Weather events and verification

Applies rule-based checks to identify configured weather thresholds.
Exposes event and verification-related information through backend services and MCP tools.
Keeps rule-based event detection distinct from machine-learning forecasting; a threshold-triggered event is not proof that a real-world incident has occurred.

Machine-learning forecasts

Generates a 3-hour temperature prediction using the Random Forest forecasting component.
Displays model-generated predictions with weather intelligence views where data is available.
Does not claim a validated accuracy score unless that score has been measured and documented separately.

Interactive analytics

Dashboard visualizations use Apache ECharts and Plotly for supported analytics views.
Geospatial views use Leaflet.
Charts and summaries should be interpreted alongside source coverage, timestamps, and freshness indicators.

Local conversational weather assistant

Provides a custom Alexa+-style weather assistant simulator at /alexa in the frontend.
Uses a local Ollama model for language generation, the Strands Agents SDK for agent orchestration, and a self-hosted MCP server exposed over Streamable HTTP.
Offers a deterministic demo mode for predictable demonstrations without relying on a language model.

Clarification: The Alexa+-style simulator is a project-built interface; it is not an official Amazon Alexa+ production integration. The primary documented agent workflow uses local Ollama, Strands, and MCP. Any alternative provider path should be documented separately only if it is intentionally supported and tested.

Architecture and data flow

flowchart TD
    A[Configured weather providers] --> B[Live weather producer]
    B --> C[Kafka: weather.observations]
    C --> D[Kafka weather worker]
    D --> E[Validate and normalize]
    E --> F[(PostgreSQL + PostGIS)]
    E --> G[Rule-based event detection]
    E --> H[Random Forest temperature prediction]
    F --> I[FastAPI services and MCP server]
    G --> I
    H --> I
    I --> J[Frontend dashboard and maps]
    I --> K[MCP Streamable HTTP]
    K --> L[Strands agent + local Ollama model]
    L --> M[Alexa+-style simulator]

Main observation pipeline

A configured producer retrieves weather data and publishes observation messages to the weather.observations Kafka topic.
The Kafka worker consumes each message and passes it to the weather-processing service.
The processor validates and normalizes the record and stores the observation in PostgreSQL.
Rule-based event detection evaluates the observation against configured thresholds.
The forecasting service generates a short-horizon temperature estimate when its required inputs are available.
Backend API and MCP tools expose the available observations, events, forecasts, and related status to the dashboard and conversational agent.

This describes the intended main path. Actual outputs depend on the services running, provider availability, configured stations, and the data available in the database.

Technology stack

Area	Technology
Backend API	Python, FastAPI
Data validation and application models	Pydantic and Python service modules
Persistence	PostgreSQL 17
Geospatial data	PostGIS
Streaming	Apache Kafka, Docker Compose
Database migrations	Alembic, SQLAlchemy
Machine learning	scikit-learn Random Forest forecasting component
Frontend	Python, NiceGUI
Mapping	Leaflet
Analytics charts	Apache ECharts, Plotly
Agent orchestration	Strands Agents SDK
Local language model	Ollama (configured model: `llama3.2:1b`)
Agent/backend interface	MCP Streamable HTTP
Python environment and dependencies	`uv`

Repository layout

The repository is organized around backend services, frontend views, data and ML work, pipelines, documentation, and tests. The exact contents may change as development continues.

WeatherSentinel/
├── backend/       # FastAPI app, MCP server, ingestion and processing services
├── frontend/      # NiceGUI dashboard, maps, analytics and Alexa+-style simulator
├── data/          # Project data and data-related assets
├── docs/          # Project specifications, architecture and design notes
├── ml/            # Machine-learning components and supporting assets
├── pipelines/     # Data pipeline code and related workflows
├── tests/         # Project-level tests, where maintained
└── README.md

Local development setup

Prerequisites

Install or configure the following before starting the full pipeline:

Windows 11 or another supported development environment.
Python compatible with the project's pyproject.toml files.
uv for Python dependency and environment management.
PostgreSQL 17 and PostGIS.
Docker Desktop with Docker Compose, for the Kafka service.
Ollama if you want to use the local language-model agent.

The dashboard and API can be developed separately, but the complete streaming workflow also depends on the database and Kafka being available and correctly configured.

1. Get the code and configure environment files

From the repository root, create local environment files from the checked-in examples if they do not already exist:

cd E:\WeatherSentinel\WeathersSentinel

if (!(Test-Path .\backend\.env)) {
    Copy-Item .\backend\.env.example .\backend\.env
}

if (!(Test-Path .\frontend\.env)) {
    Copy-Item .\frontend\.env.example .\frontend\.env
}

Review both .env files and fill in the values required by your local PostgreSQL, Kafka, weather-provider, frontend, and MCP configuration. The example files are templates; do not assume they contain valid credentials or machine-specific values.

2. Install backend dependencies and apply migrations

cd E:\WeatherSentinel\WeathersSentinel\backend
uv sync
uv run alembic upgrade head

Make sure PostgreSQL is running, the configured database exists, and the backend .env values match your local database setup before applying migrations.

3. Install frontend dependencies

Open a separate terminal:

cd E:\WeatherSentinel\WeathersSentinel\frontend
uv sync

4. Prepare the local model (optional for deterministic demo mode)

Ensure Ollama is installed and running, then pull the configured model:

ollama pull llama3.2:1b
ollama list

If Ollama is not already running, start it using your normal Ollama installation workflow. Do not start a second server if one is already listening.

Run the services

Start the required components in separate terminals. Check the relevant .env.example and application settings if your local ports or service names differ.

1. Start Kafka

From the repository root:

cd E:\WeatherSentinel\WeathersSentinel
docker compose up -d kafka

Confirm that the Kafka service is healthy before starting the producer and worker.

2. Start the backend API

cd E:\WeatherSentinel\WeathersSentinel\backend
uv run uvicorn app.main:app --host 127.0.0.1 --port 8000

When the API is running, its OpenAPI documentation is typically available at http://127.0.0.1:8000/docs, provided the app's documentation routes have not been disabled.

3. Start the Kafka worker

In another terminal:

cd E:\WeatherSentinel\WeathersSentinel\backend
uv run python -m app.services.weather_kafka_worker

4. Start the live weather producer

In another terminal, if you want the live ingestion loop:

cd E:\WeatherSentinel\WeathersSentinel\backend
uv run python -m app.services.live_weather_producer

The producer depends on valid provider configuration and connectivity. The worker depends on Kafka, database configuration, and the expected topic being available.

5. Start the frontend

In another terminal:

cd E:\WeatherSentinel\WeathersSentinel\frontend
uv run python app.py

Use the URL printed by NiceGUI in the terminal. The frontend port is controlled by the frontend application configuration; verify that setting if a different port is required or the default port is already occupied.

Suggested startup order

PostgreSQL/PostGIS
Kafka
Backend API
Kafka worker
Live weather producer
Frontend
Ollama agent mode, if using language-model responses

Not every mode requires every component. The deterministic demo mode is intended to reduce dependence on live language-model inference.

Conversational agent and MCP

The frontend's Alexa+-style simulator uses the backend's MCP interface to request weather information. The agent should rely on results returned by tools rather than inventing live observations.

The MCP endpoint used by the local setup is:

http://127.0.0.1:8000/mcp

The MCP tool set described by the project includes capabilities such as:

search_weather_locations — find registered weather locations.
get_weather_summary — retrieve a weather summary.
get_weather_events — inspect detected event records.
get_verification_status — retrieve verification-related status.
get_temperature_forecast — retrieve the available temperature forecast.
get_evidence_based_briefing — assemble a briefing from available evidence.

The actual tool schema and result fields are defined by the MCP server implementation. A tool may return incomplete or stale data if its underlying source records are missing or old.

Local agent modes

Ollama agent mode: Uses the configured local Ollama model with Strands and MCP tool access. Language-model output depends on the local model and available tool responses.
Deterministic demo mode: Provides a predictable path for demonstrations without requiring a generative model response. It should not be presented as an LLM-generated answer.

Forecasting and event detection

Random Forest temperature prediction

WeatherSentinel's forecasting component uses a Random Forest model to produce a temperature estimate for a three-hour horizon. A model output is a prediction, not an observed measurement. Displayed estimates should be considered alongside the prediction timestamp, source observation, location, and model availability.

This README does not claim a specific forecast accuracy, calibrated uncertainty range, or operational performance guarantee. Report such metrics only after a reproducible evaluation against aligned observed data, with the evaluation period and methodology documented.

Rule-based weather event detection

The event detection service applies configured threshold rules to incoming observations. This is distinct from ML-based anomaly detection and does not independently verify that an impact, emergency, or citizen report occurred. Thresholds and units should be checked in the implementation and configuration used for the running deployment.

Data freshness and limitations

Freshness matters: Weather records must be interpreted with their source timestamp and ingestion time. A recent page load does not make an old observation current.
Delayed or stale records: A location may still appear when its last observation is delayed or stale. Check the timestamp and freshness indicator before comparing it with other locations.
Station identity: Historical or provider-specific station records may differ from live locations with similar names. Confirm that the displayed station, observation, and forecast refer to the same location before drawing conclusions.
Provider availability: Missing observations can result from upstream outages, connectivity, provider limits, or ingestion failures.
Model limitations: The Random Forest output is only as useful as its inputs and evaluation. It should not replace an official forecast or warning.
Prototype status: Event detection, verification, analytics, and agent responses are software features, not a substitute for official weather warnings or emergency guidance.

Configuration and security

Use backend/.env.example and frontend/.env.example as configuration references.
Never commit .env files, API keys, passwords, tokens, private certificates, or other secrets.
Keep local MCP endpoints bound to loopback for local development unless you have intentionally configured and secured a remote deployment.
Do not expose PostgreSQL, Kafka, or development servers publicly without appropriate authentication, network restrictions, and operational hardening.
Verify service URLs, model settings, topic names, database settings, and ports against the checked-out code before deployment.

Testing

Run the tests in the relevant project directory so uv uses that project's dependencies:

# Backend tests
cd E:\WeatherSentinel\WeathersSentinel\backend
uv run pytest

# Frontend tests — run in a separate command/terminal after backend tests
cd E:\WeatherSentinel\WeathersSentinel\frontend
uv run pytest

Tests may include mocked and deterministic paths. Passing unit tests does not, by itself, prove that the live weather providers, Kafka, database, MCP server, Ollama model, and full end-to-end pipeline are all reachable. Perform a separate integration check when validating a complete local setup.

Troubleshooting

API port is already in use

Check the process using port 8000 before launching another backend instance. Stop or reuse the existing process rather than starting duplicate servers.

Get-NetTCPConnection -LocalPort 8000 -State Listen -ErrorAction SilentlyContinue |
    Select-Object LocalAddress, LocalPort, OwningProcess

Kafka messages are not being processed

Confirm that Docker reports the Kafka service as running, the producer is publishing to weather.observations, the worker is subscribed to that topic, and both services use compatible Kafka connection settings.

The dashboard shows delayed or stale observations

Inspect the source timestamp, ingestion logs, producer status, Kafka worker output, and the station identity. Do not describe delayed records as real-time observations.

The local agent cannot answer using weather data

Confirm that the backend is available at the configured MCP URL, the MCP server is registered, and the required tool can retrieve data independently of the language model. Then check that Ollama is running and that the configured model is installed.

Database migration errors

Check the active database URL and migration history before retrying. Take care with PostGIS-managed objects; do not manually drop extension-owned tables to fix an unrelated migration issue.

Contributing

For a contribution, keep changes focused, add or update relevant tests, and update documentation when interfaces, configuration, data sources, or behavior change. Before submitting a change, run the tests for the affected project and inspect the final diff.

License

The applicable license should be specified by the repository's LICENSE file. If no LICENSE file is present, add one before describing the project as open source or granting reuse permissions.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
backend		backend
docs		docs
frontend		frontend
ml		ml
pipelines/spark		pipelines/spark
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml

Latest commit

History

Folders and files

Repository files navigation