Tech/Weather Data Pipeline

Weather Data Pipeline

Multi-model ensemble forecasts from Open-Meteo for 10 global stations

Open-Meteo API

Open-Meteo provides free, high-quality weather forecast data from multiple numerical weather prediction models. The API requires no authentication and returns JSON with hourly forecasts up to 16 days ahead. We query temperature_2m, wind_speed_10m, relative_humidity_2m, and precipitation for each station.

Ensemble Models

Four independent weather models provide diverse forecasts that capture model uncertainty. GFS (Global Forecast System) from NOAA runs at 0.25 degree resolution. ECMWF (European Centre) is generally the most accurate global model. ICON (DWD Germany) excels in European coverage. GEM (Canadian) provides independent North American perspective.

GFS: NOAA, 0.25° resolution, updated every 6 hours
ECMWF: European, highest accuracy, updated every 12 hours
ICON: DWD Germany, excellent for Europe, updated every 6 hours
GEM: Canadian, good for North America, updated every 12 hours

Feature Engineering

Raw forecasts are transformed into 38 features per station/date. Ensemble statistics (mean, std, min, max, range) capture model agreement. Temporal encodings use sin/cos transforms of day-of-year for smooth cyclical representation. Historical accuracy metrics from recent predictions provide adaptive bias correction.

Ensemble stats: mean, std, min, max across 4 models
Forecast horizon: hours until market resolution
Diurnal patterns: expected daily min/max from all models
Precipitation flag: binary indicator for wet conditions