All posts

The architecture behind MeteoTrail: aggregating stations, webcams and lightning in real time

Published Jun 15, 20224 min read
  • MeteoTrail
  • Engineering
The architecture behind MeteoTrail: aggregating stations, webcams and lightning in real time

Key takeaways

  • A checkpoint isn't raw data from one source, it's a reconciled view built from whichever agency, webcam, station and lightning feed actually cover that exact point.
  • Deciding when to notify organizers needs a threshold-based rule engine, not raw data forwarding, or every noisy reading turns into an email.
  • A meteorologist's manual report has to visibly and clearly outrank the automated feed for a checkpoint, with a clear scope and expiry, or the override loses its meaning.

In the previous post about MeteoTrail I focused on the why: race directors don't need a regional forecast, they need to know what's going to happen at kilometer 43, at 2,400 meters, in four hours, because that's where their runners will be. This post is about the how — the actual engineering problem behind turning what's happening at this exact point in the mountains into something a race director can trust enough to act on. Some of what follows describes decisions we made deliberately; other parts are closer to "this is the kind of architecture a system like this needs," reasoned out from the constraints rather than a line-by-line account of every internal detail.

1. Many sources, one checkpoint

A checkpoint in a mountain ultra is not a single, stable location the way a city is. It's a point along a route that a race defines months in advance, that may sit exactly on a national border — races through Val d'Aran cross into France, and sometimes Andorra, within the same stage — and that needs to be resolved, in real time, against whichever official weather source actually covers that patch of terrain.

For each checkpoint we need to work out, ahead of the race, which national agency's alert zone it falls in, which public webcams along the route are close enough and pointed usefully at it, whether there's a private or semi-official weather station nearby with better resolution than the closest official one, and where lightning-detection data should be sampled from for that exact radius. That's four or five different upstream systems, each with its own API shape, its own refresh interval, and its own failure mode.

  • Official weather-agency APIs (current conditions, 48-hour and 7-day forecasts, alerts) — e.g. AEMET, Météo-France, IPMA, Aeronautica Militare
  • Public webcam feeds along the route
  • Real-time lightning-strike detection
  • Private or semi-official weather stations near the checkpoint

A checkpoint, in other words, isn't the raw data itself — it's the reconciled view we build around a geolocated point, sourced from whichever combination of feeds is actually available and trustworthy at that moment. The 48-hour forecast a race director sees for a given checkpoint is already the output of picking, per data type, the source we currently trust most for that specific location — not a single feed forwarded untouched.

2. Deciding what's significant enough to notify

Aggregating data is only half the problem. The harder half is deciding when a change is worth waking someone up over. A race director doesn't want an email every time the wind reading moves by half a kilometer per hour, but they absolutely want one within minutes of an official storm alert being issued for a checkpoint their runners will reach in three hours.

The rough shape of the solution is a threshold-based rule engine that runs on every meaningful update per checkpoint, not just on a timer: crossing a wind or temperature threshold, a new official alert appearing where there wasn't one before, lightning strikes detected inside a radius around the checkpoint, or a sharp temperature drop over a short window. Something like this, simplified a lot from what a real rule set would need:

def evaluate_checkpoint(checkpoint, previous_state, current_reading):
    triggers = []

    if crosses_threshold(previous_state.wind_kmh, current_reading.wind_kmh, threshold=60):
        triggers.append(Alert("wind", severity="high"))

    if current_reading.has_new_agency_alert and not previous_state.has_agency_alert:
        triggers.append(Alert("agency_alert", severity=current_reading.alert_level))

    if current_reading.lightning_strikes_last_10min(radius_km=15) > 0:
        triggers.append(Alert("lightning", severity="critical"))

    if current_reading.temperature_drop_c(last_hours=3) > 8:
        triggers.append(Alert("temp_drop", severity="medium"))

    if not triggers:
        return None

    return build_notification(checkpoint, triggers, previous_state)

The genuinely hard part isn't writing rules like these, it's tuning severity and radius so the system escalates the storm bearing down on kilometer 40 without also emailing every organizer every time a webcam glitches or a station reports a noisy reading. That tuning has to account for the fact that a single race can have checkpoints in three countries' jurisdictions at once, each agency with a different alert vocabulary and severity scale.

3. Letting a human override the automation

None of this replaces judgment. MeteoTrail's backoffice lets a hired meteorologist look at everything the automated pipeline is seeing for a race and publish a manual report that goes out to the whole organization, overriding what the automated feed is showing for that checkpoint.

The interesting design problem is what happens the moment after that report is published: the automated pipeline doesn't stop. So the system needs a clear, visible precedence rule — a human report has to outrank the automated view for that checkpoint for as long as it's meant to remain the authoritative word, without silently disappearing the moment the next automated refresh comes in, and without the two ever rendering side by side as if they carried equal weight. In practice that means a report needs an explicit scope, a clear provenance, and an expiry or superseding rule, so a stale manual report from six hours ago doesn't keep blocking a fresh, materially different automated read forever.

4. What's still hard

Some things about this problem don't have a clean answer yet. Sources disagree with each other more often than you'd like, and picking a winner per data type per checkpoint is still partly heuristic. Border checkpoints are their own category of annoying — a stage that starts under one agency's alert zone and finishes under another's needs both. Webcam reliability is entirely outside our control. And a single meteorologist covering multiple simultaneous races during a busy summer weekend has to prioritize, which means the manual-override layer has to be genuinely optional and the automation has to be good enough to stand on its own most of the time.

None of this is exotic distributed-systems work, and it doesn't need to be. But it is a real integration problem wearing the disguise of a weather app. If the March post was about why race directors needed this, this one is about what it takes, underneath, to make "what's the weather at checkpoint 7 right now" a question with a trustworthy answer.