every detector, every 15 minutes
Melbourne
Traffic Pulse
Victoria publishes a vehicle count for every loop detector at every traffic signal, every 15 minutes. This pipeline takes those files into a Databricks lakehouse each day, checks and reconciles them, and publishes what the city's traffic is doing. It runs like a production service: scheduled, monitored, with a freshness target, reconciliation checks and an incident record.
The pulse map
Every signal site in metropolitan Melbourne, placed by its coordinates and sized by its average weekday traffic over the last 28 days. No map underneath: the roads appear from the data. The glow follows the city's day, hour by hour.
The city's day
Average vehicles per hour across all signal sites, last 28 days.
Weekdays Weekends
Busiest intersections
Average weekday vehicles, last 28 days.
Every day, against its baseline
Vehicles counted per day. The line is the median of the same weekday over the previous four weeks; days more than 15% away from it are marked.
How it runs
One Databricks job, seven steps, every morning at 06:30 Melbourne time. Code lives in Git and deploys to Databricks automatically when it changes.
Reconciliation and quality checks
Detector health, newest day
Network-wide gaps
Intervals where every detector reported a fault at once: a system event (a clock change or a central outage), not a broken detector. Kept out of detector health and listed here.
Performance tuning
Incident record
Incidents open automatically when a run fails, the publisher's data is late, or a quality check warns, and close when the condition clears. Each one gets a root-cause note.