Formula 1 Analytics Platform
Ingest F1 telemetry, transform it into a warehouse model, and serve BI dashboards — end to end.
Overview
Formula 1 releases high-frequency telemetry that is messy, wide, and updated mid-race. Turning it into reliable dashboards requires ingestion, modeling, orchestration, and a serving layer. This project builds all of that in the open.
Architecture
// interactive diagram coming next — click nodes to inspect
End-to-end data flow
- 01Airflow triggers dlt ingestion of OpenF1 endpoints on a per-session cadence.
- 02Raw JSON lands as partitioned Parquet in the lake zone.
- 03DuckDB reads Parquet directly; dbt materializes staging → intermediate → marts.
- 04Marts expose to Power BI via ODBC; exposures track downstream dashboards.
- 05Data tests + freshness checks run as post-ingest Airflow tasks.
Hard problems solved
ODBC Driver Error 126
Power BI on Windows failed to load the DuckDB ODBC driver. Root cause was a missing VC++ runtime dependency; documented install steps + a diagnostic script.
DuckDB write-lock contention
Concurrent Airflow tasks touching the same .duckdb file caused writer conflicts. Switched heavy writes to per-run Parquet + read-only DuckDB attach for serving.
Schema drift from OpenF1
Upstream added/removed fields mid-season. Adopted dlt's schema evolution + dbt contracts on marts to fail loud, not silently.
Lessons learned
- →Boring storage (Parquet) + a fast engine (DuckDB) beats a fancy warehouse for community-scale analytics.
- →Contracts belong on marts, not staging — let raw evolve, protect the serving layer.
- →Airflow is worth it once you have >3 dependent tasks with retries; scripts are fine before that.