Data Engineering Weekly Notes
Dec 29 – Jan 4 · 2026-W01
1 event this week
🧠 Feynman Takeaway
The data engineering community started the year by declaring AI pipelines, data contracts, and real-time streaming as the three pillars of 2026.
2026 DE Trend Reports Drop
In plain English
Every January, the data engineering community publishes predictions. For 2026, the consensus was unusually specific: (1) AI-native pipelines where LLMs write and optimize ETL code, (2) data contracts as the standard for cross-team data sharing, and (3) streaming-first architectures replacing batch ETL as the default pattern.
Analogy
Like three restaurant critics all naming the same dish as 'the thing to eat this year' — except the dish is 'let AI build your data pipelines while streaming replaces your nightly batch jobs.'
Why it matters
This turned out to be remarkably prescient. By July, AI agents wrote 80%+ of new databases on Databricks, and Lakestream (streaming + lakehouse) became the dominant architecture pattern.