Better decisions don't come from bigger models. They come from richer context.
So if it isn't the data, and it isn't the model — what is it?
For a decade, the playbook was simple: collect more, clean it better, put it somewhere everyone can reach it. Whole teams, budgets and roadmaps were built around that one idea — that better data, in greater volume, would eventually translate into better decisions.
And honestly, that part worked. Data today is everywhere. Streaming in, piling up, queryable from a notebook in under a minute. Most teams I talk to do not have a data shortage anymore. They have a data surplus.
But the AI sitting on top of all of it still wobbles. The same question asked twice gives two answers. A confident output turns out to be quietly wrong. Nothing breaks loudly — it just slowly stops being trustworthy. That is not a data volume problem. That is the model not understanding what it is looking at.
The bottleneck shifted. It used to be access. Now it is meaning.
And the stakes are quietly going up. AI is no longer just answering questions in a chat window — it is calling tools, moving money, updating records, taking actions on our behalf. An ungrounded answer is awkward. An ungrounded action is expensive. The gap between “the data is there” and “the system actually understands it” is the gap this whole site is about.
So let's see what that gap actually feels like
Toggle context signals and watch the answer transform in real time.
Based on general knowledge, tomatoes are a popular item. Consider reordering.
No store data. No seasonality. No supplier info.
Store 12 sells 45 units/day. Current stock: 20 units. Supplier lead time: 2 days. Holiday weekend in 3 days (expected 2.3× demand). Last stockout caused $2,400 loss. Sunny weekend = BBQ surge. URGENT: Reorder 180 units immediately. Priority: CRITICAL.
6 context signals · grounded in your lakehouse
The model did not change. The context changed.
Five simple ingredients. One trustworthy decision.
“Reorder tomatoes” needs data (stock levels), meaning (what “low stock” means for this store), history (last 30 days of sales), rules (supplier minimum order = 50 units), and situation (holiday weekend in 3 days). Strip any one out and the AI's answer stops being trustworthy.
Rules, definitions, policies, glossaries, brand voice. The slow-moving layer that tells the model what words mean in your world.
History, retrieval results, sensor readings, the conversation so far, the situation right now. The live layer the model must read fresh.
Static context teaches the model your world. Dynamic context tells it what is happening in it.
Prompt engineering tells the model what to do. Context engineering tells it what is true.
Pull only what matters. Semantic search, RAG, governed lookups — never the whole haystack.
“Context is precious. Spend it wisely.”
Summarize, rank, dedupe. Fit the window with signal, not noise.
“Every token competes for attention.”
Tie answers to governed facts, definitions, and sources the business already trusts.
“Trust comes from provenance, not eloquence.”
Carry history, state and prior decisions forward across turns, sessions and agents.
“An agent without memory restarts every minute.”
The new craft of AI is not bigger models. It is better context.
Five layers turn raw rows into decisions you can ship.
The idea is universal. The architecture is a choice. A governed Lakehouse is one practical way to make context real, end to end.
Not a sales pitch — just the architecture that quietly makes the rest of this site possible.
Every event, record, signal — ingested as-is.
Raw POS streams, IoT events, API payloads.
Remove noise, fix types, deduplicate — make it trustworthy.
Conformed orders, validated customers, unified SKUs.
Days of supply, risk scores, velocity — context is born here.
stockout_risk, customer_ltv, fraud_score.
One definition used everywhere — dashboards, Genie, agents, apps.
UC Metric Views, certified KPIs.
Now the model knows what it's looking at.
Genie, agents, retrieval over governed metrics.
Reorder. Discount. Promote. Donate. Not guessing — deciding.
Lakebase writes, workflow triggers, approvals.
Click a card to flip — see the same AI, with context added.
Six questions. One honest score.
Move a slider to begin
Do your AI systems access governed, trusted data?
0.0Are your business metric definitions centralized?
0.0Can your AI explain WHY it made a recommendation?
0.0Do your dashboards and AI tools use the same definitions?
0.0Can a non-technical user ask a question and get a trustworthy answer?
0.0Do your AI actions include a confirmation step before execution?
0.0Challenge a teammate. See how context-ready their AI really is.
Three live apps. Three different kinds of context. One idea.
Explore Context in Action ↑Retail. Entertainment. Risk. Different domains, same lesson — AI becomes useful when it understands the context.
Different apps. Same lesson. AI becomes useful when it understands the context.
Eight checks that separate a data pipeline from a context layer. Tick what is real in your org today — not what is on the roadmap.
Where this idea was first written down — plus follow-ups on freshness, consumers, hidden context, failure modes, and the business case. Read them right here.
Why the next leap in AI will not come from bigger models, but from giving them the data, meaning, history, rules and situation they need to actually understand the work.
A field guide for data engineers: how the lakehouse becomes the context layer, and why the new craft of AI is engineered context — not cleverer prompts.
Moving data is the old job. The new job is making data understandable for AI — with meaning, rules, quality, governance, and a trusted gold layer underneath it all.
Freshness is part of meaning. A stale feature, a cached customer profile, a yesterday's price — all of it quietly turns a confident AI into a wrong one.
Context is not one shape. The same business truth has to be served as a chart, a tool call, a one-line answer, and a defensible audit trail — without forking the truth itself.
Some of the most important context in any company has never been written down. Until you capture it, the model is missing the parts that actually decide the answer.
Bad context does not fail loudly. It fails as a confident, well-formatted, completely wrong answer — which is far more expensive than no answer at all.
Industry surveys keep saying the same thing in different words: the AI works in the demo, then it meets your data. The gap between those two moments is the cost of missing context.
This site is the interactive companion. The essays are the source.
The talks, papers, docs, and product references that shaped the context-first view of AI. Read them yourself — do not take this site's word for it.
Why context — not prompt — is the unit of work for serious agent systems.
Patterns that all assume strong, well-shaped context as a foundation.
How production ML/LLM systems actually live or die on their data layer.
Natural language over a governed semantic layer — context made consumable.
One metric defined once, used by every consumer — the canonical anti-fork pattern.
Bronze → Silver → Gold, the standard way to layer meaning onto raw data.
Adoption is up. Production value is gated on data and governance — not model choice.
Buyers consistently report data integration as the #1 blocker, not model quality.
Long-running research on why most AI initiatives never reach business value.
External links open in a new tab. This site has no affiliation with the sources beyond citing them.
Databricks Community Champion · Data Engineer · Builder
Creating practical Data + AI apps that turn raw data into context, decisions, and real business value. Focused on Databricks, lakehouse architecture, AI-powered analytics, and making complex data concepts simple, useful, and inspiring for every professional.
“I believe the hardest part of AI is not the AI. It is getting the right data to the right place in the right shape at the right time with the right meaning attached to it. That is context. And that is what data engineers build.”