Why text-to-SQL needs a semantic layer

Q: Does adding a semantic layer slow queries down?

No. The semantic layer shapes SQL *generation*; execution still runs on an engine suited to the data's size - DuckDB in-process for small data, Trino or Spark for large distributed scans.

Sushrut Ikhar, Co-founder, Matih Labs — 2026-07-30

Why text-to-SQL needs a semantic layer

TL;DR — A schema tells a model what your tables are named, not what they mean. So schema-only text-to-SQL produces queries that run successfully and return the wrong number: fanned-out joins that double-count revenue, "active users" computed three different ways in one quarter, aggregates that quietly include test rows. A semantic layer - entities, relationships, governed metric definitions - is what turns SQL generation into something you can verify instead of something you have to take on faith. Here are the three failure modes we designed Matih's grounding workflow around, and exactly where a semantic layer intervenes in each.

We built Matih because we kept watching the same thing happen: someone asks a data question in plain English, a model writes SQL that parses, executes, and returns a clean-looking number - and the number is wrong in a way nobody notices until it's in a board deck. The failure is almost never SQL syntax. Modern models write fluent SQL. The failure is meaning, and meaning isn't in the schema.

What goes wrong with joins? The fan-out that inflates revenue

Ask "what was July revenue for shipped orders?" against a typical commerce schema and a schema-only generator will happily produce:

SELECT SUM(o.amount)
FROM orders o
JOIN shipments s ON s.order_id = o.id
WHERE s.shipped_at >= '2026-07-01';

This query is valid. The foreign key is real. And if any order ships in more than one box, its amount is counted once per shipment. Revenue inflates, silently, by exactly the fan-out of your fulfillment process - which means the error changes month to month as your shipping behavior changes. Nothing about the schema warns the model: cardinality lives in your operational reality, not in information_schema.

The semantic fix is knowing that orders → shipments is one-to-many and that amount is an order-grain fact, so a shipment-side filter needs an EXISTS (or a deduplicated join), not a naive inner join. That knowledge has to be modeled somewhere. In Matih it lives in the knowledge graph - tables and columns from your catalog, fused with the relationships and business concepts your documents and governed definitions describe - and the generator reasons over that graph, not just over column names.

Why do metrics drift? One question, three definitions of "active"

"How many active users did we have last month?" is not one question. Is "active" logged in? Performed a billable action? Does it exclude internal and test accounts? Every company has a real answer - usually written down in a dbt model, a BI layer, or one analyst's head - and a schema-only generator has no access to any of them. It picks an interpretation, confidently. Ask twice on different days and you can get different interpretations, which is how the same "active users" ends up as three different numbers in three different decks.

This is the difference between generating SQL and answering from a governed definition. When Matih answers from a curated glossary term or metric, the answer cites that definition as a source

a [S1] marker you can click to see exactly which governed definition was used. And the answer carries a trust badge that tells you which path it came through: Verified (served verbatim from a query a human approved), Governed (computed from a governed metric definition), or Ad-hoc (freshly generated SQL - correctly grounded, but no human sign-off yet). The badge doesn't make the answer smarter. It makes the answer auditable, which is the property the board deck actually needed.

Which rows shouldn't count? The operational sediment problem

Production tables accumulate operational sediment: status = 'test' orders from QA, soft-deleted rows, refunds recorded as negative line items, internal accounts with employee discounts. Humans who query these tables daily filter this sediment out by reflex. A schema-only model doesn't know the sediment exists - status is just a varchar, and nothing in the DDL says three of its seven values shouldn't count toward revenue.

This one is insidious because the error is small - a few percent, usually - so it passes the sniff test and ships. The fix, again, is unglamorous modeling: the semantic layer records which entity states are real business activity, so every generated query inherits the same exclusions instead of each query re-deciding them from scratch.

Trust is a workflow, not a model property

Here's the opinionated part: we don't believe any generator - ours included - should be trusted on output alone. A wrong query and a right query look identical in a results grid. So the design bar we hold ourselves to is that every answer must be checkable in one click:

Claims in an answer carry [Q1]-style citations - hover to see the exact SQL behind that claim and a preview of the rows it returned.
A grounding trace ("What I knew when answering") lays out the tables, definitions, and prior queries the answer relied on.
If a quality safeguard couldn't run on a turn, a Reduced confidence chip says so, with the concrete reasons - rather than pretending nothing happened.
Good ad-hoc answers can be promoted: Propose as verified query turns today's checked answer into tomorrow's Verified one, so trust compounds instead of resetting every conversation.

If your text-to-SQL tool can't show you why it chose a join or which definition of a metric it used, it isn't asking for trust. It's asking for faith.

Doesn't this just move the work?

Partly - and honestly, that's fine. A semantic layer needs curation: someone approves verified queries, someone confirms what "active" means. The difference is that this work is done once, by the person who knows, and then every subsequent question inherits it - instead of every question re-deriving business logic from column names and hoping. That's also why the Ad-hoc badge exists rather than us pretending everything is governed on day one: the layer grows as you use it, starting from your catalog and the sample catalog that ships with your workspace.

And once the system understands the data it's touching, semantics stop being just a correctness layer: the same knowledge lets Matih route each query to a suitable engine

DuckDB in-process for small data, Trino or Spark at scale - so generation, verification, and execution all operate on your data, end-to-end.

FAQ

Isn't a data catalog the same thing as a semantic layer?

No. A catalog documents what exists - tables, columns, owners. A semantic layer defines what the data means: how entities relate, which joins are safe to aggregate across, and how business metrics are officially computed. Text-to-SQL needs the second to be trustworthy.

Can't a better LLM just infer the semantics from the schema?

It can guess more plausibly, but the failure modes above aren't inference problems - the information genuinely isn't in the schema. No model can read your definition of "active user" out of a varchar column. It has to be told, once, and then held to it.

Does adding a semantic layer slow queries down?

No. The semantic layer shapes SQL generation; execution still runs on an engine suited to the data's size - DuckDB in-process for small data, Trino or Spark for large distributed scans.

How do I know whether a given answer used governed definitions?

Every Matih answer carries a trust badge - Verified, Governed, or Ad-hoc - and citation markers that show the exact SQL and sources behind each claim, so the answer's provenance is visible instead of implied.