Let’s say you are building an agent that is managing your finances. You ask it a question like “How have my spending habits changed since I moved?”.

Now your agent needs 3 things:

1. Date on which you moved
2. More semantic context on where you’ve moved, some other facts perhaps
3. Access to your transactions

task
  ↓
retrieve relevant memories
  ↓
put memories in context window
  ↓
LLM reasons
  ↓
answer/action

If your agent just has a RAG driven memory system, it might find out the date you moved and then try to find your transactions. Maybe they are in some table you can query with SQL or worse, they are in your email inbox as a PDF file.

If you go the PDF route you’d spend all these tokens in trying to parse that PDF file to a local SQLite file and then query it. If you ask it a follow up to correlate this to something else, it would go on another rampage to find context and cobble together the answer in a very inefficient way.

It works completely fine for personal use cases, but if you multiply this with the nature of analytical workloads inside an enterprise, you quickly realise how costly this would be, both in time and tokens.

Traditionally agent companies are approaching it like a memory problem. Agent memory is a very narrow use case of equipping an agent with the data. Memory assumes that recall solves the full problem. That placing the right string in the context will solve the problem. But I feel it’s time to improve that thinking and take it a step further.

Memory should not be a pile of text that is retrieved and shoved into the agent’s context window. It should be a continuously maintained semantic model of the world in which it’s operating.

                ┌── semantic knowledge ──┐
                │                        │
question ───────┼── historical events ───┼── computation ──→ answer/action
                │                        │
                └── structured data ─────┘

I’ve been experimenting with a system in a personal capacity that I have started calling the Personal Data Factory. The idea is to build this continuously maintained semantic model over all the data that an agent needs to operate on.

The Missing Domain #

In the older world, it was easier to imagine the data stack as three domains: movement, storage and meaning. Movement is where companies like Fivetran operated by moving data across various sources and tools, storage is the classic cloud data warehouses that offer storage and compute, and then finally we have the final domain of data meaning. In here you have companies like dbt Labs and all the BI tools which help teams derive meaning and insights from the stored data.

Data transformation to BI is all an effort of encoding meaning. You do it when you define your dbt models and run them, you do it when you set up a reusable widget in your BI tool, you do it when you set up schedules to refresh certain tables faster than others.

I think it’s about time that the data meaning domain became a lot more autonomous and agent-friendly. Because most of the data work is now done not only by humans but also by agents. Agents need access to so much data and context to do whatever they are trying to do.

Earlier, there were not a lot of tools in the modern data stack that took responsibility for actually executing the workflows that were supposed to be run based on the insights generated by the data meaning layer.

Now we are seeing more and more of the investment and focus shifting the system in a way that the modern data stack not only generates insights but also mobilizes fleets of agents on those insights to do some task.

Given this scenario, I wanted to rebrand and re-imagine the entire data meaning domain in a way that helps set the direction for how we should build products in this space that solve actual problems rather than adding narrow solutions.

I’ve written about the longer history of this answering-machine idea separately in The Prophecy of the Answering Machine.

The Obituary and Comeback #

Semantic layers are a shared platform that BI tools can connect to and find reusable SQL definitions of popular metrics in an enterprise. When enterprises struggled with dashboard sprawl in their orgs, one of the common problems was the divergence in metric definitions and lack of alignment across different departments. The number of customers metric would show up differently in dashboards used by finance vs engineering, because they both would calculate it differently. This is the problem semantic layers would solve under the assumption that most analytics was consumed via BI.

On these platforms, teams can define semantic models and metrics as structured YAML. These definitions are done in a way that makes it easy for machines to “compile metrics”, which is just exposing an API to generate a parameterised SQL so that the BI tool can render the number or chart. These were usually a pain to maintain because someone has to do the leg work of authoring the commonly agreed upon metric definitions across the org and keep updating these with changing consensus.

When ChatGPT came around and LLMs were able to generate SQL from natural language, a lot of people thought that the semantic layer as we knew it would not be needed anymore. This was presumed because we were witnessing the tremendous progress in raw model capability.

You could just ask your LLM questions and it would get perfect SQL queries you could run via MCP and get all the answers you needed. This phase did not last very long as we quickly ran into issues while generating real SQL queries directly from questions. The NL2SQL dream did not really pan out even after the exponential growth in model capabilities because every organisation has its own way of organising data and that information is not really available anywhere for the LLMs to pre-train on.

So naturally, we found LLMs generating plausible SQL for questions which were plausibly correct but actually wrong because the table with all the data about orders is not just orders but something like fct_customer_order_fulfillment_lifecycle_daily_snapshot_v2.

So now, the agents need direction and hand-holding.

All the talk-to-data prototypes encoded this direction in markdown files and then they ran into having to perpetually update it, because, like life, even in the data stack, the only constant is change. Tables get deleted, new tables come in, new dbt models are materialised, and those markdown files are never able to keep up with the constant drift that happens in production.

As LLMs became more and more capable, the semantic layer especially has become ripe for a rebirth.

From Metrics to Semantic Objects #

Traditionally modern data stack companies are approaching this problem by getting their old-world semantic layer systems to the agents by adding MCPs or APIs. The semantic layer of the old world was built to solve a central definition problem. Right now with agents I think we need to move semantic layers into a more latent space where they hold a lot more than just SQL metrics.

A lot of talk-to-data implementations usually sit on top of existing semantic layers, the idea being if your question is about a pre-existing metric, let’s just search, compile and execute the metric, instead of trying to generate the SQL from scratch.

Semantic layers need to serve more of the semantic workload than just be a metric compiler, so it becomes a place where agents can consume semantic context with drill down capabilities compared to just something that stores a parameterized version of your SQL query.

I think it’s about time we expand the scope of the semantic layer from being just a SQL translation layer to offering full fledged semantic objects that the harness can use.

SQL abstraction
+ metrics
+ dimensions
+ joins

Business ontology
+ entity resolution
+ relationships
+ metrics
+ events
+ concepts
+ policies
+ provenance
+ uncertainty
+ semantic functions
+ context compilation
+ execution planning

I also think one of the reasons the context layer in general has come out to be such a favoured solution is because continual learning in LLMs hasn’t really been solved.¹ You could try to fine-tune your own models with your organization’s semantic meanings, but that’s just expensive. Building a system that can provide the relevant context and tools to your agents is a much cheaper option to solve the problem.

Just like there was software 2.0, we are witnessing the second coming of the semantic layer, with a new name mostly. Folks are calling these automated context layers or ontology layers.

The Personal Data Factory #

The mission has been the same as always. There is a data store or a set of data stores and you need to build a machine that can do data analytics faster and cheaper, preferably with the help of agents.

The process of dreaming in memory systems would translate to agent loops that enrich these semantic objects based on the recent operations being done on them. If I am constantly asking my agent to tell me how much money I have spent in a given month, it should eventually be able to run a single SQL query and get that answer. The first time it can spend time figuring out the table etc. but in the next run, the semantic object needs to be enriched to point to the exact table and then perhaps in the next run it has a set of recommended filters that I have mostly used.

It’s like a piece of software that tries to embed meaning from immutable data. Modelling this is relatively trying to model “reality” inside your semantic layer.

This is the idea behind the Personal Data Factory. In the next post, I want to write about what I have built, and then after that write about how the system has changed with regular usage.

The semantic layer is dead. Long live the semantic layer.