Write-Audit-Publish for Analytics Agents
Ju Data Engineering Weekly - Ep 103
Bonjour!
I’m Julien, freelance data engineer based in Geneva 🇨🇭.
Every week, I research and share ideas about the data and AI engineering craft.
Not subscribed yet?
Analytics agents make it easy to generate answers, charts, and dashboards on demand.
That is useful.
But it is also unpredictable.
Nobody knows how to make agents 100% reliable today, so we need to live with that uncertainty and manage the risk properly.
One way to manage that risk is to add a proper audit process: a way to cross-check, replay, and inspect the computation behind an agent output.
In this post, I first look at how more mature engineering workflows solved a similar problem in software and data engineering with WAP (Write-Audit-Publish) pattern, then explore how to adapt the same idea to analytics agents.
Thanks to Hussain and the Xorq team for supporting this exploration.
I know Hussain personally and have been working with him over the last few months.
He is a brillant engineer and is eager to talk with people interested by LLM reliability for analytics.
Don’t hesitate to book a quick call with him here:
And to get on Xorq’s waiting list.
1. WAP & Software Engineering
Ensuring safe code releases is a well-known problem in software engineering.
One possible strategy is blue-green deployment, where two identical environments are set up.
The blue environment hosts the current application version, while the green one runs the new version.
Once testing has been completed on the green environment, live application traffic is directed to the green environment, and the blue environment is deprecated.
By allowing fast rollback, we reduce the risk of new code reaching production.
2. WAP & Data Engineering
Data engineering faced the same problem, but with another challenge: our code, like SQL transformations, is closely tied to the underlying data it processes.
Traditionally in data engineering, we run materialization and testing sequentially:
First, the data is materialized (e.g. dbt run).
Then, it undergoes testing (e.g. dbt test).
The problem? If the test fails, we already have bad data in production.
To solve this, a pattern emerged: WAP, for Write-Audit-Publish — or blue-green deployment for data engineering.
With WAP, data is staged in a temporary environment for testing before being released to production:
Models are materialized in a temporary environment.
Data quality tests are run there.
If the tests pass, the models are moved to the production environment and made available to downstream consumers.
Various implementations of this pattern exist:
in orchestrators directly, with dbt schema swaps or SQLMesh view pointers;
in warehouses, with zero-copy clones;
in the storage layer, with Iceberg branches.
3. The agentic analytics shift
AI is now changing how we structure data platforms.
Before, the role of a data team was to expose highly refined artifacts:
Today, we can rely on agents to do the last-mile customization for our users: we increasingly expose raw artifacts and let agents refine them on demand.
This adds flexibility, but also much more risk: LLMs are not deterministic, so we need to add a new validation layer.
Compared with WAP in software and data engineering, this validation loop is more complex because it lives at the edge, inside the last-mile generation workflow, not inside a space the development team fully owns.
In the agentic world, we have already started WAP in some places:
coding: first we ask the agent to create a plan, then we review it, and only then do we let it execute.
agent skills: plan-validate-execute
But how would this work for an agentic analytics workflow?
4. WAP & Analytics Agents
A typical data-agent loop is split into three phases:
Interpret the user intent and map it to context definitions.
“What were our sales last quarter?”
becomes:
sales = the revenue column in the sales table;
last quarter = a date filter between X and Y.Perform the request, or sequence of requests.
Render it to the user: inline chat, dashboard, chart, etc.
To implement WAP for an analytics agent, we need two building blocks:
Write: a way to persist and replay the ad hoc transformation sequence created by the agent.
Audit: once a transformation-chain artifact has been generated, a way to test it before publishing.
The important nuance is that “correctness” is not one thing.
It breaks down into 3 layers:
Interpretation: did the model understood the user request correctly ?
Trust: did the computation use trusted sources, such as a semantic model or catalog object that a human has already approved?
Faithfulness: given the computation that was actually run, did the agent render the answer faithfully, without inventing numbers or claims?
Let’s look at a real-world WAP implementation with Xorq.
5. Xorq as the audit layer
Xorq is an open-source framework built on top of Ibis that makes it possible to persist data pipelines as replayable, shareable artifacts.
For example, if a user asks:
How many cars did we sell last month?
The agent produces Ibis chained expressions instead of a one-off SQL string:
cars_sold = (
sales
.filter(_.sold_at.between(start_of_last_month, end_of_last_month))
.aggregate(cars_sold=_.sale_id.nunique())
)But why not just use a SQL query?
Because this expression is actually a DAG under the hood where each node carries its own schema and can be composed with other nodes.
Write → Capture the agent's work
On top of the Ibis nodes, Xorq has introduced a primitive called TeeNode that makes it possible to store an expression output in a temporary place:
a Parquet file;
a database table;
an Iceberg branch.
candidate = cars_sold.tee(make_sink_with_parquet(staging_path))This is the Write step of WAP: the output of candidate is still the same table as cars_sold, but the rows have also been staged somewhere else.
Audit → Validate the agent's work
So what about the Audit step now ?
The philosophy of Xorq is that an agent turn should produce two artifacts:
the result expression = the computation that produces the answer.
the audit expression = the computation that proves the answer is grounded.
Like the result itself, the audit is also an Ibis expression that can be replayed, inspected, and shared:
wap_expr = (
cars_sold
.tee(make_sink_with_parquet(staging_path))
.aggregate(
data_passed=audit_expr(cars_sold, audit_fn=data_audit, name="data_passed")
)
)Xorq focuses its audit layer on two of the three failure modes we discussed earlier: trust and faithfulness.
The audit expression captures both:
Lineage: proving that the result was derived from trusted data and semantic objects.
Verification: a replayable certificate showing how the final claim was validated against the computed result.
For example, if the agent answers:
The busiest route is JFK-LAX, with 1,757 flights.
The associated verification expression filters and aggregates the underlying computation to prove that the value 1,757 is indeed present in the computed result.
This is not just a log. It is a replayable artifact that can be inspected, shared, and rerun.
It is the equivalent of asking the agent to "write a Python script that proves your answer," but in a systematic, token-efficient, and composable way.
Audit is a first-class citizen in their UI:
Each agentic analytics answer displays a verification block with the detailed audit that was run:
Because the system is composable, we can also chain other kinds of verification:
PII checks: is the agent using PII columns as expected?
LLM-as-judge checks: use an external LLM to spot interpretation errors.
Publish → Share trusted work
Once the result has been audited, it can be safely presented to the user in the most appropriate format: a text answer, a chart, a dashboard, or another visualization.
But publishing can go one step further than simply rendering the answer.
Because both the result and the audit are represented as Ibis expressions, they remain replayable, composable, and reusable.
Rather than disappearing after an answer has been delivered, they can be published to a shared catalog, where other people and agents can discover, inspect, and build on top of them.
Compound Analytics = Composability + Reuse
I really like the combination of WAP + Ibis/Xorq expressions:
WAP gives analytics agents trust.
Replayable expressions give them memory.
Together, they create a true decentralized analytics flywheel.
Instead of every agent starting from raw data, they build on an ever-growing collection of trusted, audited computations.
That's what I call Compound Analytics: a world where knowledge compounds because validated computations become reusable building blocks instead of disposable answers.
Thanks for reading and Xorq team for the support,
Ju
