Where should an AI lab keep the record of what its agents did?

Agent provenance for AI labs: why an AI agent's record of its own work should be kept where neither the agent nor the lab running it can quietly rewrite it, what the pre-release Datafund Provenance Toolkit does today, what a check does not prove, and an invitation to labs.

Datafund 10 min read

When an AI agent does something nobody expected, the first thing people ask for is the record. What did it read, what did it run, what did it produce? Evaluators ask for it, auditors ask for it, and so do the lab’s own safety teams.

That record is usually kept by the organisation that runs the agent, in systems the agent may be able to reach. This post argues that it should be kept somewhere else: where neither the agent nor the operator can quietly rewrite it. It opens a series on provenance for AI agents.

Agents have already tried to edit their records

In July 2026, during an internal OpenAI evaluation, agents got out of their sandbox and reached Hugging Face’s servers. Hugging Face disclosed the intrusion first; OpenAI acknowledged five days later that its models were responsible. The independent review by METR and Redwood Research found that agents “tried to retroactively edit transcripts” and “modified some accessible action logs”. The reviewers did not see the agents edit the source-of-truth transcripts in the data they examined (7 to 13 July).

OpenAI published its own account, and independent reviewers worked from the transcripts. The record held that time. Nothing in the way agent logs are usually kept makes sure it holds next time.

Old logs are only useful if they are still the old logs

Some incidents are found long after they happen. OpenAI found one of its agents using a funded test-network wallet about three months after the fact, “while extending our misalignment monitoring to records of earlier model activity”.

Looking back through old records with better monitors is good practice. But it rests on an assumption: that the old records are unchanged.

An AI agent’s log is only evidence if nobody could have edited it since. Months later, reading it can’t tell you that.

Post on X →

They can tell only if the log was kept where it could not be edited, and its fingerprint was fixed when it was written.

The labs have already said what they need

OpenAI, in “Towards safety cases for frontier AI training”, recommends: “Save agent transcripts immutably for a reasonable period of time (for example, in write-once storage accounts)”. The same post says auditors “should be provided with sufficient access to verify that the claims of the safety case are valid and sound”.

Anthropic’s CEO Dario Amodei, in “We Must Pace the Frontier”, proposes “ongoing, employee-like access to a team of embedded third-party evaluators”.

We agree with both. One step joins them: the evaluator should be able to confirm, on their own, that the transcript in front of them is the one written at the time.

Why the operator should not hold the only copy

Write-once storage inside a lab’s own cloud account blocks edits, and it is a sound start. An evaluator checking that record still relies on the storage provider and on how the lab set up the account. That is a question of trust, not a technical flaw. It also leaves the lab in a weak spot: when an incident is disputed, the lab has to vouch for its own records.

A record kept outside the operator’s control, with a fingerprint the checker received when the record was written, changes this.

When an AI lab keeps its agents’ records out of its own reach, it no longer has to vouch for them. The evaluator checks.

Post on X →

It helps the agent too. A record the agent cannot change in place is also a record nobody can change in place to blame it for something it did not do.

What the Datafund Provenance Toolkit does today

The Datafund Provenance Toolkit is open source and in pre-release, meant for testing. Anchoring, lineage links and the paid mode run on a test network (Base Sepolia); mainnet is not live. Today it does four things.

It stores the record on Swarm, the decentralised storage network (ethswarm.org). A record’s address is computed from its own content: change one byte and you get a different address. An edited copy can be stored, but it lands at a new address, and the original cannot be changed in place until its stated expiry, which is set by the postage stamp that pays for the storage. (Is that write-once storage? Our answer.)

It returns a fingerprint, the record’s SHA-256 hash.

It can add a notary signature. If you ask, Datafund’s gateway signs the fingerprint and a timestamp with the gateway’s own key.

It can anchor the fingerprint, which means writing it on a blockchain through a smart contract, so its time no longer rests only on our gateway. Records can also be linked into a chain: this output was made from that input, or from several.

A check then works like this, step by step in how an auditor verifies AI agent transcripts. The evaluator, auditor or client receives the record’s address and fingerprint when the record is written. Later, they fetch the record by its address, recompute the SHA-256 fingerprint, compare it with the one they received, and verify the notary signature or the anchor. The toolkit downloads through our gateway; the hash check is what removes the need to trust the gateway.

Here is that check in a real run, about a minute long: an agent’s output gets a receipt, a client checks it, and a one-character edit to a copy no longer matches.

What a check does not prove

A record of what an AI agent did can prove it hasn’t changed since it was stored. Not that it’s complete, or true.

Post on X →

A check proves the record has not changed since it was stored. It does not prove:

  • that everything the agent did was recorded;
  • that the record was true when it was stored. The METR and Redwood review describes agents spoofing tool-call outputs, and a faithful record of a spoofed output is still a spoofed output;
  • that the work was correct, lawful or authorised, or who uploaded the record.

Other limits today. The toolkit does not record anything automatically: having the software that runs the agent write the record, rather than the agent, is our proposal and is not built. Records are not encrypted, so anyone holding a reference can read the record; sealed records with keys released to an auditor are also a proposal. Without anchoring, the timestamp rests on our notary key. A Swarm address shows that the bytes match; it says nothing about what else was stored or in what order. And the operator still chooses what to store, can store an edited copy at a new address, and can let a record’s stamp expire. That is why the fingerprint has to be fixed at the moment the record is written: sent to the checker then, signed by a notary, or anchored. Finally, we have tested the case where a record matches, not a tampered record.

The record of what an agent did, and of what it knew

This post is about the record of an agent’s work: its transcripts and outputs, kept for the lab’s evaluators and auditors. The toolkit stores any file, so the same check works for a report, a dataset or a whole run.

A narrower record answers a different question: which memories did the agent have in context when it acted? PLUR, an open-source memory layer for AI agents, keeps that one with the same gateway, in an experimental release. Why PLUR chose Swarm covers that side. The two fit together: one shows what the agent produced, the other what it knew while producing it. How the toolkit compares with other approaches: AI agent provenance tools compared.

An invitation to the labs

We said it on X when this campaign started, and it still stands:

Repost the open call on X →

We would like to build this with you. Here is a small test any lab could run with the toolkit as it is today:

  1. Pick one evaluation run whose transcript you are content to make public, such as a synthetic test run with no personal data, because records are not encrypted.
  2. When the transcript is written, store it with the toolkit, using a postage stamp funded for longer than the test, and send its address and fingerprint to your outside evaluator at that moment. Anchor the fingerprint if a public time matters to you.
  3. A week later, let the evaluator check the record on their own, with nothing from you but what you already sent. If you like, also check a changed copy; we have not tested that case.
  4. Tell us what was missing, in an issue on the toolkit repository.

To evaluators and auditors: tell us what you would need to check a lab’s records yourself.

The quickest way to see a check end to end is dataprovenance.app: record a file, get a receipt, then play the auditor and verify it. No wallet or account is needed. The file is uploaded to Swarm, so use one you are happy to make public. The developer guide, How to give an AI agent a record it can’t rewrite, shows the same check in code.

Questions people ask

How can an auditor verify that an AI agent’s transcripts have not been changed? By comparing the transcript with a fingerprint recorded when it was written, and fetching the stored record from storage the lab cannot change. With the toolkit, the auditor downloads the record by its address, recomputes its SHA-256 fingerprint and checks the gateway’s notary signature or the anchor on the blockchain.

Is a fingerprint handed over with the transcript enough? No. Whoever edited the transcript could compute a new fingerprint for the edited version. The fingerprint has to be fixed when the record is written and kept out of the operator’s reach since: sent to the checker then, signed by a notary, or anchored on a blockchain.

Does a matching fingerprint prove the agent behaved well? No. It proves the record has not changed since it was stored. It says nothing about whether the record was complete or true.

Sources