Agentic Dataset Reference
A framework-independent behavioural contract for governed, agent-facing datasets.
15 normative assertions, 85 language-neutral vector steps
reference architecture 4 runtimes x 2 dataset boundaries 15/15 each
independent implementation shares no code with the above 15/15
mutation analysis 19 / 19 targeted violations detected
15 / 15 assertions independently exercised
2.2 detecting assertions per mutant (mean)
execution safety 0 / 576 prohibited executions
0 / 24 in the evaluation set
tests 407 passed
Authorized Recall@5 filter after truncation 0.853
filter before truncation 0.960
+0.107
Start here
An agentic dataset describes itself, advertises bounded capabilities, accepts a semantic intent, decides whether an action is admissible, executes only what was admitted, refuses the rest, and leaves evidence.
The specification is fifteen assertions, each naming a failure it rules out. Everything else exists to check them.
GRANTED -> approval token -> execution reachable
REFUSED -> no token -> execution unreachable
INDETERMINATE -> no token -> execution unreachable
INDETERMINATE is not a refusal. An evaluator that is unreachable or out of
budget has not decided anything, and recording that as a refusal invents an
authority nobody exercised.
Check an implementation
pip install agentic-dataset-conformance
agentic-dataset-conformance run --subject yourmodule:your_factory
The harness imports no implementation — not even the reference one — and a test asserts it. The worlds and vectors are CC0; export them and write a runner in another language if you prefer.
What this is not
No deployment, no real data, no model in the loop by default, no latency or
cost claim, and no security guarantee. capabilities() is the subject's own
report of itself, so a subject that under-reports passes AD-002 while hiding a
tool. Conformance is a claim an implementation makes about itself, made
checkable — not an adversarial audit.
Claims is frozen and lists everything asserted, and the one thing
explicitly not.
Status
Released. v0.2.0 is the current version, on PyPI and archived with its own DOI (citation); v0.1.0 was the first. A finding changes the next version rather than an old one. The independent implementation is a 250-line toy written by the same
person who wrote the specification: that establishes independence from the
reference code, not from its author's reading of the contract.
Interpretive independence is the next threshold, and it needs somebody else. How to do it.
Where this is published
This documentation is served from two addresses, both maintained and both carrying the same build:
| https://agenticdatasets.org/reference/ | canonical — cite this one |
| https://agentic-datasets.github.io/reference/ | GitHub Pages mirror |
They are independent copies rather than a redirect pair. The domain resolves to its own host, so Pages cannot serve it and neither address forwards to the other; if you are reading one of them, the other has identical content.
Use the domain for anything durable — a citation, a DOI, a reference in a paper — because it is the address that survives a change of hosting. The mirror exists so the documentation stays reachable if the domain does not.