Ask a business that has automated a few tasks with AI what one of those automations actually does, and the honest answer is often "let me check." The inbox sorter is a script a freelancer wrote. The quote drafter is a chain of steps in a no-code tool. The invoice reminder lives in a vendor dashboard. Each one works, and none of them can be checked without reading code or clicking through screens, so nobody checks. In August, an author who builds these systems inside large European companies published a paper arguing for a different build, one where checking an automation means reading one plain-language file.[1] It is written for enterprises, but the idea at its centre scales down well.
If you want the checklist first, skip to the five questions.
Cheap to build, expensive to read.
The paper opens on a cost that anyone who has hired a freelancer for automation will recognize. AI models made writing custom code cheap: a problem too small to ever earn a budget now takes an afternoon. "The cost of reviewing, understanding, and maintaining that code afterwards hasn't collapsed at all."[1] Every solution drifts from the next, and understanding one means reading it from scratch.
Large companies answer that cost by refusing: a central team builds everything, on one framework per use case, and a review board decides who may build at all. A small business answers it by not looking. The automation runs until something goes wrong, and then someone discovers what it was doing. Neither answer is good. The paper's proposal is to make the thing cheap to read, so that looking becomes routine.
One runtime, one file per job.
An agent runtime (the paper calls it a harness) is the software wrapped around an AI model that lets it work on its own: the loop that keeps it going, its memory, its access to files and its command line. Claude Code, which we use, is one. The proposal takes one move. Run the same runtime, unmodified, behind every automation. The only things that differ from one job to the next are a plain-language instructions file and a short configuration file. Then, in the paper's words, "auditing an enterprise's N agent solutions collapses to reviewing N version-controlled instruction files."[1] Each file, the author adds, "is shorter and more legible than the pull request it would otherwise have taken to ship the same change."[1]
The deployment path is "fork, configure, push"[1]: copy the shared template, fill in its files, publish the change. Whoever owns the problem runs the agent live against the real task first, watches what works, and writes that down as the instructions file. Anything reusable goes into a separate skill document. A handful of fields go in the configuration: who owns it, what problem it solves, what it may reach, which model, when it runs. The paper's own example, abridged:
owner: crm-team@company.com
problem: "Triage inbound CRM requests and draft first-pass responses"
tool_groups: [crm-triage, sharepoint-readonly]
skills: [crm-triage-playbook, sharepoint-search-patterns]
business_area: customer-service
model: claude-sonnet-5
schedule: "*/15 * * * *"
trigger: cronThis is close to how we set up our own work: one agent runtime, and per job an instructions file and a small set of skills, with the person accountable for the job reading those files. It is also why, in September, we could audit the 163 instruction files in our own toolkit against the vendor's new prompting guides at all: the behaviour we wanted to check was written down in plain text. That audit is in our note on AI instructions having a shelf life.
Four mechanisms, minus the enterprise plumbing.
The paper builds four mechanisms on that idea. Most of the machinery (directory groups on Azure, a tool gateway, two registries, a container per deployment) is out of reach and out of proportion for a small business. The shape underneath each one is not.
| mechanism | the paper's enterprise version | what a small business keeps |
|---|---|---|
| access at the credential | Each backend system gets one generic request tool and a scoped, identity-bound credential from a gateway; "access is controlled at the credential, not policed operation by operation."[1] | Each automation runs under its own account, with only the access its job needs (what security people call least privilege): read-only where reading is enough. |
| authorization outside the agent | A gateway's permission file, edited by an administrator and kept under version control, is the only thing deciding what a caller can see and call. | Permissions live in the systems you already use (the mailbox, the drive, the accounting software), never only in the prompt. |
| registration on deploy | Shipping an automation registers it automatically (owner, purpose, tools, business area) and runs overlap and risk checks first. | One list: each automation, its owner, its purpose, its file, the access it holds. |
| a fresh judge for risky calls | Every action carries a "risky" flag the agent sets itself; a flagged action is sent to a fresh, separate instance of the agent to review, and only what that review cannot clear reaches a person. | Decide in advance which actions wait for a person (sending to a client, paying, deleting). |
The paper sums up the fourth mechanism as "a fresh context judging the call instead of the one that composed it grading its own work."[1] Our note on AI judges grading their own homework covers why that separation matters. One thing to notice, and this reading is ours: the review only triggers on calls the agent itself flags as risky. A risky call it fails to flag goes straight through, which is why the access grant, not the judge, has to be the hard limit.
Reach is an access decision, behaviour is a file.
Split that way, a hard question about any automation that touches client information (from a client, from your insurer, or in a privacy review under Quebec's Law 25) comes apart into two easy ones. What can it reach? That is a list of access grants, decided outside the agent, that the agent cannot widen. What does it do with what it reaches? That is a file you can print and read.
The practical rule falls out of the split. "Do not open the payroll folder" written in an instructions file is a request. No access to the payroll folder is a control. Put the limits that matter in the access grant, and use the file for how the job should be done. Our note on an AI agent running an ad account applies the same split to one system: the account that reads is not the account that writes.
Runtime or model: what moved the score?
The paper's case for one shared runtime leans on a claim that the runtime around a model "accounts for most of the variance in agent benchmark results, more than model choice does," citing a May 2026 paper.[1] We read that paper.[2] Its claim is narrower, and the narrowing is the useful part. It covers long, multi-step tasks, compared across models of similar capability.
Its controlled test took three models picked because they sat within about a point of each other on a coding leaderboard, and three runtime setups built to span the range: a minimal one with no retries and no checks, an improved one with trimmed context and structured retries, and a full one that adds a self-check at each step, drift checks and rollback. Each pair ran on 100 software bug-fix tasks, twice.
Inside that grid, changing the runtime moved a model's score by 8.5 to 13 points, while changing the model inside one runtime moved it by 2.5 to 5. Each setup had a different top model (GPT-5.4 under the minimal one, Kimi K2.6 under the improved one, GLM-5.1 under the full one), and across the three pairs of models, each compared over three changes of setup, the order reversed in six of nine comparisons.[2] The authors are careful to call this a small grid whose range was designed to be wide, and they do not claim the ratio is universal.
Now set the second measurement beside it. Tencent reports that its Hy3 model's scores on the same benchmark stay within 4% across three mature agent tools.[3] That is self-reported, and it measures something different: the spread across three finished tools, not the spread from no checking loop to a full one. Both can be true. Between a bare loop and one that checks its work, the runtime can matter as much as the model, or more. Between good finished tools, much less. One more detail from the grid: the spread between models was widest under the fullest runtime (5.0 points), so a better runtime does not make the model irrelevant.
For a business, two things follow. Before paying for a bigger model, look at what surrounds the one you have: whether it retries, checks its own steps, and stops cleanly. And any comparison of models you run inside one setup holds for that setup only. Our note on switching AI models mid-task covers a related cost.
Where a fixed workflow still wins.
The paper treats designed workflows (graph frameworks, low-code platforms) as the pattern to replace, because a path fixed at build time cannot improve when the model does. We draw the line differently. Steps that need no judgment (a schedule, a webhook, copying a record from one system to another) belong in a workflow tool such as n8n, where every run takes the same path and that sameness is the point. Steps that need judgment (read this email, decide what it is, draft the reply) are where a general agent with an instructions file earns its place. Our notes on custom orchestrators and on n8n's licence cover the workflow side.
The paper's durability argument also needs a qualifier. It says every model upgrade improves every deployed automation "with zero code changes."[1] An upgrade can also make old instructions misfire: the vendor's own guides for its newer models say instructions written for older ones can degrade the output, and that warning is what our September audit was checking for. Reading the files is the maintenance as well as the audit.
How do you audit an AI automation? Five questions.
Run this test on any AI automation you already have; it needs no new software. These questions are ours, built on the paper's design; the paper does not propose them in this form.
- Can you name the file that says what it does?
For each automation, one plain-language file should say what it does, for whom, and what it must never do. If the behaviour lives in code or in a vendor's dashboard, write the file anyway, from what the automation actually does today. Writing it is often the first time anyone has checked. Our note on grading the brief before you blame the AI covers how to judge whether the file itself is clear enough to work from.
→ one file per automation, in words its owner can read - Is its access listed outside that file?
What it may reach (which mailbox, which folder, read or write) should be granted to the account it runs under, not promised in its instructions. If the only thing keeping it out of a system is a sentence in a prompt, that system is open to it.
→ its own account, its own list of grants - Is it on one list, with an owner and a history?
Keep one list of every automation you run: its owner, its purpose, its file, and the access it holds (the small-business version of the paper's registry). Keep the files where changes are dated and attributed (a version-controlled folder, or at minimum a shared drive with version history), with one named owner per automation. When the behaviour changes, you can see who changed it and when.
→ one list: automation, owner, purpose, file, access - Which actions wait for a person?
Decide in advance which actions never run alone: sending to a client, paying, deleting. A second review before a person sees it is a good filter, but only for the actions the automation flags; the access grant still has to be the hard stop.
→ a short list of actions that never run alone - Do you re-read the file when the model changes?
A model upgrade is a change to every automation at once. Re-read the files against the new model's guidance, and run a few known cases before trusting the result.
→ upgrade day is review day
What this does not claim.
A position paper with no benchmark. The author says so: no comparison was run for the paper itself. Its evidence base is engagements at European companies in a handful of sectors, built on Azure, and the author calls portability to other clouds "still an open claim."[1]
One author, with a package to promote. The paper presents the author's own software as its reference implementation. Its claims that simple agent runtimes are enough for business work rest on papers it cites and we have not read.
Reading the file is not the same as safe. The paper concedes it has no dedicated mechanism for four known agent failures (lazy shortcuts, invented system state, dropped constraints, overconfidence); running the task live first and writing down what worked is its only mitigation. And nothing in its design scans the skill documents an agent loads for injected instructions. A file audit only covers what the files say. Checking what a run actually produced is a separate job: see our note on a healthy run with a wrong answer.
New work only. "New work built this way is governed by construction; old work isn't touched."[1] Automations you already run stay as readable as they are today until you write their files.
The grid is small, and about code. Three models, three setups, 100 bug-fix tasks, two runs, and a range chosen to be wide. Hy3's 4% is the vendor's own figure over a narrower range. Neither licenses "the runtime matters more than the model" without the range attached.
One claim we leave out. The paper also argues for a particular way of defining the tools an agent calls. Our own experience runs the other way, and the paper concedes the claim is untested beyond well-documented public systems, so we do not carry it here.
Running a few AI automations and not sure anyone could say what they do? Describe them through the contact form and you get a free written first read on which ones need a file, an owner and their own access first. Work like this falls under our assistants and agents services.