Every AI agent eventually meets a request with a hole in it. The client did not say which materials grade they chose. The brief does not say bullets or paragraphs. The ticket does not name the account. The agent now has two moves: fill the hole with a default and carry on, or stop and ask. Many agents are built to do one of those every time, by reflex. One kind guesses and ships confident work that is wrong. The other asks about everything and turns a five-minute task into a conversation nobody wanted. Both reflexes have a price, and there is a better rule than either.
Two moves, usually picked by reflex.
The paper puts the choice plainly: faced with an incomplete request, "the system can commit early and risk being wrong, or it can acquire more context first."[1] Acquiring context is broader than asking a person. It covers a clarifying question, a lookup in a file or database, a tool call, or a quick test before the real run. Each one costs something: tokens, time, and, when the question goes to a person, that person's attention.
The reflexes come from how agents get built. A prototype that guesses demos well, because the demo request was complete. A team that has been burned by a confident wrong answer then adds "if anything is unclear, ask the user" to the instructions, and the agent starts asking about everything, because almost everything is a little unclear. Neither version decided anything. A good build decides per input, every time an input is missing.
When should an AI agent ask? When the answer is worth the question.
The paper's core result, stripped of its notation, is a single decision rule: spend on more context only when what the answer is expected to tell you is worth more than what getting it costs.[1] Decision theory has a name for this test: the value of information. The authors measure "what it tells you" as how much the answer would reduce the agent's uncertainty about the task, and the cost in tokens. As an operator, you can run the same test on any missing input with three plain questions: how likely is the default to be wrong, how much does a wrong guess cost to fix, and how much does the question cost to ask?
An AI agent should ask a clarifying question only when the chance its default is wrong, multiplied by the cost of fixing a wrong guess, is bigger than the cost of asking. If the default is almost always right, do not ask. If a wrong guess is cheap to fix, do not ask. If a file or a system already holds the answer, look it up instead, because a lookup costs a few tokens and no one's attention. Putting a human in the loop is the most expensive context there is, so a question to a person is the last resort, not the first move.
Weight it by what gets checked.
The obvious way to decide what to ask is to ask about whatever the agent is most unsure of. The paper adds a variant that weights each possible question by how much the final check actually looks at that detail, and reports that it raised the pass rate over asking nothing or asking at random, for a modest token cost.[1] The authors' own summary: asking a few well-chosen questions can be worthwhile, "but only when those questions buy information the final output will actually be judged on."[1]
In their writing task, the hidden preferences were tone, length and format, and the automated check scored length and format but never tone. An agent equally unsure about all three would happily spend a question on tone, and that question would buy nothing the check could see. The detail the reviewer, the client or the downstream system will reject the work over is the one worth a question; the detail nobody will ever look at is the one to default and move on.
The model cannot infer this on its own: the agent does not know what your reviewer checks unless someone writes it down. So the first job is yours: list what the finished work is judged on. That list is what tells the agent which gaps are worth a question.
A worked example: the quote agent.
Take a small renovation contractor that has an agent draft quotes from site-visit notes, about 100 a month. The owner reviews every quote before it goes out, against a short house checklist: the price lines match the materials grade the client chose, a validity period is stated, the client's address is right. Nobody checks the tone of the cover note. The figures below are illustrative, ours, and sized for a business like this one.
On a typical request four things can be missing: the materials grade, the validity period, the client's address and the tone of the cover note. Run each through the flow and the answers differ. The tone is never checked, so it gets the house default. The address is in the client file, so the agent looks it up. The validity period is checked, but the house standard of 30 days is almost never changed, so the agent uses it and says so. The materials grade is checked, often missing from the notes, and a wrong guess means redoing every price line. That is the one worth a question, and only when the notes do not already say it.
Read the chart from left to right. Asking nothing is the cheapest per quote and the costliest in rework: 55 of every 100 drafts go back to be redone. Asking everything gets the best first-pass rate, by two points, and pays for it with four times the extra tokens and 400 interruptions a month, mostly to answer questions whose answer was the default or was already in the file. Asking smart gets about 95% of the gain for a quarter of the extra tokens. The token difference is small money for a business this size. The interruptions are not: 400 a month is roughly twenty a working day, the owner answering the agent that was supposed to take work off their plate.
Our note on cost as a first-class metric argues for logging cost per agent step. Questions to a person belong in that log too, priced in minutes, because for a small team they are often the bigger cost.
Do not count on the model to work this out alone.
It is tempting to leave all of this to the model: give it a strong system prompt, tell it to ask when it needs to, and trust it to find the efficient questions. The paper is a reason not to. It ran current frontier models from several vendors through its twenty-questions game, where the best possible questioning strategy can be computed exactly and every answer is true. The models narrowed down the answer steadily, and yet stayed less question-efficient than the best strategy.[1] This is not a ranking of models and should not be read as one. The finding is that none of them were question-efficient out of the box, on a task built to make efficiency easy to see.
A real request is harder than that game: answers are vague, the person changes their mind, and the facts are spread across files the model cannot see. The authors say as much about their own benchmark.[1] If models over-ask on the easy version, assume yours will over-ask (or under-ask) on the real one, and decide the policy yourself. A goal that moves mid-conversation is its own failure, covered in our note on changing your mind; a well-placed early question is one of the cheapest ways to avoid it.
Put it in the agent's instructions.
In practice, all of this becomes a short table in the agent's instructions: every input the task can be missing, whether the finished work is checked on it, where the answer lives, the default, and what the agent does. For the quote agent it looks like this.
| missing input | checked? | where the answer lives | default | agent does |
|---|---|---|---|---|
| cover-note tone | no | nowhere needed | house tone | proceed never asks |
| client address | yes | the client file | none | look it up asks only if the file is empty |
| validity period | yes | house standard | 30 days | proceed states the default on the quote |
| materials grade | yes | the site notes, sometimes | standard | ask when the notes do not say, one question |
Four habits make the table work in practice:
- Write down what the work is judged on.
The reviewer's checklist, the downstream system's required fields, the client's standing rules. This is the weight the paper is talking about, and only you can supply it.
- Give the agent its lookups before its questions.
Connect the client file, the price list, last month's quote. Every answer the agent can look up is a question nobody has to answer. When it has a tool, require it to ask for a missing required value rather than invent one; a made-up argument to a tool is a guess that looks like data.
- Batch the questions and propose the default.
One message with the one or two questions that matter, each with the answer the agent will use if nobody replies, beats a drip of single questions. The person can answer in a word or not at all.
target: one interruption per task, at most - Count the questions, then tune.
Log every question the agent asks and whether the answer changed the output. A question whose answer is always the default should become a default. A default that keeps getting the work sent back should become a question.
The other half of this decision is when to stop gathering and act, which the paper treats as part of the same loop. Our note on a planner that knows when to give up covers that side.
What this does not claim.
No figure from the paper. The paper reports results from small synthetic tests with simulated answers and tools switched off on purpose. We do not repeat its numbers. Every number in this note comes from our illustrative quote-agent example.
What the benchmark can and cannot show. The twenty-questions game is deliberately simple: noiseless table-lookup answers, a fixed list of questions, no real tools. That is what makes its result clean, and also why it says little about any specific product. We use it for the rule and for one direction (models ask more than they need to), not for magnitudes.
No model ranking. The paper names the models it tested. We do not, because the point is a design rule for your agent, not a verdict on a vendor, and hosted models change underneath any such result.
One line on the other use. The authors note that the same technique that helps an assistant ask fewer, better questions could help someone pull sensitive details out of a person. If your agent talks to the public, put limits on what it may ask about.
If your team is building an agent that keeps asking, or one that never does, the contact form is the fastest way in. Tell us what the agent does and who reviews its work, and we will send back a written read on which of its questions are worth keeping, free. Writing rules like these into an agent's instructions is part of our assistants and agents work.