When agents build the software, people must judge what the rules mean

When coding agents do the building, the scarce skill is judging the spec. That is our argument, drawn from a talk by Rajath Mahendrappa Kotyal of AWS on the AI Engineer show. He described a tool in which the model and a solver only surface unclear rules. The human decides what a rule means.

What was said

Kotyal presented a checker for Kiro that tests a specification before any code is written. The show notes say a coding agent will faithfully build every flaw it is given. His example was an online shop. One rule sent orders for out-of-stock items to back order. Another rule banned back orders. Each reads fine alone, but both cannot hold.

In his pipeline the tool does the finding. A model first tries to tidy the wording. If that fails, the tool turns the English into formulas for a solver, a program that checks whether rules can all be true at once. It also groups the model's different readings of a line. Disagreement between those readings points to a vague rule.

Then a person chooses. For a rule that says to remove a record, the user picks between deleting it and keeping it hidden. Kotyal said: "The LLM can interpret anything it wants but we shouldn't allow that to happen." His summary: "The LLM translates, the solver checks, and the human decide."

Why it matters

Our reading: detection can be automated, but choosing cannot. Which rule wins depends on how the business works. Kotyal said the model lacks that domain knowledge. So the person's job shrinks from spotting the problem to arbitrating it.

For managers, that changes what to train and hire for. Reviewers who once read code now need to read requirements and make calls between competing readings. People who know the business become more valuable, because the agent will build whichever reading it is handed. A spec nobody has judged simply gets built faster.

The other side

Kotyal was clear that the tool does not remove the burden. He said nothing but the user can confirm a specification is correct, and the user must check every part of the rule book. A tired or rushed reviewer could still wave a bad rule through.

He also said the evidence for using formal methods, the maths-based way of proving rules consistent, in agents is thin and still emerging. When the user gives no answer, the tool uses LLM judges or declines to implement that piece, so the human is not always the last word. The talk did not say how people learn this judging work, or how well they do it.