An AWS speaker says unclear prompts cut code accuracy and wants specs checked first

Rajath Mahendrappa Kotyal of AWS argued on the AI Engineer show that the written rules given to a coding agent should be checked before the agent builds from them. The show notes say an agent will faithfully build every flaw in a spec. Kotyal cited research, which he did not name, that unclear prompts cut code accuracy by 20 to 40 percent. He applied that worry to specs.

What was said

Kotyal gave an e-commerce example. One rule says that when an order arrives and stock is out, the shop places it as a back order. A second rule in the same file says no order may be placed as a back order.

He said each rule reads sensibly alone. A language model reading both sees the clash: "both cannot hold true at the same time." His team's tool first lets the model try to clean up the wording. If that fails, the rules become formal logic statements. A solver, a program that checks whether statements can all be true together, then tests them.

The tool also asks the model to translate a sentence seven or ten times and groups the results by meaning. The show notes call the model's own disagreement a signal of ambiguity. Kotyal summed up the roles: "The LLM translates, the solver checks, and the human decide."

Why it matters

Our reading: the cheapest moment to catch a rule clash is before an agent builds on it. A person who meets two clashing rules can stop and ask. An agent may pick one rule or blend them without saying so. Kotyal did not claim this; it is our inference.

The solver step matters because it answers a narrow question: can these rules all be true at once? The answer is true or false. Kotyal said a model alone gives no guarantee.

For a budget signer, that suggests questions to ask. Who owns each spec? Who reads it first? Is there a consistency check before an agent starts? Who decides when the check flags a rule?

The other side

Kotyal did not claim the tool replaces people. He said the model lacks much domain knowledge, and only the user can give a guarantee. The final call stays with a human who knows the business.

The excerpts also leave gaps. His accuracy figure came from research he called public and recent, and he did not name it. His example is an obvious clash, and the excerpts do not show how well the checker finds subtler ones. The show notes say the talk covers where solvers beat a model acting as judge, and where they do not. The excerpts omit that part.