Give agents a menu of vetted reports, not the power to write SQL

Jeff Smith has spent 25 years building database tools at Oracle. On the AI Engineer show he argued that agents reading company data should pick from approved reports. That removes one place the model can go wrong: writing SQL, the language databases use for queries. He described it as an option above natural language to SQL.

What was said

Smith called natural language to SQL amazing but dangerous. His worry was quiet failure. He said, "The LLMs are going to frequently run queries that look like they're returning the right data, but they're not." The query runs, so nothing flags a problem.

He listed other problems. The people using these tools do not necessarily understand what the agents are doing on their behalf. It is also expensive, he said. The agent combs the database's data dictionary, the catalogue of what it holds, which costs tokens, the units AI services charge by, and time.

Oracle's answer starts from something teams already own: years of compiled reports. Smith described loading them into the database or its MCP server (a standard way to give agents tools). Each report gets a description, sample questions and defined inputs. The inputs are parameterised, meaning the agent supplies values, while the code underneath is largely fixed.

Smith said he presents two tools: one that lists reports and one that executes them. He said 500 reports would not mean 500 tools, which would swamp the agent. His core point: "the LLM might screw up by picking the wrong report, but the LLM is not going to screw up by generating the wrong SQL."

Why it matters

Our reading: this design moves the failure rather than removing it. Free-form SQL can return a believable wrong number. A wrong report can return a believable number too, and a reader must know it is the wrong report to notice.

Smith said the agent can ask for reports matching a key phrase, then run one. This is our inference: the pick rests on how well each report is described, so vague descriptions would lead to wrong picks.

The wrong choice is visible only if the report name and inputs are logged and someone reviews them. Smith did not say that. Buyers should ask vendors whether tool calls are recorded, and whether free-form SQL can be switched off. Smith said Oracle's servers can be set to offer only custom tools and reports.

The other side

Smith called reports "one less place it can mess up." He did not claim the agent stops making mistakes, and it can still choose the wrong report.

There is a cost in effort. Someone must describe each report, its inputs and its sample questions. A menu also limits what can be asked. The excerpts do not say what happens when no report fits a question.

Smith described an optional mode as well. With the right scope in an authenticated session, the tool can show the agent the SQL behind a report, so it can write better queries itself. That reopens generated SQL. So the safety of the setup depends on how each team configures it.