A wealth-firm AI chief says models confidently use outdated tax brackets

On the Latent Space podcast, Aman Singh, who leads AI at the wealth firm Arca, described a risk he sees in money work. In his experience, models use outdated figures without hesitation. Arca adds a check that steers the model to verified facts or web search when financial rigor is needed.

What was said

Singh spoke with host Richard MacManus at AI Engineer New York. He said that when models handle financial data, they show great confidence in what they already know. He called his example a small one. In his words, "models will very happily use the tax brackets from twenty twenty-five" to build an analysis. He called that plainly wrong.

Arca's fix sits in its harness, the custom software that wraps around its AI agents. Singh said a separate decision model spots when the agent is close to giving money advice or taking a step that demands precision. It then nudges the agent toward Arca's knowledge base of verified facts about what is true today. It can also send the agent to web search.

Singh added that the firm's safety rules work differently. He said "all of our safety restrictions are, are fully deterministic, and they, they take the decision-making off the, the agent's hands." He also described a nightly process. A meta agent reviews real traces and suggests harness changes, including missing knowledge in the knowledge base.

Why it matters

Our reading: Arca's approach suggests that a firm using AI for tax, pricing, payroll or advice should not rely on the model's own memory. It needs a source of facts that a named person keeps current. It also needs a check that fires when money is involved. Singh did not argue that better models would or would not remove this need. That part is our opinion.

For buyers, that suggests simple questions to put to a vendor. Where do the figures come from? Who updates them? What happens when the model ignores the source? Singh's description also shows the knowledge base is never finished. His team feeds gaps from live use back into it.

The other side

Singh offered the tax brackets as one small example. He gave no figure for how often models get such facts wrong. He also did not explain how the decision model spots a moment that needs rigor. Describing it as a nudge, he left open whether the model can still ignore it.

Arca also has advantages many buyers lack. Singh said Arca is not a SaaS company but the firm itself, and he tied the ability to build these checks to owning the harness. A firm renting an off-the-shelf tool may not be able to add them.

Arca also limits the risk elsewhere. Singh said its agents do not trade on their own. They propose trades, and the firm's experts decide and execute them.