- Flywire runs AI model steps inside fixed workflows with limited access, and its engineers review the workflow files and the merge requests the agents produce.
- Quality checks are still young: a speaker said new agents usually go live after a vibe check, and golden datasets are only run on demand.
Flywire's Orbit platform fences model steps inside fixed workflows. That suggests engineering work moves from writing the change to defining the workflow and judging the output. Eduardo Simon and Javier Casal of Flywire described the platform on the AI Engineer show. They described its design, not a shift in skills. That link is ours.
What was said
Each Orbit agent is defined in one YAML file, a plain-text configuration format, kept in the company's repository. Colleagues review any change to that file just as they review ordinary software changes. A merge request is a proposed code change that colleagues check before it is accepted.
The code review agent shows the pattern. Fixed code steps fetch the merge request details, rate its risk and copy the repository. Then one model step reads the code. It cannot change it, and it must answer in a strict format. A final code step posts the findings. The speakers said this agent completed over 9,700 runs in 30 days.
People stay in the loop. A speaker on the show said: "We operate in a regulated environment, so people still review and approve those merge requests."
Judging quality is the harder part. A speaker was frank about it: "Today, a new agent usually reaches production after a vibe check." After launch, the team traces real runs and adds them to a golden dataset, a saved set of real cases used for testing. The dataset sits in the repository next to the code. The team reruns it after changing a prompt or a model.
Why it matters
Our reading: the work that remains for engineers looks different. Reviewing a workflow file means deciding which tools and access each step gets. Building a golden dataset means choosing which real cases count as good evidence. Both are acts of judgment, not typing.
For managers, this raises a practical question. Do your reviewers know how to read a workflow definition and spot a step with too much access? Can they say whether a test set covers the cases that matter? The Flywire description suggests these checks sit where the code-writing used to be.
The other side
The speakers were open about gaps. They said they do not yet have online evaluations, which would test agents in live use. They also said the setup is not full sandboxing yet, though they are moving toward it. Golden datasets run on demand, not continuously.
The talk also left questions open. It did not say how review workload changes when agents open many merge requests, or how engineers learn this judging skill. A speaker linked human review to Flywire's regulated setting, so other companies may choose differently.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- AI Engineer: Agents in Orbit: Workflows, Not Autopilot — Eduardo Simon & Javier Casal, Flywire (2026-10-12)
