Help Net Security reported on 21 October that Google released TorchTPU as open source. It lets developers who build models in PyTorch, a popular AI framework, run them on Google's TPUs, its custom AI chips. Meta says key production systems for language models, ads and recommendations already run on TPUs this way. The code is free on GitHub.
Developers debug in eager mode, where each step runs at once, then use torch.compile to build a tuned graph for production. Google says models move with minimal code changes. In a training test on Qwen 3 8B, TorchTitan reached up to 52% model FLOPs utilization, about half of a chip's theoretical math capacity. It scaled almost in step with chip count across up to 8,192 Ironwood TPUs. A separate fourfold gain came from llm-d, an inference router, not from TorchTPU. Zenteiq.ai says the gain comes from sending each request to a replica that already has its context loaded. The sources give no independent test and no GPU price comparison.
If porting cost falls, the choice turns on price, performance and running costs. The sources answer none of these yet. Before a pilot, decide whether your workload is training, like the Qwen 3 8B test, or inference, like the llm-d case. Then measure your current GPU utilization to compare with the 52% figure.