Google’s Inference Cage
A couple of weeks ago, I highlighted a growing tension inside Google, one with the potential to push the company’s culture past a point of no return.
The TPU Trap
Yesterday, I explained the transformation Google is going through, one that is reshaping the company at its core and challenging the business model assumptions that defined its success for nearly two decades.
Follow me on The AI Supercycle as well!
That tension emerged from the breakdown of a long-standing assumption: Google built hardware primarily for its own internal needs. The rise of what may become the largest technology market in history, AI inference, has fundamentally changed that logic.
Commercializing Google’s TPUs, particularly for closed frontier AI model providers, makes clear strategic and financial sense. But it also introduces a different operating model, new external dependencies, and significant internal pressure.
The commercial opportunity is obvious. The cultural cost may be less visible, but it could prove just as consequential.
On August 5, in a single memo from Sundar Pichai, Google rewired the top of its artificial-intelligence organization and lost the most senior technical figure in its history on the same news cycle. The two events were announced together, and they were not separate stories.
The leadership changes:
Demis Hassabis is stepping back from running Google DeepMind day to day. He becomes Chair of Google DeepMind and, one level up, Chief Scientist of Alphabet — an advisory, AGI-strategy mandate. He keeps Isomorphic Labs, the drug-discovery spinout, stays based in London, and will continue working with Pichai on long-horizon questions. He is not leaving. He is moving from operator to statesman.
Koray Kavukcuoglu takes over the engine room — as Senior Vice President, not CEO. DeepMind’s longtime chief technology officer and Google’s chief AI architect, a thirteen-year veteran who built the lab’s deep-learning team and helped ship WaveNet and DQN, will now own Gemini model development, frontier research, the Gemini app, and Google’s AI developer platforms, reporting directly to Pichai. His first and defining task is Gemini 4.
The two title moves point in opposite directions, and that is the story. The chief-scientist mantle floats up to the Alphabet level for the man ascending; the chief-executive title of Google DeepMind is quietly retired for the man taking the controls. The person now responsible for building Google’s next flagship model holds a rank below the one his predecessor held. In an organization sending a signal about where AI sits in its priorities, the rank of the job that builds the model just went down.
Then the departures — and they are not ordinary ones.
Jeff Dean is leaving after twenty-seven years. Google’s chief scientist, employee number thirty, the engineer whose name sits on MapReduce, Bigtable, Spanner, and the Google File System — the distributed-systems foundation that made internet-scale computation possible in the first place — is gone. He is co-founding Discovery Loop, a Delaware public-benefit corporation whose purpose is to automate the scientific method itself: AI systems that propose experiments, run thousands in parallel, learn, and iterate, starting with machine-learning research and expanding into hardware design, drug discovery, and clean energy.
He is taking a founding team of legends with him: Sanjay Ghemawat, a Google senior fellow and Dean’s collaborator of more than two decades; Oriol Vinyals, a DeepMind research VP and co-technical lead of Gemini; and Quoc Le, a Google Brain co-founder. Between them they wrote the sequence-to-sequence paper that established the encoder-decoder paradigm and the early work on pre-training large models. This is not a team Google can quietly backfill.
The cap table is the tell. Discovery Loop’s seed round is co-led by Radical Ventures and Khosla Ventures, with Kleiner Perkins, Lightspeed, Doerr Capital — and Alphabet — participating. Alphabet is a founding investor and the startup’s cloud provider for at least its first year. The venture thesis, in Vinod Khosla’s phrasing, is a shift from humans using AI to do research to AI as the researcher.
Read that arrangement slowly. Google is funding, and hosting, the company its own departing scientists are forming to pursue the exact mission — automated scientific discovery — that DeepMind itself vacated when it dissolved the AlphaFold team into Gemini earlier this year. Alphabet shares fell around four to five percent on the day.
This week did not come out of nowhere. It is the loudest moment in a year-long exodus. In a single week in June, Noam Shazeer — a co-lead of Gemini and one of the authors of the 2017 transformer paper that the entire industry is built on — left for OpenAI, and John Jumper, who shared the 2024 Nobel Prize in Chemistry for AlphaFold, left for Anthropic. A senior DeepMind researcher had already gone to a better-resourced startup; another long-tenured figure left in February. Of the eight authors of “Attention Is All You Need,” almost none remain at the company where they wrote it.
And all of this is happening to the company with the best hand in the industry. Google owns custom silicon, hyperscale cloud, frontier models, and the world’s most-used consumer surfaces — a complete, top-to-bottom AI stack that no rival can match. Yet over the past year it has struggled to ship a consistently leading frontier model, and OpenAI, Anthropic, and a rising cohort of Chinese labs have all fielded systems that beat Gemini on dimensions that matter, coding chief among them. There is a bookend worth noticing: in 2023, when Google merged Brain and DeepMind into a single lab under Hassabis, it moved Jeff Dean out of management and into a chief-scientist role, promising to back the combined team with “the computational resources of Google.” Three years later, Dean leaves entirely — and the computational resources are exactly what turned out to be rationed.
The consensus reading of all this is a morale-and-poaching story: rivals are paying more, the culture frayed, and stars drift toward the highest bidder. That reading is not wrong. It is just shallow. The deeper reading is that a talent exodus of this shape and severity is what an architecture decision looks like three years downstream — and to see it, you have to start not with the people, but with the chip.
