Dylan Patel notes that without native FP4, TPUv7 still trails FP4 GPUs on tokens per dollar until TPUv8 arrives in Q1 2027.
FP4 GPUs still deliver more tokens per dollar at most speeds, as TPUv7 has no native FP4 support, and we will need to wait until Q1 2027 for TPUv8 with FP4 support, as tracked in our Accelerator & HBM Model. At 100 tok/s/user, GB300 and GB200 FP4 deliver 3.6x and 3.5x the tokens per dollar of TPUv7.
All statements by Dylan Patel ›
Dylan Patel says China has gigawatt-scale AI campuses comparable to Abilene, and that their speed, scale, low cost and vertical integration are astonishing.
China has campuses on a similar scale, though few people outside the country have had a look inside. In September, we visited one, and we found the speed, scale, ultra low cost, and deep vertical integration astonishing.
Dylan Patel thinks vertical integration does not yet win hyperscaler orders in China, but could matter more as power gets scarcer and power architectures change.
It is too early to say whether vertical integration will become a meaningful advantage in China’s datacenter market. For now, it gives integrated players more control over cost and schedule, but hyperscalers still choose vendors mainly on resources and track record. That could change as power becomes harder to secure and the power system inside the datacenter is redesigned.
Dylan Patel argues energy access is becoming part of what datacenter developers sell, and power architecture is shifting to high-voltage DC with batteries replacing UPS.
Across different markets, access to energy is becoming part of what a datacenter developer sells. Second, the power architecture from the grid connection to the rack is being redesigned. In the US, the industry is moving toward higher-voltage DC power, while battery backup units (BBUs) and BESS are starting to take over the role of UPS.
Dylan Patel notes China's datacenter market is fiercely competitive, with rents halved, capex down 25%, and Inner Mongolia power among the world's cheapest.
China’s datacenter market is highly competitive. Rents have roughly halved over the past five years, to about $40/kW/month excluding power. Capex has fallen about 25% over the same period, to less than $3mm per MW. Power in Inner Mongolia costs about $0.055/kWh all-in, among the lowest in the world.
Dylan Patel says Google's TPUv7 now matches or beats NVIDIA on perf per dollar in agentic inference, which has pushed NVIDIA into an urgent engineering response.
TPUs delivering better performance per dollar or competitive performance per dollar is a new experience that NVIDIA senior leadership is unwilling to accept. As soon as TPU posted its first AgentX result on Qwen3.5 397B FP8 showing that even TPUv7 FP8 could beat GB300 NVL72 FP4, NVIDIA tasked a strike force of human engineers and agent swarms to work 997, late into the night until 2am local time and on weekends, across USA and mainland China.