Ullal认为向分布式推理迁移需要更智能的网络,网络成为token效率的基石。
我们正目睹从高度集中的训练算力向分离式、分布式推理(预填充与解码)的迁移,这需要更智能的网络以及计算、存储和KV缓存资源的协同部署,而不只是关注原始速率和算力。设计良好的网络正成为最优token效率的基石。原文 (en)
We are witnessing the migration from highly centralized training compute to disaggregated and distributed inference, prefill and decode, demanding much more intelligent networking and colocation of compute, storage and KV cache resources rather than simply focusing on raw feeds and speeds and flops of compute. A well-designed network is becoming a foundational cornerstone for optimal token efficiency.