奥特曼认为本地 AI 推理会浪费全球稀缺的 DRAM,因为个人电脑中的内存大部分时间处于闲置状态,不如数据中心利用率高。
很多人想要本地推理,并想当然地认为有了开源模型问题就解决了。我们可以在自己的电脑上运行它们,就不再需要数据中心里的那些东西。但从全球资源配置的角度看,让每个人都把全球有限的 DRAM 供应放在自己的电脑里,而且只有 5% 的时间在使用,实际上非常糟糕。原文 (en)
A lot of people want local inference. And somehow, they assume that the problem is solved now because we are going to have open-source models. We’re going to be able to run them on our computers. And so we’re not going to need all the stuff in the data center. But from a global allocation perspective, actually, it’s quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer. And be used 5% of the time.