I missed
this story from a few weeks ago about the OECD’s efforts (in collaboration with, among others, Jack Clark, whose
Import AI newsletter is a must read) to create an index of the computational power (“compute”) available to different nation states. The goal is to help policymakers make investment decisions relating to their national AI strategies - which at least 80 countries now have, according to the article.
The strategic significance of compute, which we discussed in
TiB 128, remains underrated outside tech circles.
Its importance rests on an idea known as the “scaling hypothesis”: this holds that what’s required to achieve
artificial general intelligence is not a major theoretical breakthrough, but rather scaling our current machine learning models by orders of magnitude. Some recent AI achievements (e.g.
GPT-3) provide some evidence for this. And one key to scale in AI is much, much more compute. Gwern has the
definitive write up of the scaling hypothesis, which I highly recommend.
The upshot is that in a world of
AI nationalism, compute is a key strategic resource (see also my
podcast conversation with Ian Hogarth). We’re currently in a strange transition period where machine learning models are expensive to train relative to the budget of a startup (e.g. DeepMind’s AlphaStar model reportedly cost $26m to train), but small relative to those of nation states. But in a world where a single model
could soon cost a billion dollars to train, policymakers will have to think carefully about how to build or access the resources they need to compete.
