Slower Scaling Gives Us Barely Enough Time To Invent Safe AI

How Scaling Works

The main reason why artificial intelligence systems keep getting more capable is that we keep pouring larger and larger amounts of resources into training them. 

This may seem like an obvious point – of course you can get a better product if you spend more money on it. What’s interesting about the cost of AI is that you can use relatively simple formulas to accurately estimate exactly how much performance you’ll get for a larger investment. The correlation has been so tight and reliable that technologists have described it as a “scaling law.” 

For example, in the graph below, the dotted orange line shows the predicted error rate made by an AI declining as it trains on more and more data, and the solid blue line shows the actual error rate created by that AI – for most of the graph, the predictions matched up with reality so closely that the lines overlap, literally.

Are Scaling Laws Breaking Down?

Over the last few months, anonymous AI researchers have been telling the press that these scaling laws are breaking down and that future progress on AI capabilities will be much slower than predicted.

The Center for AI Policy (CAIP) has two observations on these claims:

1. There’s no evidence that the reportedly under-performing AI models were given enough resources to scale up successfully. Big AI labs have not told us how much more data, semiconductors, and electricity was used to train the new models.
2. Even if AI progress slows down from the breakneck pace of 2023 and 2024, it will still move fast enough to disrupt American society, culture, and business radically.

For example, The Information quoted OpenAI researcher Noam Brown as saying that “more advanced models could become financially unfeasible to develop,” and other researchers at his company believe that OpenAI’s latest model, Orion, “isn’t reliably better than its predecessor in handling certain tasks…such as coding.” 

Even if true, neither of these claims shows that the scaling laws are breaking down. 

Scaling laws claim you'll perform better if you spend more money on an AI model. It is not clear whether OpenAI invested enough resources in Orion to expect significantly better performance from it. The fact that someday investors might tire of pouring ever larger amounts of cash into the AI ecosystem doesn’t mean that the line on the graph that relates more money to more performance has suddenly flattened out – it just means that for now, investors have chosen to pause at a particular point along that line.

Similarly, Ilya Suskever, an AI startup cofounder, told Reuters, “That results from scaling up pre-training…have plateaued,” forcing his company to explore alternative approaches and look carefully for “the right thing” to scale up instead of scaling everything up at once. However, Suskever did not provide any numbers to back up his claims or give details about how these alternative approaches work. 

Suskever’s claims are difficult to believe because his company is much smaller than Big AI developers like OpenAI and Anthropic. Suskever does not have the cash to compete directly with these AI giants over electricity and semiconductors, so it is in Suskever’s interest to claim that clever approaches matter more than the size of your bankroll. If Suskever has data showing that there has been a significant change in established scaling laws, then CAIP encourages Suskever to publish it.

Finally, Bloomberg News reports that Google’s “Gemini software is not living up to internal expectations” and that “Anthropic…has seen the timetable slip for the release of its long-awaited Claude model called 3.5 Opus.” 

Neither of these claims have enough information to evaluate what, if anything they imply about the future of scaling laws. 

Did Google invest 10x the data, electricity, and compute in its new Gemini software (compared to the previous version) and then stumble into the unpleasant surprise that these investments failed to pay off? Or did Google skimp on resources for its latest AI model and then reap the predictable consequences of its frugality? So far, we simply don’t know.

Even a Slower Scaling Law Still Means a Rapidly Changing Future

Let’s say the growing tide of unease in Silicon Valley is completely accurate, though, and we can expect slower progress on AI over the next several years. What would that mean for our future?

The most likely answer is that humanity will develop superintelligent machines sometime in the mid-2030s instead of in 2027 or 2028. 

As Leopold Aschenbrenner, formerly of the superalignment team at OpenAI and now an AGI investor, points out in Situational Awareness, there’s no reason to expect that the leap from GPT-2 level models (which sometimes managed “to string together a few coherent sentences,” to GPT-4 level models (which “ace high-school exams”) was a one-time miracle. Instead, it’s just the logical and predictable consequence of following the scaling laws along the same general line they’ve been following for at least a decade. 

If you scale up compute, data, and electricity by 10,000 times, you should expect to get a much more powerful AI model. That’s how we got GPT-4, and we are still on track to scale up by another 10,000 times, resulting in a model that’s 100 million times the size of GPT-2 and, therefore, much better at solving problems and mimicking human behavior. 

We should expect such a model to be capable of successfully imitating the behavior of top-notch hardware engineers and machine learning researchers, meaning that these models will be able to accelerate their further development rapidly. 

Once you’ve got a thousand genius-level “virtual engineers” running in a data center and thinking up ways to make themselves even more intelligent, the pace of growth will accelerate even more rapidly.

Of course, in the real world, there are always delays. A hurricane knocks out a quartz mine needed for semiconductor manufacturing, or a hardware failure means that a months-long training run has to be restarted from scratch. A backlog at NVIDIA means that specialized AI hardware isn’t available for purchase (at any price) until a year or two after AI developers place their orders.

We can reasonably expect that some speed bumps will continue to pop up occasionally and somewhat delay the advent of superintelligence – but we are talking about delay, not cancellation. If we don’t invent AI capable of doing genius-level machine learning research until 2035, that still means that a superintelligence explosion is coming very soon, in our lifetimes. 

This means that unless America’s political leaders intensify their efforts to accelerate safety research and require that AI be designed to a more reliable standard, we will not be ready when superintelligence arrives. 

We’re Not Ready for Extremely Advanced AI

America is undoubtedly ill-prepared for superintelligent AI. 

This week, CBS News reported that Google’s supposedly well-trained Gemini chatbot had told one of its users to die. "You are not special, you are not important, and you are not needed,” wrote Gemini. “You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe.” 

"Please die," the AI added. "Please."

Does that sound like an AI that can be trusted with the future of humanity?

As former Google CEO Eric Schmidt and longtime Microsoft senior executive Craig Mundie warn in their last book, co-authored with the late Henry Kissinger, “Training an AI to understand us and then sitting back and hoping that it respects us is not a strategy that seems either safe or likely to succeed.”

Even if scaling laws slow down, we will still have only a few short years to find a better answer. The Center for AI Policy urges Congress not to waste that time. We need to pass commonsense AI safety legislation now, so that we will be prepared when superintelligence arrives.