A system has “emergent” behavior if the system suddenly develops a significant new capability or character after a relatively small and gradual change in some of the system’s parts or features.
For instance, cold water, warm water, and hot water all mostly stay put inside their containers, but if you keep increasing water’s temperature past its boiling point, then all of a sudden water undergoes a phase change into gas, which can and will escape its container. Another example is the deadly “flashover” phenomenon in firefighting: when excessive heat builds up in a room, the entire space can burst into flames at once. Similarly, a few carpenter ants are harmless, but a colony of them can synergize to destroy your house.
AI systems can also display emergent behavior. Researchers find that incrementally increasing the parameters, training data, and computational resources available to an AI system may lead to the relatively rapid emergence of new capabilities, such as:‍
More generally, AI capabilities have consistently expanded over time. This advancement—which is significantly fueled by simply feeding more computational resources to AI systems—has led to the development of diverse new AI capabilities. For instance, OpenAI was excited in 2018 that GPT-1 could answer extremely basic questions in English, whereas today, significantly larger language models like GPT-4 can translate Chinese, write poetry, play chess, pass Turing tests, diagnose diseases, debug code, and explain jokes. The speed of this progress surprised many experts.
In short, it is not safe to assume that forthcoming AI systems will be just like current AI systems, only slightly more powerful. Instead, policymakers should assume that increasingly advanced AI systems will possess new kinds of abilities, including dangerous capabilities that are mostly absent in today’s best systems.