The sigmoids won't save you
In a post on Astral Codex Ten, Scott Alexander critiques the use of sigmoid curves in AI safety discourse. He contends that these functions, which model gradual S-shaped growth, give a false sense of predictability. Instead, he points to historical examples of technological progress that were sudden and unexpected, such as the development of nuclear weapons. Alexander warns that AI capabilities could similarly leap forward without warning, undermining safety measures that rely on smooth transitions. The article, published on May 15, 2026, draws on discussions from the AI safety community and references work by Eliezer Yudkowsky. Alexander emphasizes that the assumption of a gradual takeoff is not just wrong but dangerous, as it may lead to complacency. He calls for more robust safety research that accounts for discontinuous change.
Developers should not assume AI progress will be gradual; safety measures must account for sudden capability jumps.