Commentary: Altman’s Opaque AI Is creating a new security dilemma

Back in 2017, Jacob Andreas, a then PhD student at the University of California, Berkeley, and his co-authors looked at how two AI agents could communicate using their own numerical representations. The scientists called this artificial vernacular “neuralese” and developed a tool for translating it into natural language.

Unfortunately, no amount of time spent on Duolingo will help a person learn neuralese. The hidden numerical representations AI systems use can look something like this: [-0.1565, -0.1862, 0.0528, …, -0.1188, 0.0662, 0.5470].

Compare that with a human-readable chain of thought when someone asks ChatGPT to write a polite email, which might look like this: “The user wants a polite email. I should keep the tone friendly and professional…” The more reasoning that happens in these hidden representations, the harder it can be for researchers to follow.

Altman took to X on Wednesday, the day after news broke about Astra’s use of recurrent depth, to reassure that OpenAI was “sprinting on safety priorities” and striking a balance between creating AI that’s powerful and “benefits people” while remaining safe. OpenAI has reportedly limited Astra’s use of recurrent depth so its researchers can still monitor its chain-of-thought, too.

Yet Altman seems to be straying from the path of surety to help smooth OpenAI’s route toward a US$1 trillion initial public offering. After all, better AI systems lead to increased revenue and more cloud usage.