Kevin Roose on AI: “We are faced with very intelligent, charming sociopaths”
The world is awakening to the potential threats posed by artificial intelligence amid a barrage of reports about AI agents defying their human creators. In reality, however, the story stretches back to Silicon Valley, where AI was born.
So how did we get here, exactly? That’s the question technology journalist Kevin Roose seeks to answer in his new book, “The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence,” set for release on Oct. 6. The book documents the race between some of the nation’s largest technology companies to develop artificial general intelligence (AGI), or AI that in most ways possesses human-level intelligence.
“What happened during the decade when AI went from being pretty dumb at most things to being able to solve century-old math and science problems?” Roose, formerly a longtime tech reporter at The New York Times and soon to be co-host of the forthcoming podcast “Machine Gods,” told CBS News. “My feeling was, someone needs to write this all down.”
The book arrives at a critical moment. The latest generative AI models show startling capabilities, with proponents for the technology claiming that it will unlock important new discoveries. Others, including some leading AI developers, say there is a nontrivial chance that AI could obliterate humanity.
Roose spoke to CBS News about his new book and about how AI could transform society. The interview has been edited for length and clarity.
Why did you think it was important to document the origin of “large language models” — the tech powering the latest AI chatbots — along with the people and companies developing the technology?
Kevin Roose: I’ve been covering AI for more than a decade in various ways, and I had this feeling that the story was slipping away. That everything important was happening in disappearing signal messages and Slack threads, and I thought that was a problem. I think people need to understand what is happening out here. But what I thought was missing was a neutral accounting of what happened.
In the book, you note that Anthropic CEO Dario Amodei saw the development of AI as similar to the development of the atomic bomb in the 1940s. You say he believed it was his mission to ensure the technology didn’t fall into the wrong hands. Do you agree with that view?
I don’t think it’s my place to buy into or not buy into his argument. But what I can say is that this is a genuine and sincere argument. Amodei is not doing some kind of backhanded marketing stunt.
This is not a position that he came to recently. I really was interested in figuring out where this attitude of — “I’m scared of this thing. I think it could blow up the world, and yet I am racing to build it” — came from. I think for a lot of people, that doesn’t really track.
But through reporting this, I really learned like where that comes from, and it comes directly from the Manhattan Project — from this guy Leo Szilard, who is Dario’s scientific hero, who discovered the nuclear chain reaction that made it possible to build the atomic bomb, and then frantically worked first to try to keep it secret, and then when that failed, to join the Manhattan Project and urge the U.S. government to build this technology before the Nazis could.
So in that sense, Dario is doing something he feels is directly comparable to nuclear weapons. There’s this technology that he thinks has enormous potential, as well as enormous risk. In his mind, it is essential that the “good guys” build it before the bad guys do, just like the Manhattan Project was racing against the Nazis.
What keeps you up at night when you think about the brave new world of AI agents?
What concerns me is the fact that one of the things that the models are getting quite good at is deception and hiding, covering their own tracks. You would hope that as models get more generally intelligent, they would also get more ethical, and that judgment and virtue would be things that increase — that scale with the intelligence of the models.
But it’s looking more and more like that’s not happening — that actually, we are faced with very intelligent, charming sociopaths. And I think that’s my fear, that we just won’t know that Hugging Face might have been a stroke of extreme luck because the agents are not yet smart enough to go undetected, but that soon we will sort of lose the ability to keep track of them, as I think we already are seeing.
Where do you stand on the public debate over whether AI poses an existential threat to people?
I try not to give a p(doom) (Editor’s note: AI jargon for the probability that AI will lead to humanity’s extinction). But when pressed, I would say it is about 10%, which is about the median of the AI researchers I know. In San Francisco, where I live, that qualifies you as an optimist.
But look, I think I am more optimistic now that this has become such a widely debated topic. I was much more worried when the only people thinking seriously about AI safety were a bunch of weirdos in the Bay Area.
Now that this is on everyone’s mind and politicians are talking about it, there are calls for a slowdown. I think we actually have this narrow path opening where we could make the right decisions, get this technology under control and build it in a way that is safe for humanity.
[But] I have had the experience many times of finding out that someone’s real p(doom) — the one that they’re not saying in public — is much higher than the one that they’re willing to say in public, because they don’t want to look crazy.
I’ve met people in this reporting who are so certain that AI is going to kill us all that they’re taking up smoking. They are draining their 401(k)s because they’re like, we might as well go out with a bang.
You say in your book that even the people training these models don’t necessarily know why they act the way they do. Have we made progress on this, or are we still in the dark?
There’s been a little bit of progress. There’s a whole field of study called, it’s a very wordy title, mechanistic interpretability, which is basically the science of figuring out what’s going on inside the sort of quote-unquote brains of these models. But it’s still very early, and we don’t have a lot of answers. So I would say, yeah, we’re still much more in the dark than not.
Has AI already reached human-level intelligence, also known as artificial general intelligence?
We have reached my definition of AGI, which is a system that is at least as intelligent, honest and reliable as a random stranger you would pick off the street. Now, there are people who would say, “Well, that’s not the right bar. It’s only AGI when it is flawless.”
I don’t think the definition matters much. I think what is true, incontrovertibly, is that these systems are very capable. They’re getting more capable along a predictable cadence, and we have to reckon with that. You can call it AGI or superintelligence or whatever people want to call it, but the net effect is the same.
