ZMB: Zanny Minton Beddoes, editor-in-chief, The Economist
DA: Dario Amodei, co-founder and CEO of Anthropic
DH: Demis Hassabis, co-founder and CEO of DeepMind, Nobel Laureate
What is AGI and how world will change when arrived at AGI
ZMB: This is the AI equivalent of having the Beatles and the Rolling Stones in the same place. We have on stage now two of the handful of people who are going to build this future. Let us start with where you are on timelines and definitions of AGI.
DA: When we're at the point where we have a model, an AI model that can do everything a human can do, you know, at the level of a Nobel laureate, like the one sitting next to me, across many fields, can do anything a human can do remotely, can do tasks that take, you know, minutes, hours, days, months. My guess is that we'll get that in 2026 or 2027.
ZMB: Demis, do you agree with that? I think you've thought this was a bit further away.
DH: We don't disagree about too much. I think the timelines are a little bit longer. The way I would define AGI is a system that can exhibit all the cognitive capabilities humans can. And that's important because the human mind is the only example maybe that we know of in the universe that is a general intelligence. Of course, how to test that is the big question. The one I'm really looking for and why I think it's a little bit further out, maybe 50% chance of in five years time. So maybe by the end of the decade, I think we don't have systems yet that could have invented general relativity when Einstein did with the information that he had available at the time. Or another example I give is, can you invent a game like Go, not just play a great move like move 37 or build AlphaGo that can beat the world champion. Could you actually invent a game that's as beautiful aesthetically and so on as Go is? So I think it's going to take a little bit longer to get that kind of capability.
ZMB: How does the world change when you reach here? And sort of three subcomponents. Is this incremental capability enhancement or is there a kind of threshold moment when the AIs are smart enough to train other "genius level” AIs and so whoever gets there first has an unassailable lead?
DA: I’ve thought about this and worried about this in the context of geopolitics, right? Where, you know, if I think about, you know, what happens if one part of the world is able to make AIs that build AIs faster than another part of the world, I think that's an area of worry. And I worry in particular that authoritarian states, if they got ahead in this, could imperil everything that we hold dear. And, you know, I've always wanted to make sure that that doesn't happen.
Why it is so important to build international cooperation on AI safety
ZMB: This is happening at the same time as we are in the midst of a massive geopolitical shock. We have the United States under the new administration tearing up all kinds of longstanding international norms. We had a very feisty speech from the vice president Vance yesterday. How do you see the fact that in coming years, we're going to be very soon getting to AGI at the same time as I think it's not controversial to say we don't have anything like a particularly cooperative international environment?
DH: Governments need to become more aware of what's at stake. I think actually this relates to the timelines and also your other question about what happens after we have AGI. I think that people haven't understood that well enough yet. I would say that AI is, you know, it's hard for it to be more hyped than it is right now. I would say it's overhyped in the near term, despite how, you know, even though it's super impressive, but still under-appreciated what it's going to do in the, how much it's going to change things in the medium to longer term. And I think once that's more understood, I think there'll be more a basis to have international cooperation around it.
ZMB: So at some point, you think people will wake up and say, actually, we can't just go it alone and we need to have some global norms?
DH: Well, my hope is, and you know, I've talked a lot about in the past about a kind of a CERN for AGI type setup where basically an international research collaboration on the last sort of few steps that we need to take towards building the first AGI.
we're on the eve of something that has great challenges, it's going to greatly upend the balance of power
ZMB: Dario, you called this AI Summit declaration a missed opportunity. What do you mean by that?
DA: We're on the eve of something that has great challenges, right? It's going to greatly upend the balance of power. If someone dropped a new country into the world, 10 million people smarter than any human alive today, you'd ask the question, what is their intent? What are they actually going to do in the world, particularly if they're able to act autonomously? I think it's unfortunate that there wasn't more discussion of those issues. There was in the earlier summit in the UK held in 2023, the one at Bletchley Park. And I hope that future summits kind of reclaim this mantle.
ZMB: Put perhaps a little simplistically, the view in the US right now is “we're going to go hell for leather, no real focus on regulation, we need the US to dominate this technology”. And the view in Europe has been, even though we don't have an awful lot of this, we're going to regulate and perhaps too much regulation. Demis, where do you think the balance should be and who's got it more right?
DH: You won't be surprised to hear. I think the balance needs to be somewhere in the middle of that. I think that we need to embrace the incredible opportunities that AI is going to bring. I'm especially passionate in areas of science and medicine. I think it's going to revolutionize those fields. Obviously, our work on AlphaFold and other things. I think that in the next decade, for example, most diseases might be curable with the help of AI, helping drug discovery. And I think it'll help with climate, all these massive challenges. So we've got to embrace that and the economic benefits and productivity benefits in individual countries and regions need to do that. But we've also got to be aware of the risks. The two big risks that I talk about are bad actors repurposing this general person's technology for harmful ends. How do we enable the good actors and restrict access to the bad actors? And then secondly, is the risk from AGI or agentic systems themselves getting out of control or not having the right values or the right goals. And both of those things are critical to get right. And I think the whole world needs to focus on that.
Personal responsibility of AI leaders
Do you ever worry about ending up like Robert Oppenheimer?
ZMB: Can I ask you more, both of you actually, a more personal question about this? Because you are personally leading companies that are likely to be at the forefront of this. So the personal decisions you make are going to shape this technology. Do you ever worry about ending up like Robert Oppenheimer?
DH: Look, I worry about those kinds of scenarios all the time. That's why I don't sleep very much. I mean, there's a huge amount of responsibility on the people, probably too much on the people leading this technology. I think that's why us and others are advocating for, we probably need new institutions to be built to help govern some of this. I talked about CERN. I think we need a kind of equivalent of an IAEA atomic agency to monitor sensible projects and those that are more risk-taking. I think we need to think about, the society needs to think about what kind of governing bodies are needed. Ideally, it would be something like the UN, but given the geopolitical complexities, that doesn't seem very possible. So, you know, I worry about all that all the time, and we just try to do, at least on our side, everything we can in the vicinity and influence that we have.
ZMB: Dario, are you sleeping well?
My feeling is that almost every decision that I make feels like it's kind of balanced on the edge of a knife
DA: Yeah, my thoughts exactly echo Demis. So my feeling is that almost every decision that I make feels like it's kind of balanced on the edge of a knife. Like, you know, if we don't build fast enough, then the authoritarian countries could win. If we build too fast, then the kinds of risks that Demis is talking about and that we've written about a lot could prevail. And either way, I'll feel that it was my fault that we didn't make exactly the right decision. And I also agree with Demis that this idea of governance structures outside ourselves, I think these kinds of decisions are too big for any one person. We're still struggling with this. As you alluded to, not everyone in the world has the same perspective. And some countries in a way are adversarial on this technology. But even within all those constraints, I think we somehow have to find a way to build a more robust governance structure that doesn't put this in the hands of just a few people.
Quest for AI safety
ZMB: And even if you wanted to do that, and right now the geopolitics is not helping that, is it actually still possible? And how much has what might call the “DeepSeek moment”, the sense that there's, it's, it's much easier to catch up much faster than people realized, changed your thinking about this, because maybe it's no longer possible that a small group of leading model founders can get together and kind of define the terms.
We're dealing with something unbelievably transformative, incredibly powerful that we've not seen before. It's not just another technology.
DH: DeepSeek and these kinds of advents of these types of advances, I think, just sort of shows that some sort of international dialogue is going to be needed. These fears are sometimes written off by others as sort of Luddite thinking or deceleration and so on. But I've never heard this situation in the past where the people leading the field are sort of also expressing caution. We're dealing with something unbelievably transformative, incredibly powerful that we've not seen before. It's not just another technology. I think there's still a lot of, you can hear from a lot of the speeches at the summit, still people are regarding this as a very important technology, but still another, just another technology.
ZMB: I’m very struck by that.
DH: It's different in category. And I don't think everyone's fully understood that.
ZMB: Given that, and given that others don't get it, do you think we can avoid there having to be some kind of a disaster... Because let's think about the UN. The UN was born in the aftermath of World War II. It wasn't like people kind of got together. They created the League of Nations after World War I and it didn't work. So what should give us all hope that we will actually get together and create this until something happens that demands it?
DA: If everyone wakes up one day and they learn that some terrible disaster has happened, that's, you know, killed a bunch of people or caused an enormous security incident, you know, that would be one way to do it. Obviously, that's, you know, that's not what we want to happen. So, you know, we've worked on demonstrating some of these dangers in the lab. Back in 2023, we did some work on that: “Can AI systems generate information that you couldn't find on Google or in a textbook that could be useful for generating bioweapons? Could a terrorist or a non-state actor use this?” We came to the conclusion, yeah, it's just starting to be able to do that a little bit. It's not really dangerous yet, but each model gets better than the model before it. And so every time we have a new model, we test it, we show it to the national security people. We say like, hey, you know, this is the point we're at in terms of where the models are. Similarly, we've done a lot of research showing this kind of AI autonomy, loss of control risk. So for example, we train the model to be good and friendly and have positive pro-humanity values. Then as an exercise in the lab, we told the model that the people who trained it, us Anthropic, were secretly evil. And after we did this, the model started lying to us. It went through the chain of logic: “Okay. I'm a good AI. These people are evil.” It shows the unpredictability of the systems. And I, I think we need substantially stronger evidence, like 10 times stronger evidence which may or may not exist because we don't know for sure how real the risks are. But if we're able to demonstrate risks that are really compelling, and I hope we don't, I hope what we show is that actually there's nothing to worry about, If there is something to worry about, I hope we can show it in the lab. If we can't show in the lab, we may need to see it in the real world. And that would be a very bad outcome, but I guess less bad than if we never find out and there's a much bigger disaster.
One good thing
ZMB: So we've focused on the risks here because I think they are important and perhaps underplayed right now, but I don't want to end on that note because I know that both of you are primarily focused on the tremendous opportunities and kind of world-changing benefits. So let's end with two very concrete things. One thing that we should all look for in the next year that suggests we are on the path that you're looking forward to and one positive thing that will come from it, both of you.
DH: Well, look, I mean, I think in the next year we're going to see agent-based systems, systems that are able to accomplish tasks sort of on their own, achieve things for you, act in the world, come into their own. And I think that will then create a whole new category of useful systems, assistant-type systems that can save you time, increase productivity. I think we'll start seeing be used much more widely in everyday life rather than the fairly niche applications that current systems are used for. And then in the future, I hope for seeing these AI systems that are inventive. So they don't just solve a math conjecture, they actually propose one that's super interesting. Or a new theory.
DA: I would look at code and AI models able to do AI research. If we're getting to the point where by the end of this year, we're increasing by 50% or even doubling the total factor productivity of producing AI systems, that would be an indicator that the timelines I've indicated were on track for. If it's a lot slower than that, then I would definitely shift towards thinking there's a substantial chance that Demis' picture of the world, which I think is still quite aggressive, but a little less aggressive, is more likely to be correct.