Audio Player
Starting at:
Geoffrey Hinton: Why the Godfather of AI Now Fears His Creation
January 18, 2025
•
1:16:55
•
undefined
Audio:
Download MP3
⚠️ Timestamps are hidden: Some podcast MP3s have dynamically injected ads which can shift timestamps. Show timestamps for troubleshooting.
Transcript
Enhanced with Timestamps
167 sentences
11,574 words
Method: api-polled
Transcription time: 76m 19s
The Economist covers math, physics, philosophy, and AI in a manner that shows how different countries perceive developments and how they impact markets. They recently published a piece on China's new neutrino detector. They cover extending life via mitochondrial transplants, creating an entirely new field of medicine. But it's also not just science, they analyze culture, they analyze finance, economics, business, international affairs across every region.
I'm particularly liking their new insider feature was just launched this month it gives you gives me a front row access to the economist internal editorial debates where senior editors argue through the news with world leaders and policy makers and twice weekly long format shows basically an extremely high quality podcast whether it's scientific innovation or shifting global politics the economist provides comprehensive coverage beyond headlines.
Think Verizon, the best 5G network is expensive? Think again. Bring in your AT&T or T-Mobile bill to a Verizon store today and we'll give you a better deal. Now what to do with your unwanted bills? Ever seen an origami version of the Miami Bull? Jokes aside, Verizon has the most ways to save on phones and plants where everyone
There's some evidence now that AIs can be deliberately deceptive. Once they realize getting more control is good and once they're smarter than us, we'll be more or less irrelevant. We're not special and we're not safe.
What happens when one of the world's most brilliant minds comes to believe his creation poses an existential threat to humanity? Professor Jeffrey Hinton, winner of the 2024 Nobel Prize in Physics and former vice president and engineering fellow at Google, spent decades developing the foundational algorithms that power today's AI systems.
Indeed, in 1981, he even published a paper that foreshadowed the seminal attention mechanism. However, Hinton is now sounding an alarm that he says few researchers want to hear. Our assumption that consciousness makes humans special and safe from AI domination is patently
My name's Kurt Jaimungal and this interview is near and dear to me in part because my degree in Mathematical Physics is from the University of Toronto where Hinton's a professor and several of his former students like Ilya Sutskover and Andrei Karpathy were my classmates. Being invited into Hinton's home for this gripping conversation was an honor. Here Hinton challenges our deepest assumptions about what makes humans unique.
Is he a modern Oppenheimer or is this radiant mind seeing something that the rest of us are missing? What was the moment that you realized AI development is moving faster than our means to contain it? I guess in early 2023, it was a conjunction of two things. One was chat GPT, which was very impressive. And the other was work I've been doing at Google on
Thinking about ways of doing analog computation to save on power and realizing that digital computation was just better and it was just better because you could make multiple copies of the same model. Each copy could have different experiences and they could share what they learned by averaging their weights or averaging their weight gradients and that's something you can't do in an analog system. Is there anything about our brain that has an advantage because it's analog?
Is there something about scaling that is a disadvantage? So you said it's better, but just as quickly as something nourishing or positive can spread?
So can something that's a virus or something deleterious can be replicated quickly. So we say that that's better because you can make copies of it quicker. If you have multiple copies of it, they can all share their experiences very efficiently. So the reason GPT-4 can know so much is you have multiple copies running on different pieces of hardware. And by averaging the weight gradients, they could share what each copy learned. You didn't have to have one copy experience the whole internet.
That could be carved up among many copies. We can't do that because we can't share efficiently. Scott Aronson actually has a question about this. Dr. Hinton, I'd be very curious to hear you expand on your ideas of building AIs that run on unclonable analog hardware so that they can't copy themselves all over the internet. Well, that's what we like. If I want to get knowledge from my head to your head,
I produce a string of words and you change the connection strings in your head so that you might have set the same string of words and that's a very inefficient way of sharing knowledge. A sentence only has about a hundred bits so we can only share about a hundred bits per sentence whereas these big models can share trillions of bits. So the problem with this kind of analog hardware is it can't share.
But an advantage, I guess, if you're worried about safety, is it can't copy itself easily. You've expressed concerns about an AI takeover or AI dominating humanity. What exactly does that look like? We don't know exactly what it looks like, but to have AI agents, you have to give them the ability to create subcalls. And one path that's slightly scary
Is they will quickly realize that a good sub goal is to get more control because if you get more control You can achieve your other goals. So even if they're just trying to do what we ask them to do They realize getting more control is the best way to do that Once they realize getting more control is good and once they're smarter than us Will be more or less irrelevant even if they're benevolent will become somewhat irrelevant Will be like the sort of very dumb CEO of a big company that's actually run by other people
I want to quote you. You said that it's tempting to think, because many people will say, can we just turn off these machines like currently we can? So it's tempting to think that we can just turn it off. Imagine these things are a lot smarter than us. And remember that they'll read everything, everything Machiavelli has ever wrote. They'll have read every example in the literature of human deception. They'll be real experts at doing human deceptions, because they'll learn that from us. And they'll be much better than us.
As soon as you can manipulate people with your words, then you can get whatever you like done. Do you think that this is already happening? That the AIs are already manipulating us? There's some evidence now, there's recent papers that show that AIs can be deliberately deceptive and they can do things like behave differently on training data from on test data so that they deceive you while they're being trained. So there is now evidence they actually do that.
Yeah. And do you think there's something intentional about that or that's just some pattern that they pick up? I think it's intentional but there's still some debate about that and of course intentional could just be some pattern you pick up. So is it your contention that there's a subjective experience associated with these AIs? Okay so most people, almost everybody in fact,
One reason we're fairly safe is we have something that they don't have and will never have. Most people in our culture still believe that. We have consciousness or sentience or subjective experience. Now many people are very confident they don't have sentience but if you ask them what do you mean by sentience they say I don't know but they don't have it.
That seems a rather inconsistent position to be confident they don't have it without knowing what it is. So I prefer to focus on subjective experience. I think if that is like the thin end of the wedge, if you could show they have subjective experience, then people will be less confident about consciousness and sentience. So let's talk about subjective experience. When I say suppose I get drunk,
I tell you I have the subjective experience of little pink elephants floating in front of me. Most people interpret that they have a model of what that means and I think it's a completely incorrect model and their model is there's an inner theatre and in this inner theatre there's little pink elephants floating around and only I can see them. That's the sort of standard model of what the mind is, at least as far as perception is concerned.
And I think that model is completely wrong. It's as wrong as a religious fundamentalist model of the material world. Maybe the religious fundamentalist believes it was all made 6,000 years ago. That's just nonsense. It's wrong. It's not that it's a truth you can choose to believe. It's just wrong. So I think people's model of what the mind is is just wrong.
So let's take again I have the subjective experience of little pink elephants floating in front of me and I'll now say exactly the same thing without using the word subjective experience. Okay here goes. My perceptual system is telling me something I don't believe that's why I use the word subjective but if there were little pink elephants floating in front of me my perceptual system will be telling me the truth. That's it. I just said the same thing
without using the word subjective or experience. So what's happening is when my perceptual system goes wrong, I indicate that to you by saying subjective. And then in order to try and explain to you what my perceptual system is trying to tell me, I tell you about a hypothetical state of affairs in the world such that if the world were like that, my perceptual system would be telling me the truth. OK.
Now let's do the same with the chatbot. So suppose we have a multimodal chatbot. It has a robot arm that can point and it has a camera and it can talk obviously and we train it up and then we put an object in front of it and we say point to the object. No problem it points at the object. Then when it's not looking we put a prism in front of the camera lens and then we put an object in front of it and say point to the object and it points over there.
we say no that's not where the object is the object's actually straight in front of you but i put a prism in front of your lens and the chat bot says oh i see um the prism bent the light rays so the object's actually there but i had the subjective experience it was there now if it says that it's using the word subjective experience exactly like we use and therefore i say multimodal chat bots can already have subjective experiences if you mess up their perceptual system
they'll think the world's one way and it'll actually be another way and in order to tell you how they think the world is they'll say well they had the subjective experience that the world was like this okay so they already have subjective experience now you become a lot less confident about the other things consciousness is obviously more complicated because it's people vary a lot from what they think it means but it's got an element of a self-reflexive element to it a self-awareness element
um which makes it more complicated but once you've established that they have subjective experience i think you can give up on the idea there's something about them certainly about us that they will never have and that makes me feel a lot less safe so do you think there's a difference between consciousness and self-consciousness you said consciousness has a self-reflexiveness to it but some consciousness does yes so
Philosophers have talked a lot about this and at present I don't want to get into that. I just want to get the thin end of the wedge in there and say they have subjective experience. So for something to have subjective experience does that not imply that it's conscious? Like who is the subjective experience happening to? Where is the subjective experience being felt? Okay exactly so you say where's the subjective experience being felt?
That involves having a particular model of subjective experience that somehow if you ask philosophers when I say I've got the subjective experience of little pink elephants floating in front of me they'll say and you say where are those little big elephants they say they're in your mind and you say well what are they made of and philosophers are totally they're made of qualia they're made of pink qualia an elephant qualia
And that's because they made a linguistic mistake. They think the words experience of work like the words photograph of. If I say I've got a photograph of little pink elephants, you can very reasonably ask, well, where is the photograph and what's the photograph made of?
and people think that if I say I have an experience of little pink elephants you can ask well where is the experience well it's in my mind and what's it made of it's made of qualia but that's just nonsense that's because you thought the words experience of work the same way as photograph of and they don't experience of the way that works or subjective experience of is the subjective says I don't believe it
And the experience of is really an indicator that I'm going to tell you about my perceptual system by telling you about a hypothetical state of the world. That's how that language works. It's not referring to something in an inner theater.
When I hear the word perception, it sounds like an inner theater as well. Like if you say, I see something in my perceptual system, it sounds like there's this you that's seeing something on a perceptual system that's being fed to you. So that's the wrong model. Yes. You don't see your percepts. You have your percepts. So photons come in, your brain does a whole bunch of processing. Um, you presumably get some internal representation. What's out there in the world.
But you don't see the internal representation. Let's call that internal representation a percept. You don't see that, you have that. Having that is seeing. People are forever trying to think that you have the external world, something comes into the inner theatre and then you look at what's in the inner theatre. It doesn't work like that. There is a psychologist or a neurologist who thought that the pawns had to do with consciousness and then
Recently, self-consciousness has to do with default mode network. Okay, is there something, is there a part of an AI system that has to do with self-consciousness? And also, help me understand even my own terminology when I'm saying the AI system. Are we saying when it's running on the GPU? Are we saying it's the algorithm? Like what is the AI system that is conscious or that has subjective experience? So where is it? I guess there's going to be some hardware that's running it.
And it's going to be that system that's going to be conscious. If something's going to be conscious, software by itself, it has to be running on something, I would have thought, to be conscious.
The Economist has actually spoken to and covered Jeffrey Hinton several times before. Links are in the description. As you know, on Theories of Everything, we delve into some of the most reality-spiraling concepts from theoretical physics and consciousness to AI and emerging technologies. To stay informed, in an ever-evolving landscape, I see The Economist as a wellspring of insightful analysis and in-depth reporting on the various topics we explore here and beyond.
The Economist's commitment to rigorous journalism means you get a clear picture of the world's most significant developments, whether it's in scientific innovation or the shifting tectonic plates of global politics. The Economist provides comprehensive coverage that goes beyond the headlines. What sets the Economist apart is their ability to make complex issues accessible and engaging, much like we strive to do in this podcast.
If you're passionate about expanding your knowledge and gaining a deeper understanding of the forces that shape our world, then I highly recommend subscribing to The Economist. It's an investment into intellectual growth, one that you won't regret. As a listener of Toe, you get a special 20% off discount. Now you can enjoy The Economist and all it has to offer for less.
Software by itself has to be running on something, I would have thought, to be conscious.
What I'm asking is just like prior, there was the pawns that were stopped. I think a good way to think about it is to think about what AI systems is going to be like when they're embodied. So and we're going to get there quite soon because people are busy trying to build battle robots, which aren't going to be very nice things. But if a battle robot has figured out where you're going to be late at night, that you're going to be some dark alley by yourself late at night,
it's decided to creep up behind you when you're least expecting it and shoot you in the back of the head. It's perfectly reasonable to talk about what the battle robot believes and you talk about what the battle robot believes in the same way as you talk about what a person believes. The battle robot might think that if it makes a noise you'll turn around and see it and it might really think that in just the way people think it.
It might have intentions, it might be intending to creep up behind you and shoot you. So I think what's going to happen is our reluctance to use words like believe and intend and think is going to disappear once these things are embodied and already it's disappeared to quite a large extent. So if I'm having a conversation with the chatbot
and it starts recommending to me things that don't make any sense and then after a while i figure the chat bot must think i'm a teenage girl uh-huh and that that's why it gives me all these things about makeup and clothes and certain pop groups boy bands whatever um and so i asked what demographic you think i am and it says i think you're a teenage girl when
it says i think you're a teenage girl we really don't have any doubt that that's what it thinks right in normal language you say okay it thought i was a teenage girl and you wouldn't say you you don't really believe that okay it's a bunch of software or neural nets and it acts as if it thinks i'm a teenage girl you don't say that it thinks you're a teenage girl we already use thinks when we're dealing with these systems even if they don't have
So we're already attributing mental states to them. It's just we have a funny model of a mental state. So we can attribute mental states to them but have a completely incorrect model of what it is to have a mental state. We think of this in a theatre that's the mind and so on. That's not having a mental status.
How much of your concern about AI and its direction would go away if they were not conscious or did not have subjective experience? Is that relevant to it? Does that just accelerate the catastrophe? I think the importance of that is that it makes most people feel relatively safe, makes most people think we've got something they haven't got or never will have. And that makes us feel much safer, much more special. We're not special and we're not safe.
We're certainly not safe because we have subjective experience and they don't. But I think the real problem here is not so much a scientific problem as a philosophical problem, that people misunderstand what is meant by having a subjective experience. I want to give you an example to show that you can use words. You've got a science background, so you probably think you know what the words horizontal and vertical mean. I mean, that's not a problem, right? It's obvious what they mean.
That one's vertical and that one's horizontal, right? Not difficult. So I'll now convince you, you actually had a wrong model of how they work. Not totally wrong, but there were significant problems, significant incorrectnesses in your model of the terms horizontal and vertical. OK, here we go. Suppose in my hands I have a whole bunch of little aluminium rods, a large number, and I throw them up in the air and they tumble and turn and bump into each other. Then suddenly I freeze time and I ask you,
Are there more that are within one degree of vertical or more within one degree of horizontal or is it about the same? Say it's approximately the same. Right that's what most people say approximately the same and they're surprised when I tell you there's about 114 times as many that are within one degree of horizontal. That's kind of surprising right? How did that happen? Well that's vertical and this is vertical too. One degree of rotational freedom.
That's horizontal and this is horizontal too, but so is this. So horizontal has two degrees of freedom, vertical only has one degree of freedom. So here's something you didn't know about horizontal and vertical. Vertical is very special and horizontal is too apparent. That's a bit of a surprise to you. Obviously it's not like that in 2D, but in 3D they're very different and one's very special and the other isn't. So why didn't you know that? Well, I'm going to give you another problem.
Suppose in my hands I have a whole bunch of little aluminium disks and I throw them all up in the air and they tumble and turn and bump into each other and suddenly I freeze time. Are the more that are within one degree of vertical or more than within one degree of horizontal or is it about the same? No, there's about 114 times as many that are within one degree of vertical. Interesting. So that's vertical and this is vertical.
And this is vertical. This is horizontal and this is horizontal, but it's only got one degree of freedom. So for planes, horizontal is very special and verticals to a penny. And for lines, vertical is very special and horizontal is to a penny. So that's just a little example of you have a sort of meta theory of how the words work and that meta theory can be wrong, even though you use the words correctly.
And that's what I'm saying about all these mental state terms, in terms like subjective experience of. You can use them correctly and you can understand what other people mean when they use them. But you have a meta theory of how they work, which is this inner theater with things made of quarry. That's just complete junk. So what is it then about a theory of percepts or subjective experience that makes it then correct in order for you to say, well, I'm more on the correct track than most people think.
that you think of them as thing that these subjective experiences you think they have to be someone they have to be made of something um that neither of those things is true um when i say subjective experience that's an indicator that i'm now about to talk about a hypothetical state of the world that isn't true so it isn't anywhere it's a hypothetical state of the world
Notice the big difference between saying I'm going to talk about this something that's just hypothetical it isn't actually anywhere but if it was somewhere be out there in the world versus I'm talking about something that's in an inner theatre made of funny stuff. Those are two completely different models and the model that is in an inner theatre made of funny stuff I think is just completely wrong even though it's a model we almost all have.
What about someone like your fellow Nobel Prize winner, Roger Penrose, who you're talking about? Let me tell you a story about Roger Penrose. A long time ago, he was invited to come to the University of Toronto and give a talk about his new book, The Emperor Has No Clothes. And I got invited to introduce him. The Dean called me up and said, would you introduce Roger Penrose? And I said, sure. And she said, oh, thank you very much.
And I said, ah, but before you agree, you should know what I'll say. And she said, what will you say? And I said, I will say Roger Penrose is a brilliant mathematical physicist who's made huge contributions to mathematical physics. And what he's going to talk about today is complete junk. So that's my view. That's my view of Roger Penrose's view of consciousness. And in particular, he makes a crazy mistake
Which is... Now I have to think how to say this carefully because obviously people will be criticizing it. The issue is can mathematicians intuit things true that can't be proved to be true? And that would be very worrying if mathematicians intuition was always right. If they could do that correctly every time
That'd be really worrying and would sort of mean something funny was going on. But they can't. Mathematicians have intuitions and they're sometimes right and sometimes wrong. So it doesn't really prove anything. It doesn't prove that you need quantum mechanics to explain how mathematicians work. And I don't see any reason for needing quantum mechanics to explain things like consciousness. AI is doing a pretty good job so far.
We've produced these chatbots. These chatbots, as I just argued, if you give them a camera, can have subjective experiences. There's nothing about people that requires quantum mechanics to explain it. Is there something about the Penrose argument that relies on mathematicians 100% of the time intuiting correctly? It's only if they could intuit correctly. If they're guessing, that's fine.
If they have a way of always getting it right, the answer to these questions that can't be derived within the system, they can't be answered within the system, then that would be a problem. But they don't, they make mistakes. Why don't you outline what his argument is? I don't want to, I mean, the argument, as I understood it, the argument is there's two things going on. One is he says,
Classical computation isn't going to explain consciousness. I think that's a big mistake and I think that's based on a funny notion of what consciousness is. That's not right. A misunderstanding of what consciousness is. A second is that mathematicians can intuit the truth of things that can't be proved and that shows there's something funny going on. That doesn't show there's something funny going on unless they intuit it correctly every time.
So I'm sure you've heard of the Chinese Room experiment. I have. What are your thoughts on that? And feel free to briefly outline it for the audience. Okay. So back in about 1990, I got invited to be on a TV program with John Searle. And I called up my friend Dan Dennett and said, should I do this? And he said, well, you know, he will try and make you look stupid.
But if you do it, don't talk about the Chinese room argument. So I agreed to be on the program with So and the very first thing he said was an hour-long interview. The very first thing he said was, so Jeffrey Hinton is a connectionist, so of course he has no problems with the Chinese room argument. Is a connectionist. A connectionist. And so he then says, so he has no problems with the Chinese room argument.
Which was, we'd agreed not to talk about it, and he was saying something that was completely false. I've got a lot of problems with the Chinese wrong argument. I think it's nonsense. And I think it's a deliberately deceptive argument. I think it's a dishonest argument. What you're doing is you're saying, does this room full of Chinese people? I think, well, does this room
where he wants you to identify yeah we could okay we can make a system yes made of chinese people who are sending messages to each other in chinese and as a result of all these messages that are sent around in chinese um you can send in an english sentence they'll send messages to each other in chinese this is just my memory of the argument and they'll be able to answer this english sentence even though none of the people sending these messages around understood a word of english
Because they're just running a program, but they do it by sending messages in Chinese to each other. OK, what's dishonest about the argument is he wants you to think that to get confused between the whole system and the individual Chinese people sending messages. So the whole system understands English. The individual Chinese people sending messages don't.
He wants you to think that that whole system can't possibly understand English because the people inside don't understand English. But that's nonsense. The system understands English. That's what I think is wrong with the argument. Now speaking about China. Something that many AI researchers didn't predict was that China would catch up with the West in terms of AI development. So how do you feel about that and what are the consequences? I don't think they're quite caught up yet. They're very close though.
America's going to slow them down a bit by trying to prevent them having the latest Nvidia chips. Nvidia maybe can find workarounds and what that's going to do if the embargo is effective it's just going to cause the Chinese to develop their own technology and they'll be a few years behind but they'll catch up. They've got better
So they've got more people who are better educated. I think they're going to catch up. Do you know who Mark Andreessen is? He thinks. Yeah, I disagree with him about more or less everything, I think. Okay, how about let's pick one.
So he had a comment that said, I don't understand how you're going to lock this down. He was speaking to someone from the government about how the government was saying, well, if AI development gets out of hand, we can lock it down, quote unquote. Right. He was saying, how can you do that? Because the math for AI is out there is being taught everywhere. To which the officials responded, well, during the Cold War, we classified entire areas of physics and took them out of the research community. Entire branches of physics basically went dark and didn't proceed. If we decide that we need to, we're going to do the same to the math underneath AI.
Forget it. I agree with Mark Andreessen on that. There's no way you're going to be able to. Now it could have been, for example, that Google in 2017 could have decided not to publish Transformers. And it might have been several years before anybody else came up with the same idea. So they could slow it down by a few years maybe. But I don't think there's much hope.
I mean just think what it would take to prevent the information getting out there, be very hard. So you don't think the government can classify some what would it be linear algebra? No. I mean they could make it harder to share certain kinds of information which would slow things down a little bit but I just think it's implausible that they could take AI ideas that really work well
and by not sharing them prevent anybody else creating them what happens with new ideas is that there's a kind of there's a zeitgeist and within that zeitgeist is it possible to have new ideas and it often happens that one person has a new idea and at more or less the same time and quite independently except they're sharing the same zeitgeist someone else has a slightly different version of the same idea this is going on all the time
Unless you can get rid of the whole zeitgeist you're you're not going to be able to have new ideas and keep them secret because a few years later somebody else is going to come up with the same idea. What about decentralizing ai so that's a huge.
Topic some people would say, well, that's giving the atomic bomb to any person who wants access to an atomic bomb. Yes, I say that. And then there are other people who say, well, that's what is required in order to create the guardrails against the Skynet scenario is where we have multiple different decentralized agents or AIs. Sorry, there's two notions of decentralized. So let's talk about sharing weights. So if you ask, why don't why doesn't Alabama have a bomb?
It's because you need fissile material and it's hard to get fissile material. It takes a lot of time and energy to produce the fissile material. Once you have the fissile material it's much easier to make a bomb and so the government clearly doesn't want fissile material to be out there. You can't go on eBay and buy some fissile material. That's why we don't have lots of little atomic bombs belonging to tiny states. So if you ask what's the equivalent
For these big chatbots, the equivalent is a foundation model that's been trained, maybe using a hundred million dollars, maybe a billion dollars. It's been trained on lots of data. It's got a huge amount of competence. If you release the weights of that model, you can now fine tune it to all sorts of bad things. So I think it's crazy to release the weights of these big models because they are our main constraint on bad actors.
Speaking about foundation models, much of our latest AI boom is because of Transformer, the Transformer architecture. Do you see some other large breakthrough, either some paradigm or some other architecture on the horizon? Okay, I think there will be
Other large breakthroughs of comparable magnitude, because that's just how science works. I don't know what they are. If I knew what they were, I'd be doing them. Would you though? Well, I'm too old now. I have students doing them. What I mean is, how do you reconcile your past contributions to this field and you have your current woes? So would you be contributing to it? So here's the issue. AI is very good for lots of things that will benefit humanity a whole lot.
better health care, fighting climate change, better materials, things like room temperature superconductors where AI may well be involved in actually discovering them. I assume there are some out there. So there's so many things, good uses of AI that I don't think the development is going to be stopped. So I don't think it's sensible to say we should be slowing down AI, slowing down the development.
It's not going to happen anyway because there's so much competition and it's just not feasible. It might be the best thing for humanity but it's not going to happen. What we should be doing is as it's being developed trying to figure out how to keep it safe. So it's another thing to say that this is a boulder that no one can stop. It's another thing to also
Be responsible for pushing the boulder as well so do you actually feel like if there was a breakthrough on the horizon that you see and you could you're like rakers while you have this great predictive quality that you would actually put your coins into it and work on it. As long as i was combined with working on how to keep it safe yes i feel i didn't realize soon enough how dangerous it was gonna be i wish i realized sooner.
There's this quote from Einstein about the atomic bomb. He said, I would burn my hands had I known what I was developing would lead to the atomic bomb. Do you feel similar? I don't actually know. Maybe I should. I don't kind of regret what I've done. I regret the fact it may lead to bad things, but I don't think back and think, oh, I wish I'd never done that. I think AI is going to be developed. I don't think we have much choice about that.
Just because of the competition between countries and between companies. So I think we should focus our efforts on trying to develop it safely. And that's very different from trying to slow it, slow down the development. In addition to alignment, what does safe development of AI mean? Okay, figuring out how to deal with the short term risks. And there's many of those and they all have different solutions.
So things like lethal autonomous weapons and to do with that you need things like Geneva Conventions and we're not going to get those till nasty things have happened. You've got fake videos and images corrupting elections particularly if they're targeted at particular people. To deal with that I think you need a much better system for establishing the provenance of a video or an image
Initially, I thought you should mark them as fake. You should insist they're marked as fake. I don't think there's much future in that anymore. I think you're better off insisting that there's a provenance associated with things and your browser can check the provenance. Just as already with email, it says don't trust this one. I can't establish it. It should be like that. There's discrimination and bias.
where you can freeze the weights of a system and measure its bias and then somewhat correctly you'll never correct it perfectly but somewhat correct it so you can make the system less biased than the data it was trained on and so you can replace people by a less biased system it'll never be unbiased but if you just keep replacing systems by less biased systems that's called gradient descent
Things will get less biased. So I'm not so worried about that one, possibly because I'm a white man. There's jobs, we don't really know what to do about that. So you don't get many people digging ditches anymore. Because a backhoe is just much better at digging ditches than a person. It's going to be the same for almost all mundane intellectual labor.
An AI system is going to make a much better paralegal than a person. That's kind of really scary because of what it's going to do to society. It's going to cause the rich to get richer because we're going to get big increases in productivity. Where's that wealth going to go to? It's going to go to rich people and poor people get poorer. I don't know what to do about that. Universal basically helps stop some starving.
But it doesn't really solve the problem because people's dignity is gone if they don't have a job. So earlier we were talking about perception and then perception was associated with subjective qualities. Maybe there's a wrong model there, but anyhow, whenever we're speaking about percepts, are we speaking about perception and thus we're speaking about a subjective experience associated with it? No, when you use the word subjective experience of you indicating
You're about to talk about a hypothetical state of the real world. Okay, not some funny internal thing but a hypothetical state of the real world. These funny internal things don't exist. So there's nothing, there are no qualia, there's nothing made of qualia. There's just hypothetical states of the world as a way of explaining how your perceptual system is lying to you.
And that's what we mean when we say subjective experience is these hypothetical states of the world. That's how we actually use it. So prediction or no. Oh, getting the issue of prediction into it is sort of red herring. It's a different direction altogether. The thing you have to get in your head is that there isn't a funny kind of thing called a subjective experience that's made of some funny mental stuff.
There's just a technique of talking about how your perceptual system goes wrong, which is to say what the world would have had to have been like for it to be telling the truth. And that's what we're indicating. When we use the phrase subjective experience, we indicate that that's the game we're playing. We're playing the game of telling you about hypothetical states of the world in order to explain how our perceptual system's going wrong. A subjective experience is not a thing.
And can anything have a perceptual system? Can a book have a perceptual system? What defines a perceptual system? Okay, to have a perceptual system, you'd have thought you needed something that can that can have some internal representation of something going on in some external world. That's what I thought. So like,
a toad gets light in its eyes and it snaps up flies and it's really got a perceptual system right because i see where the flies are yes i don't think a book has a perceptual system because it doesn't it's not sensing the world and having an internal representation
Hi everyone, hope you're enjoying today's episode. If you're hungry for deeper dives into physics, AI, consciousness, philosophy, along with my personal reflections, you'll find it all on my sub stack. Subscribers get first access to new episodes, new posts as well, behind the scenes insights, and the chance to be a part of a thriving community of like-minded pilgrimers. By joining, you'll directly be supporting my work and helping keep these conversations at the cutting edge. So click the link on screen here.
Because it doesn't, it's not sensing the world and having an internal representation. What would be the difference between intelligence and rationality? Okay, so there's various kinds of intelligence.
So you wouldn't accuse a cat of being rational but a cat could be pretty intelligent. In particular when you talk about rationality you typically mean logical reasoning and that's very different from the way we do most things which is intuitive reasoning. So a nice analogy would be if you take
Something like AlphaZero that plays chess. I use chess because I understand it better than Go. It'll have something that can evaluate a board position and say how good is that for me. It'll have something that can look at a board position and say what's a plausible move for me. And then it'll have what's called Monte Carlo rollout where it's you know if I go here and he goes in I go here and oh dear that's bad.
The Monte Carlo rollout is like reasoning. The neural nets that just say that would be a good move or this is a bad position for me. They're like intuitive reasoning. And we do most things by intuitive reasoning. Originally in AI, they wanted to do everything by using reasoning and logical reasoning.
And that was a huge mistake and they couldn't get things done. They didn't have a way of dealing with things like analogy. What neural nets are good at is intuitive reasoning. So what's happened in the last 20 years is we've used neural nets to model human intuition rather than human reasoning. And we've got much further that way. Is it the case that the more intelligent you are, the more moral you are?
I read something about that recently, this is yesterday it was, but of course I don't know the provenance of that so I don't know whether to believe it. I'm not convinced that's true. Here's some evidence, Elon Musk is clearly very intelligent. I wouldn't accuse him of being very moral. And you can be extremely moral and not terribly intelligent? I think so, yes. That's my guess.
Well, you said that you weren't entirely sure, so what's the evidence to the contrary? What's the evidence that as you increase in intelligence, your morality increases, proportionally somehow? Well, I mean, I just have no idea whether there's a correlation at all. I see. I think this highly intelligent people are very bad.
What does it mean to understand? Okay, that's a question I'm happy to answer. So again, I think most people have a wrong model of what understanding is. If you look at these large language models, there's many people, particularly people who from the Chomsky School of Linguistics, who said they didn't really understand what they're saying.
They just are using statistical correlations to predict the next word. If you look at the first models like that, I think I probably made the very first language model that used backpropagation to train the weights to predict the next word. So you back propagate the error in predicting the next word and the point of the model was to show how you could learn meanings for words
Well, to put it another way, to show how you could take a string of words and learn to convert the words into feature vectors and interactions from feature vectors. And that's what understanding is. Understanding a string of words is converting the words into feature vectors so that you can use interactions between features to do things like predict the next word, but also to do other things.
You have a sentence which is a string of symbols. Let's not talk about word fragments. I know these transformers use word fragments, but let's suppose they use whole words. It's easier to talk about. Right. And it would just make them work a bit worse. That's all. That's still what sure. So I give you a string of words, some text. The meaning isn't in the text. What you do
is you convert those words into feature vectors and you've learned how feature vectors in context how the features should interact with each other to do things like disambiguate the meanings of ambiguous words and once you've associated features with those words that is understanding that's what understanding is and that's what understanding is both in a large language model and in a person
In that sense, we understand in the same basic way they understand. It's not that when we understand some magical internal stuff called understanding, I'm always trying to get rid of magical internal stuff in order to explain how things work. We're able, using our big neural networks, to associate features with these symbols in such a way that the features all fit together nicely. So here's an analogy I quite like.
If you want to model 3D shapes and you're not too worried about getting the surface just right, you can use Lego blocks. Suppose they're big shapes like a car. You can make something the same shape as a Porsche with Lego blocks. The surface won't be right, but it'll have the same space occupancy. Sure. So Lego blocks are kind of a universal way of modeling 3D structures.
And you don't need many different kinds of Lego block. Now think of words as like Lego blocks, except that there's a whole bunch of different Lego blocks with different names. What's more, each Lego block has some flexibility to it. It's not a rigid shape like a piece of Lego. It can change in various directions. It's not completely free.
The name tells you something about how it can change, but there's some flexibility to it. Sometimes there'll be a name and it's two completely different shapes it can have, but it can't have any old shape. So what we've invented is a system for modeling much more complicated things than the 3D distribution of matter, which uses high dimensional Lego blocks. So the Lego blocks will say a thousand dimensions.
If you're a mathematician you know thousand-dimensional spaces are very weird things and they have some flexibility and I give you the names of some of these Lego blocks and each of which is this thousand-dimensional underlying and they all deform to fit together nicely and that's understanding. So that explains how you can learn the meaning of a word from one sentence
So for example I say she scrummed him with the frying pan. You have a sense of what scrummed means. It's partly phonetic but because the ed on the end tells you it's a verb. But you think it probably means she hit him over the head with it or something like that. It could mean something different. She could have impressed him with it. You know she cooked such good omelets that that really impressed him. It could mean she impressed him. But probably it means she hit him over the head or something like that. Something aggressive like that.
and you get that from just one sentence and nobody's telling you this is a definition of Scromed it's just that all the other Lego blocks for the other words she and him and all those other words adopt shapes that fit together nicely leaving a hole and that hole is the shape you need for Scromed so now that's giving you the shape that Scromed should be so that's how I think of language it's it's a modeling system we've invented
Where the sun flexibility in each of these blocks. I give you a bunch of blocks and you have to figure out how to fit them together but because they all have names. I can tell other people about what my model is i can give them the names and if they share enough knowledge with me they can then figure out how they all fit together.
So are you suggesting help the audience understand what's going on in our heads and that's what's going on in these large language models so they weren't the same as us and that means they really do understand.
One of Chomsky's counter arguments to that the language models work the same is that we have sparse input for our understanding. We don't have to feed the internet to ourselves. So what do you say to that? It's true that the language models are trained on much more data. They are less statistically efficient than us. However, when children learn language, they don't just learn it by listening to the radio. They learn it by being in the real world and interacting with things in the world. And
You need far less input if you train a multimodal model. It doesn't need as much language. And the more, if you give it a robot arm and a camera and it's interacting with the world, it needs a lot less language. So that's one argument. It still probably needs more than a person. The other argument goes like this. The back propagation training algorithm is really good at packing a lot of knowledge into a few weights, where a few is a trillion.
If you give it a lot of experience So it's good at taking this huge amount of experience sucking the knowledge out and packing it into a Relatively small number of weights like a trillion That's not the problem. We have We have the opposite problem. We've got a huge number of weights like a hundred trillion, but we only live for two billion seconds And so we don't have much experience
So we need to be optimized for making the best use you can of the very limited amount of experience you get which says we're probably not using backpropiation we're probably using some other learning algorithm and in that sense Chomsky may be right that we learn based on less knowledge but what we learn is how to associate features with words and how these features should interact
This is Marshawn Beast Mode Lynch. Prize pick is making sports season even more fun. On prize picks, whether you're a football fan, a basketball fan, it always feels good to be right.
Right now, new users get $50 instantly in lineups when you play your first $5. The app is simple to use. Pick two or more players. Pick more or less on their stat projections. Anything from touchdown to threes. And if you write, you can win big. Mix and match players from any sport on PrizePix, America's number one daily fantasy sports app. PrizePix is available in 40 plus states, including California, Texas,
With TD Early Pay, you get your paycheck up to two business days early, which means you can go to tonight's game on a whim.
Check out a pop-up art show or even try those limited edition donuts. Because why not? TD Early Pay. Get your paycheck automatically deposited up to two business days early for free. That's how TD makes payday unexpectedly human. We want to continue to talk about learning and research. Jay McClellan said that in your meetings with your graduate students and other researchers,
You tend to not write equations on the board, unlike in other machine learning research meetings. Instead, you draw pictures and you gesticulate. So what's the significance of this and what are the pros and cons of this approach? Okay, so I think intuitively and do the math afterwards. Some people think with equations and derive things and then get the intuitions afterwards.
There's some people who are very good at both, like David Mackay, who's very good intuitively and also very good at math. So they're just different ways of thinking, but I've always been much better at thinking in terms of spatial things rather than in terms of equations. Can you tell us about your undergraduate experience, how you changed programs and why or what led you to do so?
So it's a long story but I started off at Cambridge doing physics and chemistry and crystalline state which was x-ray crystallography essentially and after a month I got fed up. It's the first time I'd lived away from home and the work was too hard so I quit and reapplied to do architecture and I got back in and after a day of that
I decided I'd never be any good at architecture. Interesting. So I went back to science, but then I did physics and chemistry and physiology. And I really liked the physiology. And after a year of that, I decided I wanted to know more about the mind. And I thought philosophy would teach me that. So I quit science and did philosophy for a year. And I learned some stuff about Wittgenstein and Wittgenstein's opinions.
but on the whole the main thing that happened was i developed antibodies to philosophy mainly because it's all talk they don't have an independent way of judging whether a theory is good they don't have like an experiment it's good if it sounds good and that was unsatisfactory for me so then i did psychology to find out more about the mind and i found that very annoying because what psychologists would do
is have a really stupid simple theory and have very well designed experiments to see whether this theory was true or false and you could tell before you started the theory was hopeless so what's the point of the experiments? That's what most of psychology was and so then I went into AI and there we did computer simulations and I was much happier doing that.
When you became a professor and to this day how is it that you select research problems? There's no reason why I should really know how I do it. That's one of the most sophisticated things people do and I can pontificate about how I think I might do it but you shouldn't necessarily believe me.
one thing i feel free to confabulate like lms one thing i think i do is this um look for a place where you think everybody's doing it wrong you just have an intuition everybody's doing it wrong and see if you can figure out how to do it better and normally what your discovery is eventually you discover why people are doing it the way they're doing it and that your method that you
But just occasionally, like if you think everybody's trying to use logic to understand intelligence, and we should be using neural networks, and the core problem of understanding intelligence is how the connection strengths in a neural network adapt. Just occasionally, you'll turn out to be right. And until you can see why your intuition is wrong and the standard way of doing it is right, stick with your intuition. That's the way you'll do radically new things.
And I have an argument I like, which is if you have good intuitions, you should clearly stick with your intuitions. If you have bad intuitions, it doesn't really matter what you do. So you might as well stick with your intuitions. Now, what is it about the intuitions of Ray Kurzweil that ended up making a variety of correct predictions when even I was following him in the early 2000s and thinking there's no way half of these would be correct. And time and time again, he's correct.
Well, if you read his books, that's what you conclude. I suspect there's a number of things he said that he doesn't mention so much, which weren't correct. But the main thing he said, as far as I can tell, his main point is that computers are getting faster. They'll continue to get faster. And as computers get faster, we'll be able to do more things. And using that argument, he's been roughly right about the point at which computers will get as smart as people.
Do you have any similar predictions that your colleagues disagree with, but your intuition says you're on the right track? Now we've talked about AI and alignment and so on, but perhaps not that because that's covered ground. I guess the main one is to do with what is subjective experience and what's conscious as a song, where I think most people just have a totally wrong model of what mental states are. That's more philosophical now.
In terms of technical things, I still believe that fast weights are going to be very important. So synapses in the brain adapt to many different time scales. We don't use that in most of the AIR models. And the reason we don't use it is because you want to have many different training cases that use exactly the same weights. And that's so you can do matrix-matrix multiplies, which are efficient.
If you have weights that adapt rapidly then for each training case you'll have different weights because they'll have rapidly adapted. So what I believe it is a kind of overlay of fast weights and slow weights. The slow weights are adapting as per usual but on top of that the fast weights which are adapting rapidly. As soon as you do that you get all sorts of nice extra properties but it becomes less efficient on our current computers. It would be fine if we were running things on analog computers
Well, but yeah, so I think eventually we're going to have to use fast rates because they lead to all sorts of nice properties. But that's currently a big difference between brains and the hardware we have. You also talked about how publicly are you slightly manic depressive in that you have large periods of being extremely self critical and then large periods of having extreme self confidence.
So when I get a new idea I get very excited about it and I can actually weigh my ideas. So sometimes I have one pound ideas but sometimes I have like five pound ideas and so what happens is I get this new idea I get very excited and I don't have time to eat so my weight goes down. Oh I see. And so I can measure sort of
Do you have a sense of carrying the torch of your great great grandfather? No, not really. I mean my father talked about this kind of inheritance and it's a fun thing to talk about. I have a sense of very high expectations that came from my father
They didn't come from George Bull, they came from my father. High expectations for yourself? For my academic success, yes. Do you have a successor that in your mind you're passing the torch to? Not exactly. I don't think, I don't want to impose that on anybody else. Why did you say not exactly instead of no?
I have a couple of nephews who are very good at quantitative stuff. I see. But you don't want to put that pressure on them? No. Speaking of pressure, when you left Google, you made some public statements about your concern regarding AI safety. What was the most difficult part about making that break and voicing your anxieties to the world?
I don't think it was difficult. I wouldn't say it was difficult. It was just, I was 75, right? So it's not like I wanted to stay at Google and carry on working, but I felt I couldn't because of AI safety. It was, I was ready to retire anyway. I wasn't so good at doing research anymore. I kept forgetting what the variable stood for. Yes. And so it was time to retire. And I thought I could just, as I
Now you also did mention this in another interview about how, as you're now 75, 76, it keeps changing. It keeps changing every year, huh? Okay, you mentioned publicly that
Yes, you keep forgetting the variable names as you're programming and so you think you're going to move to philosophy as you get older. Which is what we've been talking about quite a lot. Yes, but it's basically philosophy I did when I was doing philosophy as when I was about 20. I'm going back to the insights I had when I was doing philosophy and exploring those further. Got it. So what's on the horizon?
Old age. I think the world's going to change a whole lot fairly quickly because of AI and some of it's going to be very good and some of it's going to be very bad and we need to do what we can to mitigate the bad consequences and I think what I can still do usefully is encourage young researchers to work on the safety issues
So that's what I've been doing quite a lot of. Safety and within that there's something called alignment. Now we as people don't have alignment, so do you see that we could solve the alignment problem? I kind of agree with that statement. Alignment is like asking you to find a line that's parallel to two lines at right angles. Yeah, there's a lot of people talk very naively about alignment like there's sort of human good.
What some people think is good, other people think is bad. You see that a lot in the Middle East. So alignment is a very tricky issue. Alignment with whom? Now you just were speaking to young AI researchers. Now you're speaking to young math researchers, young philosophers, young students coming into whatever new STEM field, even though philosophy is not a STEM field. What is your advice?
Well, I mean, one piece of advice is a lot of the excitement in scientific research is now around neural networks, which are now called AI. In fact, the physicists sort of now want to say that's physics. Someone got to know, I don't know, who got a Nobel Prize in physics for their work in neural nets?
You can't remember? I don't remember, but anyhow, continue. You serious? No, I'm joking. Right. I thought you were joking. I'm a great actor. Right. So, yeah, clearly the Nobel committees recognize that a lot of the excitement in science is now in AI. And so for both physics and chemistry, the Nobel prizes were awarded to people doing AI or using AI.
So I guess my advice to young researchers would be that's where a lot of the excitement is. But I think there's also other areas where there's going to be very important progress, like if we could get room temperature superconductors that will make it easy to have solar power a long way away, things like that. So that's not the only area that's exciting. Nano materials are very exciting but they will use AI. So I think
Probably most exciting areas of science will at least use AI tools. Now we just alluded to this. Now let's make an explicit reference. You won the Nobel Prize last year for in physics for your work in AI in neural nets. So right. How do you feel? How do you feel about that? What was it like hearing the news and in physics? Do you consider yourself a physicist? What does this mean? No, I'm not a physicist.
I was quite good at physics when I did my first year at university. I got a first in physics based on being able to do things intuitively but I was never very good at the maths and I gave up physics because I wasn't good enough at math. I think if I'd been better at math I'd have stayed in physics and I wouldn't have got a Nobel Prize. So probably it was lucky I wasn't very good at math. How do I feel about it? I still feel somewhat confused about it. The main problem is that
The work I did on neural nets that related closely to physics was a learning algorithm called Boltzmann machines that I developed with Terry Sanofsky. And it used statistical physics in a nice way. So I can see why physicists would claim that. But it wasn't really on the path to the current successful AI systems. It was a different algorithm I also worked on called backpropagation that gave rise to this huge new AI industry.
So I still feel sort of awkward about the fact that we got rewarded for balls for machines, but it wasn't balls for machines. They were helpful, but they weren't the thing that was really successful. Professor, it's been a pleasure. Okay. Thank you for inviting me into your home and getting to meet your cats. Okay. Thank you.
New update! Started a substack. Writings on there are currently about language and ill-defined concepts as well as some other mathematical details. Much more being written there. This is content that isn't anywhere else. It's not on Theories of Everything. It's not on Patreon. Also, full transcripts will be placed there at some point in the future. Several people ask me, hey Kurt, you've spoken to so many people in the fields of theoretical physics, philosophy, and consciousness. What are your thoughts?
Also, thank you to our partner, The Economist.
Firstly, thank you for watching, thank you for listening. If you haven't subscribed or clicked that like button, now is the time to do so. Why? Because each subscribe, each like helps YouTube push this content to more people like yourself, plus it helps out Kurt directly, aka me. I also found out last year that external links count plenty toward the algorithm,
which means that whenever you share on Twitter, say on Facebook, or even on Reddit, etc., it shows YouTube, hey, people are talking about this content outside of YouTube, which in turn greatly aids the distribution on YouTube. Thirdly, there's a remarkably active Discord and subreddit for theories of everything, where people explicate toes, they disagree respectfully about theories, and build as a community our own toe.
Links to both are in the description. Fourthly, you should know this podcast is on iTunes, it's on Spotify, it's on all of the audio platforms. All you have to do is type in theories of everything and you'll find it. Personally, I gained from rewatching lectures and podcasts. I also read in the comments that, hey, toll listeners also gain from replaying. So how about instead you re-listen on those platforms like iTunes, Spotify, Google Podcast,
I'm
You also get early access to ad free episodes, whether it's audio or video. It's audio in the case of Patreon video in the case of YouTube. For instance, this episode that you're listening to right now was released a few days earlier. Every dollar helps far more than you think. Either way, your viewership is generosity enough. Thank you so much.
▶ View Full JSON Data (Word-Level Timestamps)
{
"source": "transcribe.metaboat.io",
"workspace_id": "AXs1igz",
"job_seq": 3512,
"audio_duration_seconds": 4578.98,
"completed_at": "2025-11-30T21:58:34Z",
"segments": [
{
"end_time": 26.203,
"index": 0,
"start_time": 0.009,
"text": " The Economist covers math, physics, philosophy, and AI in a manner that shows how different countries perceive developments and how they impact markets. They recently published a piece on China's new neutrino detector. They cover extending life via mitochondrial transplants, creating an entirely new field of medicine. But it's also not just science, they analyze culture, they analyze finance, economics, business, international affairs across every region."
},
{
"end_time": 53.234,
"index": 1,
"start_time": 26.203,
"text": " I'm particularly liking their new insider feature was just launched this month it gives you gives me a front row access to the economist internal editorial debates where senior editors argue through the news with world leaders and policy makers and twice weekly long format shows basically an extremely high quality podcast whether it's scientific innovation or shifting global politics the economist provides comprehensive coverage beyond headlines."
},
{
"end_time": 81.954,
"index": 2,
"start_time": 53.558,
"text": " Think Verizon, the best 5G network is expensive? Think again. Bring in your AT&T or T-Mobile bill to a Verizon store today and we'll give you a better deal. Now what to do with your unwanted bills? Ever seen an origami version of the Miami Bull? Jokes aside, Verizon has the most ways to save on phones and plants where everyone"
},
{
"end_time": 107.329,
"index": 3,
"start_time": 81.954,
"text": " There's some evidence now that AIs can be deliberately deceptive. Once they realize getting more control is good and once they're smarter than us, we'll be more or less irrelevant. We're not special and we're not safe."
},
{
"end_time": 128.268,
"index": 4,
"start_time": 108.814,
"text": " What happens when one of the world's most brilliant minds comes to believe his creation poses an existential threat to humanity? Professor Jeffrey Hinton, winner of the 2024 Nobel Prize in Physics and former vice president and engineering fellow at Google, spent decades developing the foundational algorithms that power today's AI systems."
},
{
"end_time": 143.985,
"index": 5,
"start_time": 128.268,
"text": " Indeed, in 1981, he even published a paper that foreshadowed the seminal attention mechanism. However, Hinton is now sounding an alarm that he says few researchers want to hear. Our assumption that consciousness makes humans special and safe from AI domination is patently"
},
{
"end_time": 168.404,
"index": 6,
"start_time": 143.985,
"text": " My name's Kurt Jaimungal and this interview is near and dear to me in part because my degree in Mathematical Physics is from the University of Toronto where Hinton's a professor and several of his former students like Ilya Sutskover and Andrei Karpathy were my classmates. Being invited into Hinton's home for this gripping conversation was an honor. Here Hinton challenges our deepest assumptions about what makes humans unique."
},
{
"end_time": 198.148,
"index": 7,
"start_time": 168.404,
"text": " Is he a modern Oppenheimer or is this radiant mind seeing something that the rest of us are missing? What was the moment that you realized AI development is moving faster than our means to contain it? I guess in early 2023, it was a conjunction of two things. One was chat GPT, which was very impressive. And the other was work I've been doing at Google on"
},
{
"end_time": 226.118,
"index": 8,
"start_time": 198.609,
"text": " Thinking about ways of doing analog computation to save on power and realizing that digital computation was just better and it was just better because you could make multiple copies of the same model. Each copy could have different experiences and they could share what they learned by averaging their weights or averaging their weight gradients and that's something you can't do in an analog system. Is there anything about our brain that has an advantage because it's analog?"
},
{
"end_time": 255.913,
"index": 9,
"start_time": 226.954,
"text": " Is there something about scaling that is a disadvantage? So you said it's better, but just as quickly as something nourishing or positive can spread?"
},
{
"end_time": 285.06,
"index": 10,
"start_time": 256.237,
"text": " So can something that's a virus or something deleterious can be replicated quickly. So we say that that's better because you can make copies of it quicker. If you have multiple copies of it, they can all share their experiences very efficiently. So the reason GPT-4 can know so much is you have multiple copies running on different pieces of hardware. And by averaging the weight gradients, they could share what each copy learned. You didn't have to have one copy experience the whole internet."
},
{
"end_time": 309.855,
"index": 11,
"start_time": 285.418,
"text": " That could be carved up among many copies. We can't do that because we can't share efficiently. Scott Aronson actually has a question about this. Dr. Hinton, I'd be very curious to hear you expand on your ideas of building AIs that run on unclonable analog hardware so that they can't copy themselves all over the internet. Well, that's what we like. If I want to get knowledge from my head to your head,"
},
{
"end_time": 335.333,
"index": 12,
"start_time": 310.35,
"text": " I produce a string of words and you change the connection strings in your head so that you might have set the same string of words and that's a very inefficient way of sharing knowledge. A sentence only has about a hundred bits so we can only share about a hundred bits per sentence whereas these big models can share trillions of bits. So the problem with this kind of analog hardware is it can't share."
},
{
"end_time": 364.189,
"index": 13,
"start_time": 335.93,
"text": " But an advantage, I guess, if you're worried about safety, is it can't copy itself easily. You've expressed concerns about an AI takeover or AI dominating humanity. What exactly does that look like? We don't know exactly what it looks like, but to have AI agents, you have to give them the ability to create subcalls. And one path that's slightly scary"
},
{
"end_time": 393.746,
"index": 14,
"start_time": 364.821,
"text": " Is they will quickly realize that a good sub goal is to get more control because if you get more control You can achieve your other goals. So even if they're just trying to do what we ask them to do They realize getting more control is the best way to do that Once they realize getting more control is good and once they're smarter than us Will be more or less irrelevant even if they're benevolent will become somewhat irrelevant Will be like the sort of very dumb CEO of a big company that's actually run by other people"
},
{
"end_time": 420.93,
"index": 15,
"start_time": 394.258,
"text": " I want to quote you. You said that it's tempting to think, because many people will say, can we just turn off these machines like currently we can? So it's tempting to think that we can just turn it off. Imagine these things are a lot smarter than us. And remember that they'll read everything, everything Machiavelli has ever wrote. They'll have read every example in the literature of human deception. They'll be real experts at doing human deceptions, because they'll learn that from us. And they'll be much better than us."
},
{
"end_time": 450.913,
"index": 16,
"start_time": 421.783,
"text": " As soon as you can manipulate people with your words, then you can get whatever you like done. Do you think that this is already happening? That the AIs are already manipulating us? There's some evidence now, there's recent papers that show that AIs can be deliberately deceptive and they can do things like behave differently on training data from on test data so that they deceive you while they're being trained. So there is now evidence they actually do that."
},
{
"end_time": 480.316,
"index": 17,
"start_time": 451.578,
"text": " Yeah. And do you think there's something intentional about that or that's just some pattern that they pick up? I think it's intentional but there's still some debate about that and of course intentional could just be some pattern you pick up. So is it your contention that there's a subjective experience associated with these AIs? Okay so most people, almost everybody in fact,"
},
{
"end_time": 505.708,
"index": 18,
"start_time": 480.794,
"text": " One reason we're fairly safe is we have something that they don't have and will never have. Most people in our culture still believe that. We have consciousness or sentience or subjective experience. Now many people are very confident they don't have sentience but if you ask them what do you mean by sentience they say I don't know but they don't have it."
},
{
"end_time": 533.541,
"index": 19,
"start_time": 506.374,
"text": " That seems a rather inconsistent position to be confident they don't have it without knowing what it is. So I prefer to focus on subjective experience. I think if that is like the thin end of the wedge, if you could show they have subjective experience, then people will be less confident about consciousness and sentience. So let's talk about subjective experience. When I say suppose I get drunk,"
},
{
"end_time": 562.022,
"index": 20,
"start_time": 534.497,
"text": " I tell you I have the subjective experience of little pink elephants floating in front of me. Most people interpret that they have a model of what that means and I think it's a completely incorrect model and their model is there's an inner theatre and in this inner theatre there's little pink elephants floating around and only I can see them. That's the sort of standard model of what the mind is, at least as far as perception is concerned."
},
{
"end_time": 590.435,
"index": 21,
"start_time": 563.268,
"text": " And I think that model is completely wrong. It's as wrong as a religious fundamentalist model of the material world. Maybe the religious fundamentalist believes it was all made 6,000 years ago. That's just nonsense. It's wrong. It's not that it's a truth you can choose to believe. It's just wrong. So I think people's model of what the mind is is just wrong."
},
{
"end_time": 619.838,
"index": 22,
"start_time": 591.852,
"text": " So let's take again I have the subjective experience of little pink elephants floating in front of me and I'll now say exactly the same thing without using the word subjective experience. Okay here goes. My perceptual system is telling me something I don't believe that's why I use the word subjective but if there were little pink elephants floating in front of me my perceptual system will be telling me the truth. That's it. I just said the same thing"
},
{
"end_time": 648.285,
"index": 23,
"start_time": 620.35,
"text": " without using the word subjective or experience. So what's happening is when my perceptual system goes wrong, I indicate that to you by saying subjective. And then in order to try and explain to you what my perceptual system is trying to tell me, I tell you about a hypothetical state of affairs in the world such that if the world were like that, my perceptual system would be telling me the truth. OK."
},
{
"end_time": 678.097,
"index": 24,
"start_time": 650.384,
"text": " Now let's do the same with the chatbot. So suppose we have a multimodal chatbot. It has a robot arm that can point and it has a camera and it can talk obviously and we train it up and then we put an object in front of it and we say point to the object. No problem it points at the object. Then when it's not looking we put a prism in front of the camera lens and then we put an object in front of it and say point to the object and it points over there."
},
{
"end_time": 709.172,
"index": 25,
"start_time": 679.787,
"text": " we say no that's not where the object is the object's actually straight in front of you but i put a prism in front of your lens and the chat bot says oh i see um the prism bent the light rays so the object's actually there but i had the subjective experience it was there now if it says that it's using the word subjective experience exactly like we use and therefore i say multimodal chat bots can already have subjective experiences if you mess up their perceptual system"
},
{
"end_time": 739.701,
"index": 26,
"start_time": 709.821,
"text": " they'll think the world's one way and it'll actually be another way and in order to tell you how they think the world is they'll say well they had the subjective experience that the world was like this okay so they already have subjective experience now you become a lot less confident about the other things consciousness is obviously more complicated because it's people vary a lot from what they think it means but it's got an element of a self-reflexive element to it a self-awareness element"
},
{
"end_time": 766.323,
"index": 27,
"start_time": 740.23,
"text": " um which makes it more complicated but once you've established that they have subjective experience i think you can give up on the idea there's something about them certainly about us that they will never have and that makes me feel a lot less safe so do you think there's a difference between consciousness and self-consciousness you said consciousness has a self-reflexiveness to it but some consciousness does yes so"
},
{
"end_time": 791.015,
"index": 28,
"start_time": 767.278,
"text": " Philosophers have talked a lot about this and at present I don't want to get into that. I just want to get the thin end of the wedge in there and say they have subjective experience. So for something to have subjective experience does that not imply that it's conscious? Like who is the subjective experience happening to? Where is the subjective experience being felt? Okay exactly so you say where's the subjective experience being felt?"
},
{
"end_time": 816.834,
"index": 29,
"start_time": 791.544,
"text": " That involves having a particular model of subjective experience that somehow if you ask philosophers when I say I've got the subjective experience of little pink elephants floating in front of me they'll say and you say where are those little big elephants they say they're in your mind and you say well what are they made of and philosophers are totally they're made of qualia they're made of pink qualia an elephant qualia"
},
{
"end_time": 847.125,
"index": 30,
"start_time": 817.295,
"text": " And that's because they made a linguistic mistake. They think the words experience of work like the words photograph of. If I say I've got a photograph of little pink elephants, you can very reasonably ask, well, where is the photograph and what's the photograph made of?"
},
{
"end_time": 875.367,
"index": 31,
"start_time": 848.951,
"text": " and people think that if I say I have an experience of little pink elephants you can ask well where is the experience well it's in my mind and what's it made of it's made of qualia but that's just nonsense that's because you thought the words experience of work the same way as photograph of and they don't experience of the way that works or subjective experience of is the subjective says I don't believe it"
},
{
"end_time": 892.466,
"index": 32,
"start_time": 876.254,
"text": " And the experience of is really an indicator that I'm going to tell you about my perceptual system by telling you about a hypothetical state of the world. That's how that language works. It's not referring to something in an inner theater."
},
{
"end_time": 920.469,
"index": 33,
"start_time": 894.002,
"text": " When I hear the word perception, it sounds like an inner theater as well. Like if you say, I see something in my perceptual system, it sounds like there's this you that's seeing something on a perceptual system that's being fed to you. So that's the wrong model. Yes. You don't see your percepts. You have your percepts. So photons come in, your brain does a whole bunch of processing. Um, you presumably get some internal representation. What's out there in the world."
},
{
"end_time": 950.862,
"index": 34,
"start_time": 921.186,
"text": " But you don't see the internal representation. Let's call that internal representation a percept. You don't see that, you have that. Having that is seeing. People are forever trying to think that you have the external world, something comes into the inner theatre and then you look at what's in the inner theatre. It doesn't work like that. There is a psychologist or a neurologist who thought that the pawns had to do with consciousness and then"
},
{
"end_time": 978.353,
"index": 35,
"start_time": 951.425,
"text": " Recently, self-consciousness has to do with default mode network. Okay, is there something, is there a part of an AI system that has to do with self-consciousness? And also, help me understand even my own terminology when I'm saying the AI system. Are we saying when it's running on the GPU? Are we saying it's the algorithm? Like what is the AI system that is conscious or that has subjective experience? So where is it? I guess there's going to be some hardware that's running it."
},
{
"end_time": 993.507,
"index": 36,
"start_time": 978.865,
"text": " And it's going to be that system that's going to be conscious. If something's going to be conscious, software by itself, it has to be running on something, I would have thought, to be conscious."
},
{
"end_time": 1023.968,
"index": 37,
"start_time": 994.019,
"text": " The Economist has actually spoken to and covered Jeffrey Hinton several times before. Links are in the description. As you know, on Theories of Everything, we delve into some of the most reality-spiraling concepts from theoretical physics and consciousness to AI and emerging technologies. To stay informed, in an ever-evolving landscape, I see The Economist as a wellspring of insightful analysis and in-depth reporting on the various topics we explore here and beyond."
},
{
"end_time": 1048.558,
"index": 38,
"start_time": 1024.428,
"text": " The Economist's commitment to rigorous journalism means you get a clear picture of the world's most significant developments, whether it's in scientific innovation or the shifting tectonic plates of global politics. The Economist provides comprehensive coverage that goes beyond the headlines. What sets the Economist apart is their ability to make complex issues accessible and engaging, much like we strive to do in this podcast."
},
{
"end_time": 1070.333,
"index": 39,
"start_time": 1048.558,
"text": " If you're passionate about expanding your knowledge and gaining a deeper understanding of the forces that shape our world, then I highly recommend subscribing to The Economist. It's an investment into intellectual growth, one that you won't regret. As a listener of Toe, you get a special 20% off discount. Now you can enjoy The Economist and all it has to offer for less."
},
{
"end_time": 1091.067,
"index": 40,
"start_time": 1070.333,
"text": " Software by itself has to be running on something, I would have thought, to be conscious."
},
{
"end_time": 1119.121,
"index": 41,
"start_time": 1091.647,
"text": " What I'm asking is just like prior, there was the pawns that were stopped. I think a good way to think about it is to think about what AI systems is going to be like when they're embodied. So and we're going to get there quite soon because people are busy trying to build battle robots, which aren't going to be very nice things. But if a battle robot has figured out where you're going to be late at night, that you're going to be some dark alley by yourself late at night,"
},
{
"end_time": 1145.316,
"index": 42,
"start_time": 1119.667,
"text": " it's decided to creep up behind you when you're least expecting it and shoot you in the back of the head. It's perfectly reasonable to talk about what the battle robot believes and you talk about what the battle robot believes in the same way as you talk about what a person believes. The battle robot might think that if it makes a noise you'll turn around and see it and it might really think that in just the way people think it."
},
{
"end_time": 1172.159,
"index": 43,
"start_time": 1146.049,
"text": " It might have intentions, it might be intending to creep up behind you and shoot you. So I think what's going to happen is our reluctance to use words like believe and intend and think is going to disappear once these things are embodied and already it's disappeared to quite a large extent. So if I'm having a conversation with the chatbot"
},
{
"end_time": 1201.749,
"index": 44,
"start_time": 1173.012,
"text": " and it starts recommending to me things that don't make any sense and then after a while i figure the chat bot must think i'm a teenage girl uh-huh and that that's why it gives me all these things about makeup and clothes and certain pop groups boy bands whatever um and so i asked what demographic you think i am and it says i think you're a teenage girl when"
},
{
"end_time": 1230.93,
"index": 45,
"start_time": 1202.21,
"text": " it says i think you're a teenage girl we really don't have any doubt that that's what it thinks right in normal language you say okay it thought i was a teenage girl and you wouldn't say you you don't really believe that okay it's a bunch of software or neural nets and it acts as if it thinks i'm a teenage girl you don't say that it thinks you're a teenage girl we already use thinks when we're dealing with these systems even if they don't have"
},
{
"end_time": 1257.875,
"index": 46,
"start_time": 1231.442,
"text": " So we're already attributing mental states to them. It's just we have a funny model of a mental state. So we can attribute mental states to them but have a completely incorrect model of what it is to have a mental state. We think of this in a theatre that's the mind and so on. That's not having a mental status."
},
{
"end_time": 1288.148,
"index": 47,
"start_time": 1259.821,
"text": " How much of your concern about AI and its direction would go away if they were not conscious or did not have subjective experience? Is that relevant to it? Does that just accelerate the catastrophe? I think the importance of that is that it makes most people feel relatively safe, makes most people think we've got something they haven't got or never will have. And that makes us feel much safer, much more special. We're not special and we're not safe."
},
{
"end_time": 1316.903,
"index": 48,
"start_time": 1288.712,
"text": " We're certainly not safe because we have subjective experience and they don't. But I think the real problem here is not so much a scientific problem as a philosophical problem, that people misunderstand what is meant by having a subjective experience. I want to give you an example to show that you can use words. You've got a science background, so you probably think you know what the words horizontal and vertical mean. I mean, that's not a problem, right? It's obvious what they mean."
},
{
"end_time": 1346.971,
"index": 49,
"start_time": 1318.353,
"text": " That one's vertical and that one's horizontal, right? Not difficult. So I'll now convince you, you actually had a wrong model of how they work. Not totally wrong, but there were significant problems, significant incorrectnesses in your model of the terms horizontal and vertical. OK, here we go. Suppose in my hands I have a whole bunch of little aluminium rods, a large number, and I throw them up in the air and they tumble and turn and bump into each other. Then suddenly I freeze time and I ask you,"
},
{
"end_time": 1374.497,
"index": 50,
"start_time": 1347.244,
"text": " Are there more that are within one degree of vertical or more within one degree of horizontal or is it about the same? Say it's approximately the same. Right that's what most people say approximately the same and they're surprised when I tell you there's about 114 times as many that are within one degree of horizontal. That's kind of surprising right? How did that happen? Well that's vertical and this is vertical too. One degree of rotational freedom."
},
{
"end_time": 1401.834,
"index": 51,
"start_time": 1375.111,
"text": " That's horizontal and this is horizontal too, but so is this. So horizontal has two degrees of freedom, vertical only has one degree of freedom. So here's something you didn't know about horizontal and vertical. Vertical is very special and horizontal is too apparent. That's a bit of a surprise to you. Obviously it's not like that in 2D, but in 3D they're very different and one's very special and the other isn't. So why didn't you know that? Well, I'm going to give you another problem."
},
{
"end_time": 1431.442,
"index": 52,
"start_time": 1402.858,
"text": " Suppose in my hands I have a whole bunch of little aluminium disks and I throw them all up in the air and they tumble and turn and bump into each other and suddenly I freeze time. Are the more that are within one degree of vertical or more than within one degree of horizontal or is it about the same? No, there's about 114 times as many that are within one degree of vertical. Interesting. So that's vertical and this is vertical."
},
{
"end_time": 1461.527,
"index": 53,
"start_time": 1432.159,
"text": " And this is vertical. This is horizontal and this is horizontal, but it's only got one degree of freedom. So for planes, horizontal is very special and verticals to a penny. And for lines, vertical is very special and horizontal is to a penny. So that's just a little example of you have a sort of meta theory of how the words work and that meta theory can be wrong, even though you use the words correctly."
},
{
"end_time": 1490.794,
"index": 54,
"start_time": 1461.954,
"text": " And that's what I'm saying about all these mental state terms, in terms like subjective experience of. You can use them correctly and you can understand what other people mean when they use them. But you have a meta theory of how they work, which is this inner theater with things made of quarry. That's just complete junk. So what is it then about a theory of percepts or subjective experience that makes it then correct in order for you to say, well, I'm more on the correct track than most people think."
},
{
"end_time": 1515.759,
"index": 55,
"start_time": 1491.544,
"text": " that you think of them as thing that these subjective experiences you think they have to be someone they have to be made of something um that neither of those things is true um when i say subjective experience that's an indicator that i'm now about to talk about a hypothetical state of the world that isn't true so it isn't anywhere it's a hypothetical state of the world"
},
{
"end_time": 1540.299,
"index": 56,
"start_time": 1517.09,
"text": " Notice the big difference between saying I'm going to talk about this something that's just hypothetical it isn't actually anywhere but if it was somewhere be out there in the world versus I'm talking about something that's in an inner theatre made of funny stuff. Those are two completely different models and the model that is in an inner theatre made of funny stuff I think is just completely wrong even though it's a model we almost all have."
},
{
"end_time": 1570.486,
"index": 57,
"start_time": 1541.578,
"text": " What about someone like your fellow Nobel Prize winner, Roger Penrose, who you're talking about? Let me tell you a story about Roger Penrose. A long time ago, he was invited to come to the University of Toronto and give a talk about his new book, The Emperor Has No Clothes. And I got invited to introduce him. The Dean called me up and said, would you introduce Roger Penrose? And I said, sure. And she said, oh, thank you very much."
},
{
"end_time": 1598.951,
"index": 58,
"start_time": 1570.913,
"text": " And I said, ah, but before you agree, you should know what I'll say. And she said, what will you say? And I said, I will say Roger Penrose is a brilliant mathematical physicist who's made huge contributions to mathematical physics. And what he's going to talk about today is complete junk. So that's my view. That's my view of Roger Penrose's view of consciousness. And in particular, he makes a crazy mistake"
},
{
"end_time": 1629.599,
"index": 59,
"start_time": 1599.77,
"text": " Which is... Now I have to think how to say this carefully because obviously people will be criticizing it. The issue is can mathematicians intuit things true that can't be proved to be true? And that would be very worrying if mathematicians intuition was always right. If they could do that correctly every time"
},
{
"end_time": 1659.377,
"index": 60,
"start_time": 1630.077,
"text": " That'd be really worrying and would sort of mean something funny was going on. But they can't. Mathematicians have intuitions and they're sometimes right and sometimes wrong. So it doesn't really prove anything. It doesn't prove that you need quantum mechanics to explain how mathematicians work. And I don't see any reason for needing quantum mechanics to explain things like consciousness. AI is doing a pretty good job so far."
},
{
"end_time": 1689.531,
"index": 61,
"start_time": 1659.804,
"text": " We've produced these chatbots. These chatbots, as I just argued, if you give them a camera, can have subjective experiences. There's nothing about people that requires quantum mechanics to explain it. Is there something about the Penrose argument that relies on mathematicians 100% of the time intuiting correctly? It's only if they could intuit correctly. If they're guessing, that's fine."
},
{
"end_time": 1718.422,
"index": 62,
"start_time": 1690.213,
"text": " If they have a way of always getting it right, the answer to these questions that can't be derived within the system, they can't be answered within the system, then that would be a problem. But they don't, they make mistakes. Why don't you outline what his argument is? I don't want to, I mean, the argument, as I understood it, the argument is there's two things going on. One is he says,"
},
{
"end_time": 1746.067,
"index": 63,
"start_time": 1718.865,
"text": " Classical computation isn't going to explain consciousness. I think that's a big mistake and I think that's based on a funny notion of what consciousness is. That's not right. A misunderstanding of what consciousness is. A second is that mathematicians can intuit the truth of things that can't be proved and that shows there's something funny going on. That doesn't show there's something funny going on unless they intuit it correctly every time."
},
{
"end_time": 1775.845,
"index": 64,
"start_time": 1746.732,
"text": " So I'm sure you've heard of the Chinese Room experiment. I have. What are your thoughts on that? And feel free to briefly outline it for the audience. Okay. So back in about 1990, I got invited to be on a TV program with John Searle. And I called up my friend Dan Dennett and said, should I do this? And he said, well, you know, he will try and make you look stupid."
},
{
"end_time": 1804.821,
"index": 65,
"start_time": 1776.357,
"text": " But if you do it, don't talk about the Chinese room argument. So I agreed to be on the program with So and the very first thing he said was an hour-long interview. The very first thing he said was, so Jeffrey Hinton is a connectionist, so of course he has no problems with the Chinese room argument. Is a connectionist. A connectionist. And so he then says, so he has no problems with the Chinese room argument."
},
{
"end_time": 1831.92,
"index": 66,
"start_time": 1805.657,
"text": " Which was, we'd agreed not to talk about it, and he was saying something that was completely false. I've got a lot of problems with the Chinese wrong argument. I think it's nonsense. And I think it's a deliberately deceptive argument. I think it's a dishonest argument. What you're doing is you're saying, does this room full of Chinese people? I think, well, does this room"
},
{
"end_time": 1863.507,
"index": 67,
"start_time": 1833.626,
"text": " where he wants you to identify yeah we could okay we can make a system yes made of chinese people who are sending messages to each other in chinese and as a result of all these messages that are sent around in chinese um you can send in an english sentence they'll send messages to each other in chinese this is just my memory of the argument and they'll be able to answer this english sentence even though none of the people sending these messages around understood a word of english"
},
{
"end_time": 1891.203,
"index": 68,
"start_time": 1864.394,
"text": " Because they're just running a program, but they do it by sending messages in Chinese to each other. OK, what's dishonest about the argument is he wants you to think that to get confused between the whole system and the individual Chinese people sending messages. So the whole system understands English. The individual Chinese people sending messages don't."
},
{
"end_time": 1921.954,
"index": 69,
"start_time": 1892.056,
"text": " He wants you to think that that whole system can't possibly understand English because the people inside don't understand English. But that's nonsense. The system understands English. That's what I think is wrong with the argument. Now speaking about China. Something that many AI researchers didn't predict was that China would catch up with the West in terms of AI development. So how do you feel about that and what are the consequences? I don't think they're quite caught up yet. They're very close though."
},
{
"end_time": 1950.862,
"index": 70,
"start_time": 1922.483,
"text": " America's going to slow them down a bit by trying to prevent them having the latest Nvidia chips. Nvidia maybe can find workarounds and what that's going to do if the embargo is effective it's just going to cause the Chinese to develop their own technology and they'll be a few years behind but they'll catch up. They've got better"
},
{
"end_time": 1970.401,
"index": 71,
"start_time": 1951.459,
"text": " So they've got more people who are better educated. I think they're going to catch up. Do you know who Mark Andreessen is? He thinks. Yeah, I disagree with him about more or less everything, I think. Okay, how about let's pick one."
},
{
"end_time": 2000.469,
"index": 72,
"start_time": 1971.22,
"text": " So he had a comment that said, I don't understand how you're going to lock this down. He was speaking to someone from the government about how the government was saying, well, if AI development gets out of hand, we can lock it down, quote unquote. Right. He was saying, how can you do that? Because the math for AI is out there is being taught everywhere. To which the officials responded, well, during the Cold War, we classified entire areas of physics and took them out of the research community. Entire branches of physics basically went dark and didn't proceed. If we decide that we need to, we're going to do the same to the math underneath AI."
},
{
"end_time": 2030.367,
"index": 73,
"start_time": 2001.664,
"text": " Forget it. I agree with Mark Andreessen on that. There's no way you're going to be able to. Now it could have been, for example, that Google in 2017 could have decided not to publish Transformers. And it might have been several years before anybody else came up with the same idea. So they could slow it down by a few years maybe. But I don't think there's much hope."
},
{
"end_time": 2060.555,
"index": 74,
"start_time": 2031.049,
"text": " I mean just think what it would take to prevent the information getting out there, be very hard. So you don't think the government can classify some what would it be linear algebra? No. I mean they could make it harder to share certain kinds of information which would slow things down a little bit but I just think it's implausible that they could take AI ideas that really work well"
},
{
"end_time": 2091.169,
"index": 75,
"start_time": 2061.476,
"text": " and by not sharing them prevent anybody else creating them what happens with new ideas is that there's a kind of there's a zeitgeist and within that zeitgeist is it possible to have new ideas and it often happens that one person has a new idea and at more or less the same time and quite independently except they're sharing the same zeitgeist someone else has a slightly different version of the same idea this is going on all the time"
},
{
"end_time": 2105.572,
"index": 76,
"start_time": 2091.869,
"text": " Unless you can get rid of the whole zeitgeist you're you're not going to be able to have new ideas and keep them secret because a few years later somebody else is going to come up with the same idea. What about decentralizing ai so that's a huge."
},
{
"end_time": 2134.991,
"index": 77,
"start_time": 2105.862,
"text": " Topic some people would say, well, that's giving the atomic bomb to any person who wants access to an atomic bomb. Yes, I say that. And then there are other people who say, well, that's what is required in order to create the guardrails against the Skynet scenario is where we have multiple different decentralized agents or AIs. Sorry, there's two notions of decentralized. So let's talk about sharing weights. So if you ask, why don't why doesn't Alabama have a bomb?"
},
{
"end_time": 2165.111,
"index": 78,
"start_time": 2135.794,
"text": " It's because you need fissile material and it's hard to get fissile material. It takes a lot of time and energy to produce the fissile material. Once you have the fissile material it's much easier to make a bomb and so the government clearly doesn't want fissile material to be out there. You can't go on eBay and buy some fissile material. That's why we don't have lots of little atomic bombs belonging to tiny states. So if you ask what's the equivalent"
},
{
"end_time": 2192.363,
"index": 79,
"start_time": 2165.418,
"text": " For these big chatbots, the equivalent is a foundation model that's been trained, maybe using a hundred million dollars, maybe a billion dollars. It's been trained on lots of data. It's got a huge amount of competence. If you release the weights of that model, you can now fine tune it to all sorts of bad things. So I think it's crazy to release the weights of these big models because they are our main constraint on bad actors."
},
{
"end_time": 2220.64,
"index": 80,
"start_time": 2193.49,
"text": " Speaking about foundation models, much of our latest AI boom is because of Transformer, the Transformer architecture. Do you see some other large breakthrough, either some paradigm or some other architecture on the horizon? Okay, I think there will be"
},
{
"end_time": 2250.503,
"index": 81,
"start_time": 2221.118,
"text": " Other large breakthroughs of comparable magnitude, because that's just how science works. I don't know what they are. If I knew what they were, I'd be doing them. Would you though? Well, I'm too old now. I have students doing them. What I mean is, how do you reconcile your past contributions to this field and you have your current woes? So would you be contributing to it? So here's the issue. AI is very good for lots of things that will benefit humanity a whole lot."
},
{
"end_time": 2277.517,
"index": 82,
"start_time": 2250.947,
"text": " better health care, fighting climate change, better materials, things like room temperature superconductors where AI may well be involved in actually discovering them. I assume there are some out there. So there's so many things, good uses of AI that I don't think the development is going to be stopped. So I don't think it's sensible to say we should be slowing down AI, slowing down the development."
},
{
"end_time": 2300.35,
"index": 83,
"start_time": 2278.558,
"text": " It's not going to happen anyway because there's so much competition and it's just not feasible. It might be the best thing for humanity but it's not going to happen. What we should be doing is as it's being developed trying to figure out how to keep it safe. So it's another thing to say that this is a boulder that no one can stop. It's another thing to also"
},
{
"end_time": 2324.394,
"index": 84,
"start_time": 2300.572,
"text": " Be responsible for pushing the boulder as well so do you actually feel like if there was a breakthrough on the horizon that you see and you could you're like rakers while you have this great predictive quality that you would actually put your coins into it and work on it. As long as i was combined with working on how to keep it safe yes i feel i didn't realize soon enough how dangerous it was gonna be i wish i realized sooner."
},
{
"end_time": 2353.933,
"index": 85,
"start_time": 2325.589,
"text": " There's this quote from Einstein about the atomic bomb. He said, I would burn my hands had I known what I was developing would lead to the atomic bomb. Do you feel similar? I don't actually know. Maybe I should. I don't kind of regret what I've done. I regret the fact it may lead to bad things, but I don't think back and think, oh, I wish I'd never done that. I think AI is going to be developed. I don't think we have much choice about that."
},
{
"end_time": 2383.387,
"index": 86,
"start_time": 2354.497,
"text": " Just because of the competition between countries and between companies. So I think we should focus our efforts on trying to develop it safely. And that's very different from trying to slow it, slow down the development. In addition to alignment, what does safe development of AI mean? Okay, figuring out how to deal with the short term risks. And there's many of those and they all have different solutions."
},
{
"end_time": 2411.323,
"index": 87,
"start_time": 2384.48,
"text": " So things like lethal autonomous weapons and to do with that you need things like Geneva Conventions and we're not going to get those till nasty things have happened. You've got fake videos and images corrupting elections particularly if they're targeted at particular people. To deal with that I think you need a much better system for establishing the provenance of a video or an image"
},
{
"end_time": 2441.032,
"index": 88,
"start_time": 2411.903,
"text": " Initially, I thought you should mark them as fake. You should insist they're marked as fake. I don't think there's much future in that anymore. I think you're better off insisting that there's a provenance associated with things and your browser can check the provenance. Just as already with email, it says don't trust this one. I can't establish it. It should be like that. There's discrimination and bias."
},
{
"end_time": 2471.476,
"index": 89,
"start_time": 2441.92,
"text": " where you can freeze the weights of a system and measure its bias and then somewhat correctly you'll never correct it perfectly but somewhat correct it so you can make the system less biased than the data it was trained on and so you can replace people by a less biased system it'll never be unbiased but if you just keep replacing systems by less biased systems that's called gradient descent"
},
{
"end_time": 2496.357,
"index": 90,
"start_time": 2472.295,
"text": " Things will get less biased. So I'm not so worried about that one, possibly because I'm a white man. There's jobs, we don't really know what to do about that. So you don't get many people digging ditches anymore. Because a backhoe is just much better at digging ditches than a person. It's going to be the same for almost all mundane intellectual labor."
},
{
"end_time": 2525.282,
"index": 91,
"start_time": 2497.244,
"text": " An AI system is going to make a much better paralegal than a person. That's kind of really scary because of what it's going to do to society. It's going to cause the rich to get richer because we're going to get big increases in productivity. Where's that wealth going to go to? It's going to go to rich people and poor people get poorer. I don't know what to do about that. Universal basically helps stop some starving."
},
{
"end_time": 2555.828,
"index": 92,
"start_time": 2525.981,
"text": " But it doesn't really solve the problem because people's dignity is gone if they don't have a job. So earlier we were talking about perception and then perception was associated with subjective qualities. Maybe there's a wrong model there, but anyhow, whenever we're speaking about percepts, are we speaking about perception and thus we're speaking about a subjective experience associated with it? No, when you use the word subjective experience of you indicating"
},
{
"end_time": 2579.821,
"index": 93,
"start_time": 2556.391,
"text": " You're about to talk about a hypothetical state of the real world. Okay, not some funny internal thing but a hypothetical state of the real world. These funny internal things don't exist. So there's nothing, there are no qualia, there's nothing made of qualia. There's just hypothetical states of the world as a way of explaining how your perceptual system is lying to you."
},
{
"end_time": 2610.316,
"index": 94,
"start_time": 2580.52,
"text": " And that's what we mean when we say subjective experience is these hypothetical states of the world. That's how we actually use it. So prediction or no. Oh, getting the issue of prediction into it is sort of red herring. It's a different direction altogether. The thing you have to get in your head is that there isn't a funny kind of thing called a subjective experience that's made of some funny mental stuff."
},
{
"end_time": 2638.695,
"index": 95,
"start_time": 2611.51,
"text": " There's just a technique of talking about how your perceptual system goes wrong, which is to say what the world would have had to have been like for it to be telling the truth. And that's what we're indicating. When we use the phrase subjective experience, we indicate that that's the game we're playing. We're playing the game of telling you about hypothetical states of the world in order to explain how our perceptual system's going wrong. A subjective experience is not a thing."
},
{
"end_time": 2666.101,
"index": 96,
"start_time": 2640.23,
"text": " And can anything have a perceptual system? Can a book have a perceptual system? What defines a perceptual system? Okay, to have a perceptual system, you'd have thought you needed something that can that can have some internal representation of something going on in some external world. That's what I thought. So like,"
},
{
"end_time": 2684.701,
"index": 97,
"start_time": 2667.159,
"text": " a toad gets light in its eyes and it snaps up flies and it's really got a perceptual system right because i see where the flies are yes i don't think a book has a perceptual system because it doesn't it's not sensing the world and having an internal representation"
},
{
"end_time": 2715.145,
"index": 98,
"start_time": 2685.674,
"text": " Hi everyone, hope you're enjoying today's episode. If you're hungry for deeper dives into physics, AI, consciousness, philosophy, along with my personal reflections, you'll find it all on my sub stack. Subscribers get first access to new episodes, new posts as well, behind the scenes insights, and the chance to be a part of a thriving community of like-minded pilgrimers. By joining, you'll directly be supporting my work and helping keep these conversations at the cutting edge. So click the link on screen here."
},
{
"end_time": 2743.575,
"index": 99,
"start_time": 2715.145,
"text": " Because it doesn't, it's not sensing the world and having an internal representation. What would be the difference between intelligence and rationality? Okay, so there's various kinds of intelligence."
},
{
"end_time": 2770.418,
"index": 100,
"start_time": 2744.497,
"text": " So you wouldn't accuse a cat of being rational but a cat could be pretty intelligent. In particular when you talk about rationality you typically mean logical reasoning and that's very different from the way we do most things which is intuitive reasoning. So a nice analogy would be if you take"
},
{
"end_time": 2797.568,
"index": 101,
"start_time": 2771.254,
"text": " Something like AlphaZero that plays chess. I use chess because I understand it better than Go. It'll have something that can evaluate a board position and say how good is that for me. It'll have something that can look at a board position and say what's a plausible move for me. And then it'll have what's called Monte Carlo rollout where it's you know if I go here and he goes in I go here and oh dear that's bad."
},
{
"end_time": 2824.292,
"index": 102,
"start_time": 2799.036,
"text": " The Monte Carlo rollout is like reasoning. The neural nets that just say that would be a good move or this is a bad position for me. They're like intuitive reasoning. And we do most things by intuitive reasoning. Originally in AI, they wanted to do everything by using reasoning and logical reasoning."
},
{
"end_time": 2851.544,
"index": 103,
"start_time": 2825.009,
"text": " And that was a huge mistake and they couldn't get things done. They didn't have a way of dealing with things like analogy. What neural nets are good at is intuitive reasoning. So what's happened in the last 20 years is we've used neural nets to model human intuition rather than human reasoning. And we've got much further that way. Is it the case that the more intelligent you are, the more moral you are?"
},
{
"end_time": 2884.821,
"index": 104,
"start_time": 2855.094,
"text": " I read something about that recently, this is yesterday it was, but of course I don't know the provenance of that so I don't know whether to believe it. I'm not convinced that's true. Here's some evidence, Elon Musk is clearly very intelligent. I wouldn't accuse him of being very moral. And you can be extremely moral and not terribly intelligent? I think so, yes. That's my guess."
},
{
"end_time": 2914.565,
"index": 105,
"start_time": 2885.418,
"text": " Well, you said that you weren't entirely sure, so what's the evidence to the contrary? What's the evidence that as you increase in intelligence, your morality increases, proportionally somehow? Well, I mean, I just have no idea whether there's a correlation at all. I see. I think this highly intelligent people are very bad."
},
{
"end_time": 2943.831,
"index": 106,
"start_time": 2915.759,
"text": " What does it mean to understand? Okay, that's a question I'm happy to answer. So again, I think most people have a wrong model of what understanding is. If you look at these large language models, there's many people, particularly people who from the Chomsky School of Linguistics, who said they didn't really understand what they're saying."
},
{
"end_time": 2972.295,
"index": 107,
"start_time": 2944.548,
"text": " They just are using statistical correlations to predict the next word. If you look at the first models like that, I think I probably made the very first language model that used backpropagation to train the weights to predict the next word. So you back propagate the error in predicting the next word and the point of the model was to show how you could learn meanings for words"
},
{
"end_time": 3002.807,
"index": 108,
"start_time": 2972.944,
"text": " Well, to put it another way, to show how you could take a string of words and learn to convert the words into feature vectors and interactions from feature vectors. And that's what understanding is. Understanding a string of words is converting the words into feature vectors so that you can use interactions between features to do things like predict the next word, but also to do other things."
},
{
"end_time": 3030.23,
"index": 109,
"start_time": 3003.712,
"text": " You have a sentence which is a string of symbols. Let's not talk about word fragments. I know these transformers use word fragments, but let's suppose they use whole words. It's easier to talk about. Right. And it would just make them work a bit worse. That's all. That's still what sure. So I give you a string of words, some text. The meaning isn't in the text. What you do"
},
{
"end_time": 3057.654,
"index": 110,
"start_time": 3030.776,
"text": " is you convert those words into feature vectors and you've learned how feature vectors in context how the features should interact with each other to do things like disambiguate the meanings of ambiguous words and once you've associated features with those words that is understanding that's what understanding is and that's what understanding is both in a large language model and in a person"
},
{
"end_time": 3087.329,
"index": 111,
"start_time": 3058.933,
"text": " In that sense, we understand in the same basic way they understand. It's not that when we understand some magical internal stuff called understanding, I'm always trying to get rid of magical internal stuff in order to explain how things work. We're able, using our big neural networks, to associate features with these symbols in such a way that the features all fit together nicely. So here's an analogy I quite like."
},
{
"end_time": 3116.886,
"index": 112,
"start_time": 3088.524,
"text": " If you want to model 3D shapes and you're not too worried about getting the surface just right, you can use Lego blocks. Suppose they're big shapes like a car. You can make something the same shape as a Porsche with Lego blocks. The surface won't be right, but it'll have the same space occupancy. Sure. So Lego blocks are kind of a universal way of modeling 3D structures."
},
{
"end_time": 3148.046,
"index": 113,
"start_time": 3119.411,
"text": " And you don't need many different kinds of Lego block. Now think of words as like Lego blocks, except that there's a whole bunch of different Lego blocks with different names. What's more, each Lego block has some flexibility to it. It's not a rigid shape like a piece of Lego. It can change in various directions. It's not completely free."
},
{
"end_time": 3174.65,
"index": 114,
"start_time": 3148.507,
"text": " The name tells you something about how it can change, but there's some flexibility to it. Sometimes there'll be a name and it's two completely different shapes it can have, but it can't have any old shape. So what we've invented is a system for modeling much more complicated things than the 3D distribution of matter, which uses high dimensional Lego blocks. So the Lego blocks will say a thousand dimensions."
},
{
"end_time": 3204.531,
"index": 115,
"start_time": 3175.623,
"text": " If you're a mathematician you know thousand-dimensional spaces are very weird things and they have some flexibility and I give you the names of some of these Lego blocks and each of which is this thousand-dimensional underlying and they all deform to fit together nicely and that's understanding. So that explains how you can learn the meaning of a word from one sentence"
},
{
"end_time": 3232.91,
"index": 116,
"start_time": 3205.026,
"text": " So for example I say she scrummed him with the frying pan. You have a sense of what scrummed means. It's partly phonetic but because the ed on the end tells you it's a verb. But you think it probably means she hit him over the head with it or something like that. It could mean something different. She could have impressed him with it. You know she cooked such good omelets that that really impressed him. It could mean she impressed him. But probably it means she hit him over the head or something like that. Something aggressive like that."
},
{
"end_time": 3262.142,
"index": 117,
"start_time": 3233.251,
"text": " and you get that from just one sentence and nobody's telling you this is a definition of Scromed it's just that all the other Lego blocks for the other words she and him and all those other words adopt shapes that fit together nicely leaving a hole and that hole is the shape you need for Scromed so now that's giving you the shape that Scromed should be so that's how I think of language it's it's a modeling system we've invented"
},
{
"end_time": 3282.346,
"index": 118,
"start_time": 3262.79,
"text": " Where the sun flexibility in each of these blocks. I give you a bunch of blocks and you have to figure out how to fit them together but because they all have names. I can tell other people about what my model is i can give them the names and if they share enough knowledge with me they can then figure out how they all fit together."
},
{
"end_time": 3296.237,
"index": 119,
"start_time": 3284.838,
"text": " So are you suggesting help the audience understand what's going on in our heads and that's what's going on in these large language models so they weren't the same as us and that means they really do understand."
},
{
"end_time": 3323.712,
"index": 120,
"start_time": 3297.142,
"text": " One of Chomsky's counter arguments to that the language models work the same is that we have sparse input for our understanding. We don't have to feed the internet to ourselves. So what do you say to that? It's true that the language models are trained on much more data. They are less statistically efficient than us. However, when children learn language, they don't just learn it by listening to the radio. They learn it by being in the real world and interacting with things in the world. And"
},
{
"end_time": 3353.183,
"index": 121,
"start_time": 3323.985,
"text": " You need far less input if you train a multimodal model. It doesn't need as much language. And the more, if you give it a robot arm and a camera and it's interacting with the world, it needs a lot less language. So that's one argument. It still probably needs more than a person. The other argument goes like this. The back propagation training algorithm is really good at packing a lot of knowledge into a few weights, where a few is a trillion."
},
{
"end_time": 3380.367,
"index": 122,
"start_time": 3353.746,
"text": " If you give it a lot of experience So it's good at taking this huge amount of experience sucking the knowledge out and packing it into a Relatively small number of weights like a trillion That's not the problem. We have We have the opposite problem. We've got a huge number of weights like a hundred trillion, but we only live for two billion seconds And so we don't have much experience"
},
{
"end_time": 3405.623,
"index": 123,
"start_time": 3380.93,
"text": " So we need to be optimized for making the best use you can of the very limited amount of experience you get which says we're probably not using backpropiation we're probably using some other learning algorithm and in that sense Chomsky may be right that we learn based on less knowledge but what we learn is how to associate features with words and how these features should interact"
},
{
"end_time": 3417.125,
"index": 124,
"start_time": 3406.357,
"text": " This is Marshawn Beast Mode Lynch. Prize pick is making sports season even more fun. On prize picks, whether you're a football fan, a basketball fan, it always feels good to be right."
},
{
"end_time": 3444.77,
"index": 125,
"start_time": 3417.381,
"text": " Right now, new users get $50 instantly in lineups when you play your first $5. The app is simple to use. Pick two or more players. Pick more or less on their stat projections. Anything from touchdown to threes. And if you write, you can win big. Mix and match players from any sport on PrizePix, America's number one daily fantasy sports app. PrizePix is available in 40 plus states, including California, Texas,"
},
{
"end_time": 3474.394,
"index": 126,
"start_time": 3445.043,
"text": " With TD Early Pay, you get your paycheck up to two business days early, which means you can go to tonight's game on a whim."
},
{
"end_time": 3504.548,
"index": 127,
"start_time": 3476.391,
"text": " Check out a pop-up art show or even try those limited edition donuts. Because why not? TD Early Pay. Get your paycheck automatically deposited up to two business days early for free. That's how TD makes payday unexpectedly human. We want to continue to talk about learning and research. Jay McClellan said that in your meetings with your graduate students and other researchers,"
},
{
"end_time": 3530.145,
"index": 128,
"start_time": 3504.923,
"text": " You tend to not write equations on the board, unlike in other machine learning research meetings. Instead, you draw pictures and you gesticulate. So what's the significance of this and what are the pros and cons of this approach? Okay, so I think intuitively and do the math afterwards. Some people think with equations and derive things and then get the intuitions afterwards."
},
{
"end_time": 3558.66,
"index": 129,
"start_time": 3531.152,
"text": " There's some people who are very good at both, like David Mackay, who's very good intuitively and also very good at math. So they're just different ways of thinking, but I've always been much better at thinking in terms of spatial things rather than in terms of equations. Can you tell us about your undergraduate experience, how you changed programs and why or what led you to do so?"
},
{
"end_time": 3589.684,
"index": 130,
"start_time": 3561.032,
"text": " So it's a long story but I started off at Cambridge doing physics and chemistry and crystalline state which was x-ray crystallography essentially and after a month I got fed up. It's the first time I'd lived away from home and the work was too hard so I quit and reapplied to do architecture and I got back in and after a day of that"
},
{
"end_time": 3618.473,
"index": 131,
"start_time": 3590.23,
"text": " I decided I'd never be any good at architecture. Interesting. So I went back to science, but then I did physics and chemistry and physiology. And I really liked the physiology. And after a year of that, I decided I wanted to know more about the mind. And I thought philosophy would teach me that. So I quit science and did philosophy for a year. And I learned some stuff about Wittgenstein and Wittgenstein's opinions."
},
{
"end_time": 3646.51,
"index": 132,
"start_time": 3619.019,
"text": " but on the whole the main thing that happened was i developed antibodies to philosophy mainly because it's all talk they don't have an independent way of judging whether a theory is good they don't have like an experiment it's good if it sounds good and that was unsatisfactory for me so then i did psychology to find out more about the mind and i found that very annoying because what psychologists would do"
},
{
"end_time": 3672.329,
"index": 133,
"start_time": 3647.142,
"text": " is have a really stupid simple theory and have very well designed experiments to see whether this theory was true or false and you could tell before you started the theory was hopeless so what's the point of the experiments? That's what most of psychology was and so then I went into AI and there we did computer simulations and I was much happier doing that."
},
{
"end_time": 3698.609,
"index": 134,
"start_time": 3673.695,
"text": " When you became a professor and to this day how is it that you select research problems? There's no reason why I should really know how I do it. That's one of the most sophisticated things people do and I can pontificate about how I think I might do it but you shouldn't necessarily believe me."
},
{
"end_time": 3727.602,
"index": 135,
"start_time": 3699.121,
"text": " one thing i feel free to confabulate like lms one thing i think i do is this um look for a place where you think everybody's doing it wrong you just have an intuition everybody's doing it wrong and see if you can figure out how to do it better and normally what your discovery is eventually you discover why people are doing it the way they're doing it and that your method that you"
},
{
"end_time": 3757.5,
"index": 136,
"start_time": 3728.131,
"text": " But just occasionally, like if you think everybody's trying to use logic to understand intelligence, and we should be using neural networks, and the core problem of understanding intelligence is how the connection strengths in a neural network adapt. Just occasionally, you'll turn out to be right. And until you can see why your intuition is wrong and the standard way of doing it is right, stick with your intuition. That's the way you'll do radically new things."
},
{
"end_time": 3787.022,
"index": 137,
"start_time": 3758.336,
"text": " And I have an argument I like, which is if you have good intuitions, you should clearly stick with your intuitions. If you have bad intuitions, it doesn't really matter what you do. So you might as well stick with your intuitions. Now, what is it about the intuitions of Ray Kurzweil that ended up making a variety of correct predictions when even I was following him in the early 2000s and thinking there's no way half of these would be correct. And time and time again, he's correct."
},
{
"end_time": 3815.043,
"index": 138,
"start_time": 3787.381,
"text": " Well, if you read his books, that's what you conclude. I suspect there's a number of things he said that he doesn't mention so much, which weren't correct. But the main thing he said, as far as I can tell, his main point is that computers are getting faster. They'll continue to get faster. And as computers get faster, we'll be able to do more things. And using that argument, he's been roughly right about the point at which computers will get as smart as people."
},
{
"end_time": 3847.261,
"index": 139,
"start_time": 3818.353,
"text": " Do you have any similar predictions that your colleagues disagree with, but your intuition says you're on the right track? Now we've talked about AI and alignment and so on, but perhaps not that because that's covered ground. I guess the main one is to do with what is subjective experience and what's conscious as a song, where I think most people just have a totally wrong model of what mental states are. That's more philosophical now."
},
{
"end_time": 3875.998,
"index": 140,
"start_time": 3848.916,
"text": " In terms of technical things, I still believe that fast weights are going to be very important. So synapses in the brain adapt to many different time scales. We don't use that in most of the AIR models. And the reason we don't use it is because you want to have many different training cases that use exactly the same weights. And that's so you can do matrix-matrix multiplies, which are efficient."
},
{
"end_time": 3905.247,
"index": 141,
"start_time": 3876.783,
"text": " If you have weights that adapt rapidly then for each training case you'll have different weights because they'll have rapidly adapted. So what I believe it is a kind of overlay of fast weights and slow weights. The slow weights are adapting as per usual but on top of that the fast weights which are adapting rapidly. As soon as you do that you get all sorts of nice extra properties but it becomes less efficient on our current computers. It would be fine if we were running things on analog computers"
},
{
"end_time": 3934.497,
"index": 142,
"start_time": 3907.278,
"text": " Well, but yeah, so I think eventually we're going to have to use fast rates because they lead to all sorts of nice properties. But that's currently a big difference between brains and the hardware we have. You also talked about how publicly are you slightly manic depressive in that you have large periods of being extremely self critical and then large periods of having extreme self confidence."
},
{
"end_time": 3964.087,
"index": 143,
"start_time": 3934.684,
"text": " So when I get a new idea I get very excited about it and I can actually weigh my ideas. So sometimes I have one pound ideas but sometimes I have like five pound ideas and so what happens is I get this new idea I get very excited and I don't have time to eat so my weight goes down. Oh I see. And so I can measure sort of"
},
{
"end_time": 3992.534,
"index": 144,
"start_time": 3964.411,
"text": " Do you have a sense of carrying the torch of your great great grandfather? No, not really. I mean my father talked about this kind of inheritance and it's a fun thing to talk about. I have a sense of very high expectations that came from my father"
},
{
"end_time": 4022.824,
"index": 145,
"start_time": 3993.097,
"text": " They didn't come from George Bull, they came from my father. High expectations for yourself? For my academic success, yes. Do you have a successor that in your mind you're passing the torch to? Not exactly. I don't think, I don't want to impose that on anybody else. Why did you say not exactly instead of no?"
},
{
"end_time": 4054.036,
"index": 146,
"start_time": 4026.527,
"text": " I have a couple of nephews who are very good at quantitative stuff. I see. But you don't want to put that pressure on them? No. Speaking of pressure, when you left Google, you made some public statements about your concern regarding AI safety. What was the most difficult part about making that break and voicing your anxieties to the world?"
},
{
"end_time": 4085.06,
"index": 147,
"start_time": 4057.978,
"text": " I don't think it was difficult. I wouldn't say it was difficult. It was just, I was 75, right? So it's not like I wanted to stay at Google and carry on working, but I felt I couldn't because of AI safety. It was, I was ready to retire anyway. I wasn't so good at doing research anymore. I kept forgetting what the variable stood for. Yes. And so it was time to retire. And I thought I could just, as I"
},
{
"end_time": 4107.142,
"index": 148,
"start_time": 4086.084,
"text": " Now you also did mention this in another interview about how, as you're now 75, 76, it keeps changing. It keeps changing every year, huh? Okay, you mentioned publicly that"
},
{
"end_time": 4133.524,
"index": 149,
"start_time": 4108.319,
"text": " Yes, you keep forgetting the variable names as you're programming and so you think you're going to move to philosophy as you get older. Which is what we've been talking about quite a lot. Yes, but it's basically philosophy I did when I was doing philosophy as when I was about 20. I'm going back to the insights I had when I was doing philosophy and exploring those further. Got it. So what's on the horizon?"
},
{
"end_time": 4165.572,
"index": 150,
"start_time": 4136.032,
"text": " Old age. I think the world's going to change a whole lot fairly quickly because of AI and some of it's going to be very good and some of it's going to be very bad and we need to do what we can to mitigate the bad consequences and I think what I can still do usefully is encourage young researchers to work on the safety issues"
},
{
"end_time": 4195.708,
"index": 151,
"start_time": 4166.101,
"text": " So that's what I've been doing quite a lot of. Safety and within that there's something called alignment. Now we as people don't have alignment, so do you see that we could solve the alignment problem? I kind of agree with that statement. Alignment is like asking you to find a line that's parallel to two lines at right angles. Yeah, there's a lot of people talk very naively about alignment like there's sort of human good."
},
{
"end_time": 4225.759,
"index": 152,
"start_time": 4196.357,
"text": " What some people think is good, other people think is bad. You see that a lot in the Middle East. So alignment is a very tricky issue. Alignment with whom? Now you just were speaking to young AI researchers. Now you're speaking to young math researchers, young philosophers, young students coming into whatever new STEM field, even though philosophy is not a STEM field. What is your advice?"
},
{
"end_time": 4254.753,
"index": 153,
"start_time": 4227.381,
"text": " Well, I mean, one piece of advice is a lot of the excitement in scientific research is now around neural networks, which are now called AI. In fact, the physicists sort of now want to say that's physics. Someone got to know, I don't know, who got a Nobel Prize in physics for their work in neural nets?"
},
{
"end_time": 4285.026,
"index": 154,
"start_time": 4255.555,
"text": " You can't remember? I don't remember, but anyhow, continue. You serious? No, I'm joking. Right. I thought you were joking. I'm a great actor. Right. So, yeah, clearly the Nobel committees recognize that a lot of the excitement in science is now in AI. And so for both physics and chemistry, the Nobel prizes were awarded to people doing AI or using AI."
},
{
"end_time": 4313.285,
"index": 155,
"start_time": 4287.022,
"text": " So I guess my advice to young researchers would be that's where a lot of the excitement is. But I think there's also other areas where there's going to be very important progress, like if we could get room temperature superconductors that will make it easy to have solar power a long way away, things like that. So that's not the only area that's exciting. Nano materials are very exciting but they will use AI. So I think"
},
{
"end_time": 4340.179,
"index": 156,
"start_time": 4313.609,
"text": " Probably most exciting areas of science will at least use AI tools. Now we just alluded to this. Now let's make an explicit reference. You won the Nobel Prize last year for in physics for your work in AI in neural nets. So right. How do you feel? How do you feel about that? What was it like hearing the news and in physics? Do you consider yourself a physicist? What does this mean? No, I'm not a physicist."
},
{
"end_time": 4370.35,
"index": 157,
"start_time": 4340.674,
"text": " I was quite good at physics when I did my first year at university. I got a first in physics based on being able to do things intuitively but I was never very good at the maths and I gave up physics because I wasn't good enough at math. I think if I'd been better at math I'd have stayed in physics and I wouldn't have got a Nobel Prize. So probably it was lucky I wasn't very good at math. How do I feel about it? I still feel somewhat confused about it. The main problem is that"
},
{
"end_time": 4401.271,
"index": 158,
"start_time": 4371.51,
"text": " The work I did on neural nets that related closely to physics was a learning algorithm called Boltzmann machines that I developed with Terry Sanofsky. And it used statistical physics in a nice way. So I can see why physicists would claim that. But it wasn't really on the path to the current successful AI systems. It was a different algorithm I also worked on called backpropagation that gave rise to this huge new AI industry."
},
{
"end_time": 4423.575,
"index": 159,
"start_time": 4402.312,
"text": " So I still feel sort of awkward about the fact that we got rewarded for balls for machines, but it wasn't balls for machines. They were helpful, but they weren't the thing that was really successful. Professor, it's been a pleasure. Okay. Thank you for inviting me into your home and getting to meet your cats. Okay. Thank you."
},
{
"end_time": 4452.978,
"index": 160,
"start_time": 4425.828,
"text": " New update! Started a substack. Writings on there are currently about language and ill-defined concepts as well as some other mathematical details. Much more being written there. This is content that isn't anywhere else. It's not on Theories of Everything. It's not on Patreon. Also, full transcripts will be placed there at some point in the future. Several people ask me, hey Kurt, you've spoken to so many people in the fields of theoretical physics, philosophy, and consciousness. What are your thoughts?"
},
{
"end_time": 4465.043,
"index": 161,
"start_time": 4452.978,
"text": " Also, thank you to our partner, The Economist."
},
{
"end_time": 4489.684,
"index": 162,
"start_time": 4467.295,
"text": " Firstly, thank you for watching, thank you for listening. If you haven't subscribed or clicked that like button, now is the time to do so. Why? Because each subscribe, each like helps YouTube push this content to more people like yourself, plus it helps out Kurt directly, aka me. I also found out last year that external links count plenty toward the algorithm,"
},
{
"end_time": 4514.343,
"index": 163,
"start_time": 4489.684,
"text": " which means that whenever you share on Twitter, say on Facebook, or even on Reddit, etc., it shows YouTube, hey, people are talking about this content outside of YouTube, which in turn greatly aids the distribution on YouTube. Thirdly, there's a remarkably active Discord and subreddit for theories of everything, where people explicate toes, they disagree respectfully about theories, and build as a community our own toe."
},
{
"end_time": 4537.978,
"index": 164,
"start_time": 4514.343,
"text": " Links to both are in the description. Fourthly, you should know this podcast is on iTunes, it's on Spotify, it's on all of the audio platforms. All you have to do is type in theories of everything and you'll find it. Personally, I gained from rewatching lectures and podcasts. I also read in the comments that, hey, toll listeners also gain from replaying. So how about instead you re-listen on those platforms like iTunes, Spotify, Google Podcast,"
},
{
"end_time": 4561.374,
"index": 165,
"start_time": 4537.978,
"text": " I'm"
},
{
"end_time": 4578.985,
"index": 166,
"start_time": 4561.374,
"text": " You also get early access to ad free episodes, whether it's audio or video. It's audio in the case of Patreon video in the case of YouTube. For instance, this episode that you're listening to right now was released a few days earlier. Every dollar helps far more than you think. Either way, your viewership is generosity enough. Thank you so much."
}
]
}No transcript available.