Audio Player
Starting at:
Elan Barenholtz Λ William Hahn: The (Terrifying) Theory That Your Thoughts Were Never Your Own
July 22, 2025
•
1:13:24
•
undefined
Audio:
Download MP3
ℹ️ Timestamps visible: Timestamps may be inaccurate if the MP3 has dynamically injected ads. Hide timestamps.
Transcript
Enhanced with Timestamps
181 sentences
12,322 words
Method: api-polled
Transcription time: 72m 29s
The Economist covers math, physics, philosophy, and AI in a manner that shows how different countries perceive developments and how they impact markets. They recently published a piece on China's new neutrino detector. They cover extending life via mitochondrial transplants, creating an entirely new field of medicine. But it's also not just science they analyze.
Culture, they analyze finance, economics, business, international affairs across every region. I'm particularly liking their new insider feature. It was just launched this month. It gives you, it gives me, a front row access to The Economist's internal editorial debates.
Where senior editors argue through the news with world leaders and policy makers in twice weekly long format shows. Basically an extremely high quality podcast. Whether it's scientific innovation or shifting global politics, The Economist provides comprehensive coverage beyond headlines. As a total listener, you get a special discount. Head over to economist.com slash TOE to subscribe. That's economist.com slash TOE for your discount.
And this is where, you know, it starts to get a little bit disturbing. Language is an autonomous informational system. One may even call it an organism. And it runs in our brains. It's downloaded against your will. By the time you're reading the waiver, it's too late. What makes you think your thoughts are your own? According to groundbreaking work by professors, Elon Barinholtz and William Hahn, every word you speak
comes from an alien intelligence that installed itself in your brain during infancy. Language, they argue, is an organism that uses human minds as its substrates.
Elon Bernholtz was interviewed before on this channel and it went viral. Link is in the description. Professor Bernholtz contends that our linguistic system operates on meaningless squiggles with no access to the rich sensory world of experience. He believes that LLMs can master language but never feel pain, for instance. Professor William Hahn, on the other hand, who's also been interviewed and it went viral, link is in the description, extends this to virtual machines and software layers.
Professor Han thinks that consciousness is just one of many programs running in parallel. I was invited to moderate this panel live at the University of Toronto with this event organized by Echo Lopto. Again, links to everything are in the description. Today, I want to thank everybody who took time out of their schedule and their busy lives doing the many things that people do here in the University of Toronto to come for this extremely special live
in-person podcast with the Kurt Jaimungal. Give it up for Kurt. He runs the biggest science and philosophy podcast in the world. It's called Theories of Everything. Looked it up on YouTube, Twitter slash X, Instagram, whatever they call it these days. Kurt Jaimungal, Theories of Everything. Very easy to say. And then we're also joined by Professor William Hahn, who came all the way from Florida. Give it up for him.
I'm extremely fortunate to be in the presence of these brilliant people today and the incredible work Kurt has done with this channel and his podcast over the last couple years is
Just nothing short of amazing. And I encourage everybody, if you want to learn about philosophy and science, technology, AI, consciousness, psychedelics like Andreas' work, maybe even hypnosis, all sorts of crazy stuff, I highly recommend you subscribe to this channel. So once again, that's theories of everything. And I think without further ado, we should get started, right? Let's rock and roll.
Okay, welcome everyone. I'd like to get started with five minutes or so on your theory of everything the way that you see the world and then we'll see how Will sees the world and then we're going to compare and contrast it. Okay, thanks. So my theory of everything is sort of a meta theory because it's a theory of theories. It's a theory of thinking and it has a couple of parts to it but I'll cram it all into five minutes. The first one is about the nature of language.
Everybody's familiar now with the large language models, things like chat, GPT, Gemini, Claude, whichever your favorite one, they all have the same basic engine. What all of these models have done is learn how to predict a word from the sequence of words that came before. And in the process of building these very sophisticated machines that are made of lots and lots of these billions of parameters to learn this predictive structure,
What we've actually done is we've discovered a property of language itself. Language is autogenerative. That's the term I've been using, which means that it contains within its structure. The corpus of language itself contains the structure needed to generate itself. What these models are doing are just learning the predictive structure that's already in language. Language is an autonomous informational system. One may even call it an organism.
And it runs in our brains. It doesn't actually have access to the other stuff going on in our brains. When we are talking about objects with features and colors, there's a sensory apparatus in our brain that is processing that information. Our linguistic system doesn't know about that. Our linguistic system is what we can call ungrounded. It only knows about symbols. I call them meaningless squiggles.
It knows the relationship between these meaningless squiggles. It knows that the word red tends to go together with the word apple and the word blue. And it knows that they're in a certain relation to one another. But it doesn't know in any deep sense or any sense at all that these actually extend outside of the world of language itself. So the first observation here is that language is its own system. And what's true in silicon and what's true in these large language models, my claim is, is true in
I have all kinds of things going on inside my brain.
I'm thinking about visual imagery. I have a sense of my corporality, my body. But there's a language model, an LLM, so to speak, that's running and speaking these words. It's doing it in conjunction with these other systems. It's receiving messages from them. But what it's really doing is just its own predictive engine. And this is a radical departure from the way we have thought about language until now. And when I say we, I mean the human species. There is something deeply intuitive or deeply unintuitive about this. There's a deep intuition that our language
is completely embedded with everything else that when we are talking, we are talking about things outside of language. So this is a very radical thesis that language has this property and that it, in fact, what is happening inside of us is in some ways is an autonomous informational system that isn't actually the same one as the rest of our cognitive architecture. One way to think about this is that culture itself has been planted within us. This informational system that's running
And for better or for worse, it is driving a lot of our behavior, but it isn't quite the same thing as the thing that feels, the thing that cares in some ways. So that's part one of the thesis, which you might think, well, that's enough. And indeed, I think that leads to a dramatically different, almost existential view of certainly of knowledge and its place in our lives.
But there's another piece with this, because not only is language autogenerative, there's something special about the way that the models do this generation. So the models do this thing called autoregression, which is a word you'll hear me say a lot if you hang out with me. And autoregression is this particular way of doing prediction, where what you do is you take a sequence, let's say the sequence, the cat sat on the, you can guess the next word, maybe it's Matt. That's a popular ending, but there's other possible endings to that.
So you might say you're predicting what that next word is. What you can do, what we do in autoregression is you take a sequence and then you produce the next most likely token or word, we can just call it a word, and then you feed that back into the system and it's recursive.
So let's say it's the cats on the mat and then what well, I mean, you know, I don't know and then it got up and it, you know, it walked around and rubbed against my leg or something like that. This is all a reasonable sequence that could come after the cat sat on the. Now what these models are doing and this is a really important and astonishing point is they are not thinking directly about the rest of that sequence. They are only ever doing the very next
We're the very next token, as it's called. My thesis is that we are doing the very same thing. What we are doing in our minds, in our brains, is the same kind of computation. We are simply guessing the next token, and then we're taking that token and we hear it. It goes back into our own loop, and then we're saying, well, now there's a sequence with another word. The cat sat on the mat, and then what's next? And we say, okay, then the cat sat on the mat and, and then we say, okay, the cat sat on the mat and what's next? And we do this over and over again.
So what's magical about this is that even though we are the models, I should say, are only doing this next token generation, there's something I call the pregnant present in the process of guessing that very next token, they're taking into consideration all of the past and also the likely future. It has to project this kind of trajectory. It's by it's like when you're let's say you're doing a dance or you're walking, you have to take the next step with the knowledge that that next step is going to lead to a next step.
and computationally this leads to this sort of miraculous ability for these models to plot out very long-range kinds of
It's just computing the function of what's the next cognitive token?
Given what's happened right up till this moment and the recent past, what should I do next? Then when you do that, that goes back into the loop and you proceed from there. And so this leads to, in some ways, a radically different view of what cognition really is. It, for example, leads to a completely different perspective of what memory is. Memory is not
We don't actually have somewhere stored in our brain events that took place or some fact that we've learned. What we actually have stored is just the capacity to autoregressively generate. If I ask you a question, what did you do last summer? And you say, well, I went to this place or you start imagining yourself jumping off of a cliff or something like that. What you're doing, according to this theory, is you're in real time, just in time,
Taking that question that prompt generating the very next output from that and then running from there what did i do last summer well i and then you continue to generate and so that really leads to a completely different perspective on what what memory is.
It's no longer about storing facts. It's about having a brain, just like a large language model, that has these potentialities for facts. I call them potentialities because they're infinite. What does a large language model know? How many questions can you ask it? You can ask it an infinite number of questions and it can give a reasonable response to all those because it does this autoregressively. It's not retrieving this fact or that fact. Rather, it has the capacity to generate all of these facts.
Finally, I'm probably over five minutes at this point, we can extend this even beyond language and linguistic responses, but more broadly to cognition. As I mentioned, thinking about a sequence, thinking about what am I going to do later on? I'm going to go hang out with my friends, maybe I'm going to have to get in my car, things like that. All of this, any thinking that takes place over time,
Well,
Fascinating. Obviously, Elon and I, we share a lot of these ideas in common. But one of the things that comes to mind right away, and we've discussed this before, is how surprisingly powerful this language model has been for doing what seems like non-language tasks.
So the example I like to think of is if you imagine building like a robot or even like some kind of spaceship and we go back a few decades, you would tend to think, all right, we're going to have this compute engine. We're going to have this kind of logic machine and that's going to churn through facts and it's going to deduce things and do inductions. And this is what AI in the 80s kind of looked like. And then maybe after you had that compute engine, you would then build a communication module that could talk kind of as like an add on.
that the ability to just communicate seems at the periphery compared to the real thinking stuff, right? But what we've seen out of these models, and I think the best example is their ability to computer program, all of these capabilities just fell out.
of this token prediction. We didn't have to build an algebra engine or something that understands trigonometry or even optics. And when you think about when it generates the images, it gets reflections right. It gets speckles and bubbles and all kinds of things. It's learned a lot about about physics. And you can do this with the language models as well. How is it that it's that all that thinking horsepower just fell out of what seemed like the walkie talkie, right? The communication module.
So in a more general sense, I've been thinking about is our brain a computer? Is it a computer-like object? Is it fundamentally a different category? Or is it like the thing that you have in your phone or your laptop? And this has been highly debated since Turing showed up on the scene. But I would argue that software, the idea of software, is the most important idea humans have come up with in maybe a thousand years.
Because it gives us a thinking tool, it gives us a token to understand a lot of these problems that previous philosophers and thinkers and cognitive scientists were just absolutely baffled by and for no fault of their own. Things like the redness of red or the pleasure and pain experience that we feel and we wonder why does the vacuum cleaner not have that and so on. And I think that the real nature of this is we've discovered that we're software.
And the more interesting thing that goes with that in the technological world in the past few decades, we've now found out that software is actually more interesting than we thought. Alan Turing and Von Neumann said, hey, why don't we just put this pattern into the machine and it'll do a complicated process.
But as we saw and heard with Levin, it's more subtle than that. You can set into motion something that the Lego blocks themselves have some sort of intelligence and maybe not at the level of the brain, but maybe like ants and that they're self-assembling. One of the big things for me was the earlier version of the neural networks that led to this transformer is one of the popular ones was called the LSTM, the long short-term memory. And this is from like 1991.
And what's fascinating about that is there's just this kind of this tape, this large vector, and it can do stuff. It's just a language. And if you put this language and you run this machine, which seems like it's just biting its own tail, it can do stuff. And I remember looking at that and thinking, well, that could be what DNA is. The DNA might not just be mapping to proteins, but I think a lot what Levin has now shown in the lab, that it's instantiating
a kind of a proto-intelligence or an intermediate intelligence or some non-human kind of problem-solving engine and that that is what is responsible for this miraculous thing that Turing was interested in our shape, right? This morphogenesis and how we get this pattern because it's not obvious that we have a genetic sequence and then jump to we've got hands and faces and really complicated structures. And I think we need to look at this intermediate software layer
which is now we're bare witness to this with these language models, that it's not a one-to-one, right? You're not really listening to my brain. My brain's not talking. Something else that lives in my brain is talking. As a side note, it feels very autoregressive because it's not, I don't know where I'm going to go with this, but as I say it, things come into mind. And something we talked about a couple of weeks ago, I think this applies to behavior in general.
Right think about when you you didn't know what you're gonna experience this event today but you knew by just showing up and being there interesting things would happen and so i think life in general it has this kind of autoregressive property where sometimes just getting out into the mix will give you all of these ideas and avenues and pathways that you didn't see before so that's that's the idea i'm excited about the most is thinking about virtual machines this is something that the
technological world you go to amazon they use what's called virtual machines and we can talk about that more but this these layers between the code and the thing that you're experiencing the app in the in the software world and technology on your laptop it's not one-to-one none of the programs you're using are talking to the chip
They're talking in some other language, C or Python, and then Python is talking to the chip with even more intermediates. And I think that we need to consider that our brain and our mind and our self, the ability to instantiate multiple selves, is because of this kind of virtualization. Yeah, and if I can just quickly, to me that is even more than sort of the realization of that this is an explanation of language, an explanation of the brain. It is this radical departure from thinking of
We thought of math for a long time as a method of description of the physical processes, but it's not about the symbols on the board. The symbols on the board in some ways were an early aperture into the fact that there's this informational life, that there's sort of computation. It's a phenomenon. In some ways, it's mind-independent.
And what we're seeing now is we're actually capturing some of these things truly coming to life in Silicon. By seeing them in Silicon, by seeing them take flight, so to speak, outside of their traditional medium, we have to step back and have almost a newfound respect for the informational itself as being maybe more fundamental. As Michael Levin mentioned, right, that our brain is channeling
What else lives there?
What do you mean, what else lives there? Well, what else lives in our minds, first of all? Running a business comes with a lot of what-ifs. But luckily, there's a simple answer to them. Shopify. It's the commerce platform behind millions of businesses, including Thrive Cosmetics and Momofuku. And it'll help you with everything you need. From website design and marketing, to boosting sales and expanding operations, Shopify can get the job done and make your dream a reality.
What other kinds of, and doesn't need to actually be an autoregressive process, but autogenerative processes, what other processes are running themselves at an informational level? I'm gesturing my hands now. I could tell a story about, well, I'm trying to express this or that idea, and I'm doing that with this gesture, and I've learned these kinds of things, that this is a good way to do that.
Or I could tell us a story where the informational sort of process is making that happen because it wants to generate the same way that language wants to generate. Language wants to express in some very strange but very real way that the trajectory that language takes, it belongs to language itself. Maybe when I'm moving my body in this way, there is a process that's causing that that isn't really captured at all.
by thinking in traditional terms of biomechanics or even in psychological or neurological terms, it may really be about long range trajectories through some sort of space. And so what are we, if not a collection perhaps of these almost alive informational systems? And then
If we potentially extend that outside of ourselves, what is what is an ecosystem? What is a society? What physics and more generally, there seems to be something very sticky about the past.
Just a moment. Don't go anywhere. Hey, I see you inching away.
Don't be like the economy, instead read the economist. I thought all the economist was was something that CEOs read to stay up to date on world trends. And that's true, but that's not only true. What I found more than useful for myself personally is their coverage of math, physics, philosophy, and AI, especially how something is perceived by other countries and how it may impact markets.
For instance the economist had an interview with some of the people behind deep seek the week deep seek was launched no one else had that another example is the economist has this fantastic article on the recent dark energy data which surpasses even scientific americans coverage in my opinion they also have the charts of everything like the chart version of this channel it's something which is a pleasure to scroll through and learn from.
Links to all of these will be in the description of course. Now the economist commitment to rigorous journalism means that you get a clear picture of the world's most significant developments. I am personally interested in the more scientific ones like this one on extending life via mitochondrial transplants which creates actually a new field of medicine.
Something that would make Michael Levin proud. The Economist also covers culture, finance and economics, business, international affairs, Britain, Europe, the Middle East, Africa, China, Asia, the Americas, and of course the USA. Whether it's the latest in scientific innovation or the shifting landscape of global politics, The Economist provides comprehensive coverage
And it goes far beyond just headlines. Look, if you're passionate about expanding your knowledge and gaining a new understanding, a deeper one of the forces that shape our world, then I highly recommend subscribing to The Economist. I subscribe to them and it's an investment into my, into your intellectual growth. It's one that you won't regret. As a listener of this podcast, you'll get a special 20% off discount. Now you can enjoy The Economist and all it has to offer.
Again, it's this respect for the richness of computation in a way that just changes everything.
But I would suggest that previous cultures or older civilizations were very aware that the world was sort of embedded with this intelligent substance and this sort of pervade, you know, everything. And then that was sort of responsible for the complexity we see in forests and animals and human beings and so on.
But just real quick, it reminds me of this idea of cellular automata that Wolfram really popularized and did all the great work in. That he kind of just showed that if you have graph paper and very simple rules, you can kind of see this ocean of this primordial soup kind of bubble up where you get non-trivial, very interesting behaviors out of what seems like as simple a system as you could possibly make it.
And I would argue that our brain might be like that. It's way more sophisticated than graph paper, but it has all the properties that graph paper has. And so if we show that these simple things like Game of Life or the Wolfram Automata can spin out these patterns and patterns that make other patterns and so on, maybe our brain or even our entire body or all biology is more like a substrate where these informational beings, I like this term, I came across actually some religious terminology, spontaneities.
Because that really captures this idea that like nobody put them there they just showed up but now they're off to the races and they're doing their own thing and I think that words Might be like this especially the early proto language were very much earworms They might have come from language of the birds I heard a beautiful mockingbird last night and if you imagine hearing that for millions of years That would do something Yvonne help me understand what's new about your model
Now, I don't mean, oh, Wittgenstein said this in 1952. I mean, you're saying, other than this fancy word, autoregression, you're saying that people are thinking in terms of a sequence, in terms of something that comes next. So when we're in the flow state, we're present and we're thinking, what are we going to do next? And when we're planning, even if we're planning for the future, we're thinking in terms of, well, what am I going to do one week from now? And then how does that affect what I'm going to do two days from now? And then one day from now, et cetera. It's always one step ahead.
So what's new here? Well, what's new is that we actually have an architecture and an example in which we can see that the capacity to do long-range thinking, the capacity to live, so to speak, in an informational space, linguistic space, can all be compressed
We didn't know that before. There was no example of anything like this that had this kind of informational richness that did stuff that was based on such a radically simple and elegant kind of formulation. What's new is we've got a theory of the brain, a theory of cognition generally that
People might have pointed to in some, you know, in fairly vague terms, but that we can now build. What's new actually, what's new is not something I developed, it's the existence of these large language models. The large language models lead to the new epiphany that this is in fact perhaps how cognition works. And so we have a novel framework and it leads to a very, very different way perspective of thinking about classical cognition. So if you want to know what's new, for example,
The classic view of memory is the storage retrieval kind of model. For example, if I ask you to repeat back a sequence of a couple of digits or letters, the canonical view is that there's this thing called short-term memory, which is this box that holds stuff for a certain amount of time and then it decays. According to the autoregressive model, there is, and let me just add, then there's this other thing called long-term memory. What's that? Well, it's another box.
That some stuff from short-term memory if you want to hold it for a longer time you put it in that box and that's like It's refrigerated so it lasts longer And so you've got these two boxes with stuff in them and then when you want to find some information What you do is you you have to have a retrieval process you go and you see well What is what's related to the what I'm talking about right now? What information where did I leave my keys or what did I do last summer? Let me go find the relevant memory and retrieve it
The autoregressive model completely obliterates, A, the distinction between short-term memory and long-term memory, and in fact, completely obviates the need for any sort of retrieval process at all.
So it is a completely different framework. Instead, what we're doing, according to this model, is we've got these stored weights. That's the large language model. It's just these billions of parameters. And those parameters, all they are, they instantiate a function. A function just, what does that function do? It says, here's a string of words, which you just turn into numbers. And then it says, what's the next one? That's our brain.
Okay, this is your brain on autoregression, right? It's just basically a function that says we're going to take in an input and we're going to produce an output. And then you've got the short-term memory is, what have I said?
Since you asked the question, here's the question, I'm producing an answer. What did Kurt say to me a little while ago that's causing me to generate this very next token and the next one? There's some residual activation, and the model is called context. In the LSTMs, it's this memory, which may or may not be the right way.
In the large language models, it's literally the context is like the whole thing, right? Whatever you've said until now, whatever the model has said in our brains, it's probably closer to the LSTMs. It's some sort of some sort of compressed version. But that is the whole flow. The whole flow is you've got a static set of weights, which is our brain and then our sort of dendritic weights. And then you've got this dynamic process of, OK, generate with those weights, generate one thing, tack it on.
to what's been generated before, and this may go a lot longer than short-term memory. It may not be 15 seconds as short-term memory dictates. The classic experiments where they ask you, can you retrieve, can you repeat this particular sequence? That's not something we ever do in regular life. We don't have to remember. What exactly did I say five seconds, 10 seconds ago?
There was a kind of error of looking where the light is in trying to build out these models. You can give people that task and they can retrieve that stuff, but that doesn't mean we've got memory whose job it is to retrieve. Instead, what I'm proposing is that we've got generation, and generation is guided by what's happened in the past.
That is guided by much, much more than the past 15 seconds. Again, it dissolves this arbitrary idea that there's a short-term memory box. All you've got is residual memory. That could reach back into time 15 seconds, a minute. We're having this conversation now. Will mentioned something when we were talking over there. That's still bouncing around. It's reaching in, again, the pregnant present, it's reaching in to what my current generation is. So it completely obliterates
I really like what you said just now at the end, but also in the beginning, that this is a theory of the brain rather than a theory of neurons or neuroscience.
And I would argue that some of these new models are some of the first real kind of connecting from both ends. We've known for a hundred years or so what neurons are all about, at least in the basic sense. And cognitive scientists and philosophers from the other direction have been trying to figure out these properties like memory and planning and logic and stuff like that.
But these modern things, they're made out of simulated neurons, right? At the bottom. They're very, very simple brain cells. They're not nearly as complicated as the ones we find in our own brain. But we can think of it as a model that has both scales at the same time. And what's to me so fascinating is not only do we not need a special box for short-term or long-term,
We don't need one for planning. We get that out of the box. Logic comes out of the box. Ability to write poems or something like that. It's all sort of latent in this just ability to predict the next token. We're just starting to see the emergence of an actual brain science and now what we need to do is go look at our neuroanatomy and see
Okay, is this what the hippocampus is doing? Is this what the amygdala is doing? Like, how are these cortical structures interacting with all these nodules? Because one thing, when we slice our brain open, it's not a homogenous mass of neurons. And evolution could have done it that way relatively easily. So these structures must have some utility. I have some theories on that we can come back to. Yeah, remind me. But a point which I know you'll certainly agree with. We shouldn't take the terminology of neural networks that seriously in the sense that
And it comes back to an earlier point we made. The brain is instantiating some sort of function here. And I really mean it when I say that at a high level that the brain is doing the same things as the LLMs. Now, does that mean that I think that we're talking about neural circuits doing the exact same thing as the computer circuits? Do I even mean that they're transformer models that are doing self-attention? Not necessarily. It's really about at this high computational level.
And, you know, matrix multiply, which is really the core engine of Transformers. You didn't really need the brain as
It's nice to call it neural networks because it sort of, well, it rhymes with it. You can do it as these connected nodes. You could do it as a graph, but you could do it as a matrix multiply. And a mathematician will look at you funny if you say this has anything to do with the brain. And that really gets to this point of substrate agnosticity. And it's not even on substrate. It's not even about whether it's made out of biological matter or it's made out of silicon.
It's really about thinking about these things, not at the architectural level, but thinking really at the functional level. And the functionalists, you know, there was a functionalist school once upon a time in psychology, but they didn't have the breadth or the depth of computational resources to really explore things this way. But I think we're becoming functionalists once again. Just real quick.
I want to get to consciousness and language, but there are two points of confusion which I think may be related. So if language just refers to itself,
Then let's rewind the clock 10,000 years when our vocabulary was smaller. And let's just say we had the words fight and mouth and left. How do those three tokens become 3000 explicated tokens? So that's one question. Then another one is, well, what about indexicles? Like pass me that. Meet me here. I'm going there. Okay. What tethers that to the world? And even worse is
There's these sort of conjunction words that it's not exactly clear what they're doing and you're touching on a very difficult problem. So the origins of language have always been fascinating and really one of the greatest challenges in cognitive science. The problem got worse, not better in some ways, by virtue of what we've discovered because this, as you're pointing out, right, the claim here is that language is a sort of
Perceptual independent, it's its own kind of organism. If we look at animal signaling, and there's very strong, you know, many decades of research, animals don't seem to do language. I think we have a better idea of what we mean by that now. They don't do autoregressive next token generation. They have nothing remotely like this. What they do have is this kind of concrete signaling that, you know, there's this kind of situation arises in the environment. You make this sound.
They don't have is, they don't have the, they don't have any of these things, they don't have anything like the true generative kind of system that we do. The fundamental question here is how on earth did we as organisms make this huge qualitative computational leap from one kind of system to a completely different kind of system where those words the and is are really doing, we're only now going to understand, we're only going to start to understand them
Within this kind of framework they don't show up in animal language they don't show up any sort of classic semiotics where it's like that symbolizes that they don't symbolize anything in the environment.
But they're very powerful in the autoregressive process. They are doing things in that framework. And so I think there's an incredibly deep mystery here as to how language sort of jumped from what was presumably a purely kind of correlational association system of
this environmental stimulus and I'm going to make this sound or I'm going to release this pheromone or whatever it is that animal communication consists of to a stimulus independent computationally autonomous system that of course interacts with that other thing but is in many ways divorced and autonomous from it.
And so I don't have an answer to that question by any means, but I think we have a refinement of the question. Where did language come from? It's now not just a question of how did we develop such a sophisticated signaling system? How do we have so many words for so many things and we're able to put them together in novel ways to express new things? It's how did we develop this autonomous, generative system? It's a very different question than even sort of the Chomsky question of origins of language. You're not going to get it from a simple genetic
Kind of a mutation or something like that that allows for more higher expressive communication abilities. It's not even communication. It's auto generation. And so it's not an answer to the question, but it is a respect for the fact that the question is extremely deep. I don't know where language came from. I have very strange thoughts about it now is similar to the conversation we had with with Michael Levin.
There are patterns in the universe that have found their ways into our brains that are much, much richer than we could ever have imagined. They're just spectacular. They are far greater than anything. If you take the greatest scientists and the greatest engineers and lock them in a room for thousands of years, no one could have come up with anything remotely like language. Maybe they could have come up with
a brilliant communication system based on the standard symbol, you know, representation. But nobody could have come up with this. Where did it come from? So it makes me think of a few things. One, this idea that at one point there were like five words. And that's I don't think it would have ever been like that in a sense, because I think we have to remember that there's this analog and digital component. I can say hello or hello.
Very different. I said the same token, but I had this analog kind of background layer to it. And so I think early on we need to think how that emerged first and then that how that morphed into the thing where we have strict things. But I came across two things recently. One, it was talking about Shakespeare and his signature and they have like maybe 30 or 40 versions of his signature and they're different. He never signed his name the same time. And I don't. So why do we even say that's a signature?
Another interesting thing I came across from a few hundred years ago, they found a letter and the letter was talking about rabbits. This one particular letter used the word rabbit I think at least two dozen times in one document. It was spelled differently every single time.
How? I'm a bad speller, but at least I'll just make up something and then maybe be consistent. So the idea that words even have a particular spelling, that that's what we mean by that word, I think is a relatively modern concept, which doesn't really make any sense to me. And another thing that comes to mind is I would argue that language, as we know it now, this autoregressive, very sophisticated kind of software thing, probably didn't evolve
out of like even small groups like tribal groups or villages I think this would have emerged at the the city state and the the empire level right ancient Sumer things like that where you had thousands of people working in in cooperation because then you get this kind of externalized memory where the words are floating around right you hear words that no one you know even knows that word like oh that's a new word to all of us for example and so in like kind of a city environment you would get that
That language is something else. Are you thinking of a particular city, perhaps? Yeah, maybe, right? And then just also the other thing that comes to mind, I really believe in this kind of apprenticeship from the animal kingdom.
I really think the language of the birds and animal calls, and we tend as modern humans to think, oh, well, they're just making noises and they don't actually mean anything. And I haven't done this, but I had the thought of like, try it, you know, try to learn a bunch of animal calls, try to learn a bunch of of bird songs. And I would, I would think this is going to fundamentally change your brain and it might start to make sense.
So you think we're not giving nearly enough credit to animal communications. We have a very narrow straw that we're sipping through. We downloaded that first and that was kind of like Windows 1.0, which no one ever saw. And then later we get the upgrade and we're like, oh, that's that. We were animals too and we were also grunting or... And we still do. Right. We still do. So this brings up another idea of bootstrapping, but I want to come back to that.
This episode is brought to you by State Farm. Listening to this podcast? Smart move. Being financially savvy? Smart move. Another smart move? Having State Farm help you create a competitive price when you choose to bundle home and auto. Bundling. Just another way to save with a personal price plan. Like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts and savings, and eligibility vary by state.
so hello and then hello actually can be tokenized differently if you put an exclamation point in front of hello that's a great point yeah the token or i think that the next generation of models are going to just be listening to the audio of youtube and listening to the radio and stuff like that watching archive television and then they will get kind of both the analog and the digital i mean systems at the same time voice models i don't know people are using those uh i used um
You know from GPT, I started using one of the voices and my wife said don't use that anymore. It sounds a little too good and it really gets rosy Really good really well and I think that depends if it's a woman's voice. Yes, indeed Where are you listening to that exactly huddled away? Like you're spending a little bit too much time with GPT. Why is the door closed? What's your next token
So the prosody that's in these things is actually extraordinary and the prosody is just like sort of the sing-song version of it and I'm confident. I've looked into it, but they don't share. OpenAI doesn't tell you anything. OpenAI, that was a joke from the beginning. I think they almost meant it as a joke. And so we don't really know how they're doing, but I'm going to guess it's using the exact same... If I would argue that it falls out. It falls out, but it falls out
In conjunction with meaning of language itself and so prosody has its own meaning and probably preceded the kind that we call words.
And so words probably bubbled up as a more strict version of prosody or something like that, right? You maybe began as closer to something like music or something like that. It reminds me of what Levin mentioned with this idea of sharing stressors. And then this kind of makes a community. Imagine we're all trying to predict the prosody, not just the token, but like the emotional valence that goes with it. And by doing that, we might be bootstrapping some sort of shared experience.
Let's make the analogy, forget about neurons. We're here at the Behan Center, so computer science center of U of T and math. Let's think in terms of matrix multiplication. The multiplying of a matrix for language. Is that a necessary or sufficient or neither condition for consciousness? Well, I don't think language has consciousness, so I am going to
go out on a kind of metaphysical limb here and say that the symbolic is kind of a different, it's a new physics is one way I think about it. Sorry, and symbolic means? Well, symbolic actually means that the representation
of any, let's say a word. Let's actually talk about matrices multiplied for a second. So what happens in these models is you take, here's a big corpus, and I want to learn the relationship between words in the corpus. So what I do is, first I tokenize it. I chop it up into different pieces like the word word is going to get a certain numerical representation. I'm going to say 00111 or whatever there. And then some other word or maybe even just a part of a word like a suffix or something is going to have a different representation. I'm going to turn that into
a vector, which is just a string of numbers.
In the course of learning, what I'm going to do is I'm going to learn what to multiply that vector by, such that it's going to have a higher dimensional representation. It's a much longer vector. And so we've got this big giant vector representations, and we can think of it as a very high dimensional space. Or if you don't like to think about high dimensional spaces, because nobody really does, because you can't really think about them, you can think about it as a three dimensional space. And each word is just a location in that space.
What language is in this environment is that this word has its meaning by virtue of its not actual representation. The point where it is in the space is arbitrary. Where it is in the space is arbitrary. What gives it its meaning is where it is in relation to the other words.
Where ever, you know, if we have the word red and blue, they're going to be in some relation to one another. That is what the models are using in order to do the next token prediction. Now when I say the symbolic, what I mean is that vectorization itself is an arbitrary kind of representation.
There is nothing in that symbol. If you were to look at it, you could study that symbol all day long and you could study, you know, try to make sense of the zeros and ones, whatever, whatever you're, you know, however you're actually, you know, representing the space. You're not going to find the meaning in there. And I would say that same thing very much holds true for natural language. The word red doesn't mean red.
Not only in my autoregressive sense in that I don't just mean that I mean the word right itself doesn't point to the qualities at all and It's just letters. It's squiggles. It has nothing to do with it Now that is very different and this is the turn where I'm gonna go to consciousness That is very very different from the kind of the neural representation of redness itself redness in my brain the kind that I think is rise to what we call conscious experience and
is red and blue actually really have true mathematical relations to one another in the representational space. There's something happening between the wavelengths and the sort of adoption release that's happening in my retina that scales in a particular way across these different cones.
such that red and blue really mean something in relation to another, not just by virtue of their embedding in this arbitrary space, but by virtue of their actual properties. And so the symbolic, I believe, doesn't give rise to consciousness because it doesn't in any way instantiate these properties of the physical universe. Consciousness, phenomenal consciousness, all of it,
is sensory at its base. You can't imagine another kind of consciousness besides something from the physical universe impinging on your sensory system. It could be visual, it could be auditory, it could be tactile, it could be your own body, your proprioceptive, it could be a sense of where you are, of your movement through space. All of these things are ultimately a continuation
Not a representation, but a continuation of the physical universe. Those patterns that are in the physical universe are simply rippling through our nervous system. Language breaks that. Language turns things into arbitrary representations, symbols. And I think that symbols do not have the quality, they don't have the quality that sensation and perception do. And so I don't think that the matrix multiply, I think the matrix multiply in the case of language, we lose that.
I think that it actually is pushed off into a different metaphysical space, so to speak, where there is no consciousness. I don't think large language models are conscious. I don't think there's any possibility of them being conscious. They cannot understand what redness is because they have no access to the analog space where these kinds of relations exist.
When I think of consciousness now, it reminds me of this virtual machine idea that we were trying to find consciousness in the brain and say, where in this, you know, bucket of meat do we get this magical experience? And I think that we need to consider that we're instantiating this other thing or maybe multiple layers of it. And I think that's where this, I don't mean to say it's not interesting, but that's where this thing we call consciousness or maybe even self
lives and I think we see this with the cases of multiple personality syndrome and disorders where people are kind of obviously instantiating more than one seemingly conscious entity in a single brain and so to me that suggests there's not a one-to-one mapping it's more like you have a phone you have apps on that phone and when we think about consciousness where we are one of those apps I think it was installed in us by our parents
I don't think it would just naturally emerge. And I think we're only now just discovering kind of the large distribution of consciousnesses on the planet. One thing I'd love to pick your brain about is sort of things like aphantasia or inner monologue or not inner monologue. And what I think is really fascinating is
These kind of emerged as interesting topics of conversation relatively recently with the emergence of internet forums. Because as you might know, psychology experiments are very expensive and usually the number of participants is like 10 or 20 or something like that. But in the last decade or so, we see now 50,000 or 100,000 people communicating on one forum. And they'll ask a question of like, when you close your eyes and you think of an apple, what do you see?
And half of the respondents will say, oh, I can see a red shiny apple. I can see it on the tree and the light shining on it. Maybe there's a little green leaf. And the other half, it's very kind of evenly split. Say, I don't see anything at all. What do you mean that you see something?
And the same conversation happens with inner monologue. Do you talk to yourself throughout the day? And again, you get this conversation split where half the people are like, yeah, of course I plan my day and talk to myself and think. And the other half says, you hear voices? It's just very, very strange. And I've met very high functioning people that don't have visual imagination. That doesn't make any sense to me. I know, I know a fellow who says he has neither. And I said, well, what do you do when you're thinking? And he says, what do you mean? I just think.
How does that work in your model? I don't know how it works at all but it's a very fair question. Is he a counterexample to your model? All it takes is one counterexample to disprove a theory in math at least. It won't disprove it at all because here's the thing. There's the doing it and then there's the ability to observe yourself doing it.
And so it may very well be that people are running some sort of process, a very similar process, but aren't, don't have this extra meta awareness of the fact that they're doing it. It may very well be there are angels and watermelons around us. But the truth is, it's actually not, it's not essential to any of this theory, right? The language models themselves don't hear themselves think.
You don't really need that. That's sort of a weird feature of our brains. Something Will and I have talked about quite a bit is that not only do we produce sort of the flow of thought, but there's somebody watching at the same time and that's observing it. That's not necessarily a given.
It is phenomenologically true we will see people report that they have it i don't know how essential it is to being else and i think i don't know i'm not up on and i have to because people ask me this. Quite often i'm not up on the current research like do we see an event if it is or not in a monologue folks like when they have to think about something.
is there time is the time that they take to think about it radically different from the time that it takes people or the class of problems that they can solve because it doesn't seem to be exactly it seems to be the same set of problems so the fact that they're so behaviorally similar
Means either that it's sort of epiphenomenal the fact that we hear the inner monologue or that i'm dead wrong and then you know that that it's just it's it's that's an illusion that inner monologue isn't even really part of the way we think i don't know but it's it's only a very interesting question but it is i would push back and say that. It's not entirely clear why we have to hear our own thoughts at all even if even if they're being generated reminds me of the idea that language might be a double edged sword.
That it might be sort of the best and worst thing that ever happened to humanity. I often refer to it as a divine parasite, in that it's a good thing, but it sort of tickovers. In that, if you know language, right, I can easily just mind control you. I think we're going to see some amazing examples of that, some real examples of that later. But I could just say something like, you're now thinking about your breathing.
And you're going to be like, oh, I don't want to pay attention to that. But I put that in your head. And the fact that you're an English listener, you might be thinking that now. Or the famous one, you know, don't think of pink elephants, and it's like too late. So there's this idea, you know, going back to the Tower of Babel, that there's sort of this danger in having a shared language. And maybe this distribution of inner monologue and not having it is a way of kind of curbing that problem. And then this is where, you know, it starts to get a little bit disturbing.
If you think about language not as being even a communication system but an operating system and what language is meant to do is to make people do things and it is this shared cultural artifact
It's downloaded against your will. You don't sign a waiver before you sign up. It's too late. By the time you're reading the waiver, it's too late. You don't get to click permission to write. Exactly. Every baby, before they're born, is already being conditioned and language is being downloaded into them. Now we know, in a way we didn't know before, that it is this kind of its own self-running system.
It's a product of society, not of any individual, and it's a product that society didn't design except in an emergent way. People didn't, you know, we don't think, I don't think neither of us thinks, either of us thinks that, you know, a bunch of, you know, very clever people got together and said, how are we going to control the populace? We need to design this system that's going to run in them. It's far stranger than that, but yet at the same time,
It isn't really of us individually, it is society. And so things like beliefs, you know, society itself is built up of all these notions that are language dependent.
And they're running in us and they make us go you know you get up in the morning cuz you gotta go to your job because you gotta make money because you got you know you have to have certain degree of of prestige in the world and and you're gonna have to you know one day you're gonna you're gonna get older and all of these things animals don't know any of these things right who put these ideas in our minds well forget about who but how and the answer is language and so there is this
Thing that's in us but not really of us that is running and it's driving most of our behavior those things we do who's who's familiar with this idea of jail breaking. A language model right so there's things they won't do i don't want to do but if you're very clever about how you ask for you can get the kind of leak out that information.
Are we subject to that same kind of vulnerability? And I would suspect and suggest that we are and that we need to be aware of this. This kind of prompt engineering or prompt hacking is probably something that's been happening since the beginning of civilization, but we didn't have any kind of meta awareness. And I think the most interesting thing in development about these AI models in general is how they're going to affect our mind, how they're going to change how we think about our language, our words, our cognition, our place in the universe.
They're giving us this, it's not quite a magnifying glass. I call it the macroscope because it's showing us the big picture all at once and it might be a terrifying view that it's opening up. Even more potentially disturbingly, we now have, now that we've captured not just language but in some ways culture and thought in a system that we can manipulate and test.
Doesn't that potentially give people, and maybe even people with ill intent, the opportunity to see what's effective in changing minds, what's effective in manipulating people's thoughts? That's a really great point you bring up because now that you have a sort of a simulated person in a jar,
You can test it. You can sort of A-B testing, right? Will they respond to this? Will they respond to that? And kind of simulate the attack a million times and then go test that sentence out. This is what scammers do. When they finally get people on the call, they have a script and that script is very like honed in. They've evolved that script over years potentially and the ones that get the scam, that determines the script that survives. So in that same kind of thing,
are we susceptible
You know, it's, as you said, it took years to develop and you know, the car salesman and all of that. These are, these have been honed over, you know, many, many generations. Now you can do it at high speed. Now we can do it at high speed, you know, in silicon, like boom, run the, run the experiment and you've just, you've just tested out a hundred thousand customers under these varying different conditions. Is this a good thing? Or is this information hazard? Is God just a token? Quite.
When you say just a token, so let's have respect for these tokens, right? Because what they really are are these, you know, almost like informational angels or something that do all this stuff within the larger context, not only in language, but in our behavior, in the ecosystem, the larger social ecosystem. What is God in that system? And the answer, in my case, it would be, well, maybe it doesn't point the same way that red doesn't really point to that cup.
But what does it point to? It's not pointing in the traditional sense, but it's doing stuff. If we think of an operating system, what does God do in our operating system? What does the token God do in our operating system? And then you have a completely different perspective. It's not about an ontology of like, does it point to some divine being, whatever that would mean. And, you know, when you try to define God, it all becomes, it falls apart in your fingers.
But what does God do in the minds of people? Well, I think we have actually some good ideas about what that idea does. Is maybe that idea itself almost synonymous with what people mean by it? Now, that's where things get really interesting. You can imagine a God so powerful, he need not actually exist. Exactly.
And something we talk about a lot, it's, you know, my thoughts are not your thoughts kind of a thing. And another joke that we follow up with, it's the thoughts that do the thinking. So if I were to phrase it, is God just a story? And if so, would that make him any less powerful? Right. And I think what you're suggesting is it's real. And I would say it's real like software.
Right. Facebook's just software. It's not real. There's no actual physical entity called Facebook. I don't mean the headquarters. I mean, like the thing that we log into or whatever. But it's real. I don't think anyone would argue that things like large software platforms or even markets or societies, you can't pin them down. Like, what is Toronto? Is it that thing? Right? No, it's somehow weirder than that, but doesn't make it any less real, right? Well, there's a difference here between God as
One traditionally conceptualizes it, whatever that means, and then God as just a story that has effects on people. Now if the broader sweep of people believed God was just a story that happened to have effects, that would also have its own effect separate from the traditional conceptualization. So wouldn't the first and second be the same thing?
So I call this third order religious, right? First order religious is just sort of, it's just true, it's real, it's out there. You don't really question or worry about it. Second order would be, well, it has utility. It helps a lot of people. It's a very powerful, you know, collective force. And I would argue what I call the third order is to realize those first two are the same thing.
It doesn't make it any less powerful or majestic or divine or inspirational or transcendent, maybe even more so because we now see how it sort of bootstraps into reality. Yeah, one way to think about this is getting outside just of sort of the very limited space of thinking about physical reality as impinging on our senses and causing these behaviors, but that the universe broadly is kind of reflected in these patterns that are taking place in our brain.
And then you just have to do a quick little switcheroo. If we call the physical universe, and maybe the physical universe is an instantiation of some sort of platonic informational space, well, let's just call that God. And then what has happened here is that that has actually molded this particular system that's talking about itself in some very, very real sense. Or constructing itself, even. Or constructing itself.
Okay, now we're in a room full of students of all ages, some researchers, some academics, and so on. What lesson do you wish you had when you were younger that can apply to them other than follow your passion? Be very, very suspicious of scientific orthodoxies. I think the process of growing up to some extent is realizing that adults have no idea what the hell they're doing.
The same is true not only for the regular adults like the ones in your life, your parents who you kind of figure that out pretty early, but it's true of sort of the greatest scientists. I think we're seeing that now in a very deep way that there are certainties that I think we held from a societal standpoint that are getting very shaky.
I personally see it as a ripping open of the foundation of what we thought knowledge was, for example. There was a certain kind of core basic findings. Well, language, people didn't even think about, really deeply think about what language is. At least they certainly didn't come to the conclusion that it's this thing that's doing this stuff. And that means that all the pious certainties of the scientific establishment
And we're in a place of science. I'm not saying, by the way, judge science more harshly than other kinds of belief systems. What I am saying is judge all belief systems with a great deal of skepticism because we are barely, barely scratching the surface of what we can actually know. We are, as much as we do know as a species and it's extraordinary, we can build incredible structures and we have figured out quite a bit about the physical universe, much more than our ancestors knew, much more than we knew 50 years ago.
relative to what's unknown, we're just getting a glimpse of that. And so I'd say go into the world with a radically open mind, but at the same time also recognize that there are systematic ways of thinking that have led us to this point. The engineering of large language models got us to the point where we can now say, wait a second, what is all this stuff? And so there is
Beautiful. I think first I'd say have the courage to believe in your own ideas. All of the stuff we're talking about now, if we go back a decade or so, it would have been almost impossible to listen to it.
We were inside of this AI winter and all of these interesting ideas were just, it was hard to get them out. And I would suggest that some of the things that you are thinking about now and you are working about and you're excited about, you're probably keeping them like close in your pocket hidden because you think, oh, the world's not ready for that idea yet.
Be bold and put it out there because you don't want to be the second person to put the idea out there. You want to say, no, I was thinking about that a while back. The other thing I would say is tolerate ambiguity and be wary of strong opinions. If you find yourself having a very strong opinion or you're talking to someone and they instantly kind of activate this very strong reaction, question that.
Question why you have this immediate and ask yourself, am I really thinking about that or did I have the answer ready to go? And if you find that you or other people have the answer ready to go, then where did that answer come from?
If you cook up that answer and you thought about it for a long time that's fine but just question question your beliefs and be aware that our mind tries to prevent us in society tries to prevent us from thinking these unthinkable thoughts and the thoughts that we're sharing today i would argue that were nearly unthinkable even a decade ago.
So we're going to open this up to audience questions and it will be exclusive to Substack so search the Substack which is on screen right now. Thank you for watching and you have a Substack as well and you have a Substack as well, correct? I have. I haven't put my material on it but I have a stuff that I'm going to be posting. We'll place those links on screen. Thank you.
Okay, we're now at the end of the Q&A with the audience. And if you liked the regular interview, this far surpassed it. The audience grilled these two. So again, that's Substack. You can search my name, Kurt Jymungle, and you'll find it. Thank you to everyone. Thank you to Elon. Thank you to Will. Thank you to Addie for putting this on. Hi there.
Kurt here. If you'd like more content from Theories of Everything and the very best listening experience, then be sure to check out my sub stack at kurtjymungle.org. Some of the top perks are that every week you get brand new episodes ahead of time. You also get bonus written content exclusively for our members. That's C-U-R-T.
j-a-i-m-u-n-g-a-l dot org. You can also just search my name and the word sub stack on Google. Since I started that sub stack, it somehow already became number two in the science category. Now, sub stack for those who are unfamiliar is like a newsletter, one that's beautifully formatted. There's zero spam. This is the best place to follow the content of this channel that isn't anywhere else. It's not on YouTube. It's not on Patreon.
It's exclusive to the Substack. It's free. There are ways for you to support me on Substack if you want, and you'll get special bonuses if you do. Several people ask me like, hey, Kurt, you've spoken to so many people in the field of theoretical physics, of philosophy, of consciousness. What are your thoughts, man? Well, while I remain impartial in interviews, this Substack is a way to peer into my present deliberations on these topics.
And it's the perfect way to support me directly. KurtJaymungle.org or search KurtJaymungle sub stack on Google. Oh, and I've received several messages, emails and comments from professors and researchers saying that they recommend theories of everything to their students. That's fantastic. If you're a professor or a lecturer or what have you, and there's a particular standout episode that students can benefit from or your friends,
Please do share. And of course, a huge thank you to our advertising sponsor, The Economist. Visit Economist.com slash Toe, T-O-E to get a massive discount on their annual subscription. I subscribe to The Economist and you'll love it as well. Toe is actually the only podcast that they currently partner with. So it's a huge honor for me. And for you, you're getting an exclusive discount. That's Economist.com slash Toe, T-O-E.
And finally, you should know this podcast is on iTunes, it's on Spotify, it's on all the audio platforms. All you have to do is type in theories of everything and you'll find it. I know my last name is complicated, so maybe you don't want to type in Jymungle, but you can type in theories of everything and you'll find it. Personally, I gain from rewatching lectures and podcasts.
I also read in the comment that toe listeners also gain from replaying, so how about instead you re-listen on one of those platforms like iTunes, Spotify, Google Podcasts, whatever podcast catcher you use. I'm there with you. Thank you for listening.
▶ View Full JSON Data (Word-Level Timestamps)
{
"source": "transcribe.metaboat.io",
"workspace_id": "AXs1igz",
"job_seq": 1895,
"audio_duration_seconds": 4349.21,
"completed_at": "2025-11-30T21:14:02Z",
"segments": [
{
"end_time": 20.896,
"index": 0,
"start_time": 0.009,
"text": " The Economist covers math, physics, philosophy, and AI in a manner that shows how different countries perceive developments and how they impact markets. They recently published a piece on China's new neutrino detector. They cover extending life via mitochondrial transplants, creating an entirely new field of medicine. But it's also not just science they analyze."
},
{
"end_time": 36.067,
"index": 1,
"start_time": 20.896,
"text": " Culture, they analyze finance, economics, business, international affairs across every region. I'm particularly liking their new insider feature. It was just launched this month. It gives you, it gives me, a front row access to The Economist's internal editorial debates."
},
{
"end_time": 64.514,
"index": 2,
"start_time": 36.34,
"text": " Where senior editors argue through the news with world leaders and policy makers in twice weekly long format shows. Basically an extremely high quality podcast. Whether it's scientific innovation or shifting global politics, The Economist provides comprehensive coverage beyond headlines. As a total listener, you get a special discount. Head over to economist.com slash TOE to subscribe. That's economist.com slash TOE for your discount."
},
{
"end_time": 92.227,
"index": 3,
"start_time": 66.323,
"text": " And this is where, you know, it starts to get a little bit disturbing. Language is an autonomous informational system. One may even call it an organism. And it runs in our brains. It's downloaded against your will. By the time you're reading the waiver, it's too late. What makes you think your thoughts are your own? According to groundbreaking work by professors, Elon Barinholtz and William Hahn, every word you speak"
},
{
"end_time": 102.073,
"index": 4,
"start_time": 92.466,
"text": " comes from an alien intelligence that installed itself in your brain during infancy. Language, they argue, is an organism that uses human minds as its substrates."
},
{
"end_time": 129.684,
"index": 5,
"start_time": 102.466,
"text": " Elon Bernholtz was interviewed before on this channel and it went viral. Link is in the description. Professor Bernholtz contends that our linguistic system operates on meaningless squiggles with no access to the rich sensory world of experience. He believes that LLMs can master language but never feel pain, for instance. Professor William Hahn, on the other hand, who's also been interviewed and it went viral, link is in the description, extends this to virtual machines and software layers."
},
{
"end_time": 156.544,
"index": 6,
"start_time": 129.684,
"text": " Professor Han thinks that consciousness is just one of many programs running in parallel. I was invited to moderate this panel live at the University of Toronto with this event organized by Echo Lopto. Again, links to everything are in the description. Today, I want to thank everybody who took time out of their schedule and their busy lives doing the many things that people do here in the University of Toronto to come for this extremely special live"
},
{
"end_time": 182.125,
"index": 7,
"start_time": 156.903,
"text": " in-person podcast with the Kurt Jaimungal. Give it up for Kurt. He runs the biggest science and philosophy podcast in the world. It's called Theories of Everything. Looked it up on YouTube, Twitter slash X, Instagram, whatever they call it these days. Kurt Jaimungal, Theories of Everything. Very easy to say. And then we're also joined by Professor William Hahn, who came all the way from Florida. Give it up for him."
},
{
"end_time": 201.391,
"index": 8,
"start_time": 185.299,
"text": " I'm extremely fortunate to be in the presence of these brilliant people today and the incredible work Kurt has done with this channel and his podcast over the last couple years is"
},
{
"end_time": 222.176,
"index": 9,
"start_time": 201.903,
"text": " Just nothing short of amazing. And I encourage everybody, if you want to learn about philosophy and science, technology, AI, consciousness, psychedelics like Andreas' work, maybe even hypnosis, all sorts of crazy stuff, I highly recommend you subscribe to this channel. So once again, that's theories of everything. And I think without further ado, we should get started, right? Let's rock and roll."
},
{
"end_time": 251.852,
"index": 10,
"start_time": 222.739,
"text": " Okay, welcome everyone. I'd like to get started with five minutes or so on your theory of everything the way that you see the world and then we'll see how Will sees the world and then we're going to compare and contrast it. Okay, thanks. So my theory of everything is sort of a meta theory because it's a theory of theories. It's a theory of thinking and it has a couple of parts to it but I'll cram it all into five minutes. The first one is about the nature of language."
},
{
"end_time": 280.265,
"index": 11,
"start_time": 252.415,
"text": " Everybody's familiar now with the large language models, things like chat, GPT, Gemini, Claude, whichever your favorite one, they all have the same basic engine. What all of these models have done is learn how to predict a word from the sequence of words that came before. And in the process of building these very sophisticated machines that are made of lots and lots of these billions of parameters to learn this predictive structure,"
},
{
"end_time": 309.275,
"index": 12,
"start_time": 280.845,
"text": " What we've actually done is we've discovered a property of language itself. Language is autogenerative. That's the term I've been using, which means that it contains within its structure. The corpus of language itself contains the structure needed to generate itself. What these models are doing are just learning the predictive structure that's already in language. Language is an autonomous informational system. One may even call it an organism."
},
{
"end_time": 335.179,
"index": 13,
"start_time": 310.196,
"text": " And it runs in our brains. It doesn't actually have access to the other stuff going on in our brains. When we are talking about objects with features and colors, there's a sensory apparatus in our brain that is processing that information. Our linguistic system doesn't know about that. Our linguistic system is what we can call ungrounded. It only knows about symbols. I call them meaningless squiggles."
},
{
"end_time": 365.52,
"index": 14,
"start_time": 336.032,
"text": " It knows the relationship between these meaningless squiggles. It knows that the word red tends to go together with the word apple and the word blue. And it knows that they're in a certain relation to one another. But it doesn't know in any deep sense or any sense at all that these actually extend outside of the world of language itself. So the first observation here is that language is its own system. And what's true in silicon and what's true in these large language models, my claim is, is true in"
},
{
"end_time": 371.203,
"index": 15,
"start_time": 365.64,
"text": " I have all kinds of things going on inside my brain."
},
{
"end_time": 401.271,
"index": 16,
"start_time": 371.493,
"text": " I'm thinking about visual imagery. I have a sense of my corporality, my body. But there's a language model, an LLM, so to speak, that's running and speaking these words. It's doing it in conjunction with these other systems. It's receiving messages from them. But what it's really doing is just its own predictive engine. And this is a radical departure from the way we have thought about language until now. And when I say we, I mean the human species. There is something deeply intuitive or deeply unintuitive about this. There's a deep intuition that our language"
},
{
"end_time": 431.596,
"index": 17,
"start_time": 401.613,
"text": " is completely embedded with everything else that when we are talking, we are talking about things outside of language. So this is a very radical thesis that language has this property and that it, in fact, what is happening inside of us is in some ways is an autonomous informational system that isn't actually the same one as the rest of our cognitive architecture. One way to think about this is that culture itself has been planted within us. This informational system that's running"
},
{
"end_time": 458.592,
"index": 18,
"start_time": 432.005,
"text": " And for better or for worse, it is driving a lot of our behavior, but it isn't quite the same thing as the thing that feels, the thing that cares in some ways. So that's part one of the thesis, which you might think, well, that's enough. And indeed, I think that leads to a dramatically different, almost existential view of certainly of knowledge and its place in our lives."
},
{
"end_time": 486.288,
"index": 19,
"start_time": 459.053,
"text": " But there's another piece with this, because not only is language autogenerative, there's something special about the way that the models do this generation. So the models do this thing called autoregression, which is a word you'll hear me say a lot if you hang out with me. And autoregression is this particular way of doing prediction, where what you do is you take a sequence, let's say the sequence, the cat sat on the, you can guess the next word, maybe it's Matt. That's a popular ending, but there's other possible endings to that."
},
{
"end_time": 502.363,
"index": 20,
"start_time": 486.681,
"text": " So you might say you're predicting what that next word is. What you can do, what we do in autoregression is you take a sequence and then you produce the next most likely token or word, we can just call it a word, and then you feed that back into the system and it's recursive."
},
{
"end_time": 530.401,
"index": 21,
"start_time": 502.363,
"text": " So let's say it's the cats on the mat and then what well, I mean, you know, I don't know and then it got up and it, you know, it walked around and rubbed against my leg or something like that. This is all a reasonable sequence that could come after the cat sat on the. Now what these models are doing and this is a really important and astonishing point is they are not thinking directly about the rest of that sequence. They are only ever doing the very next"
},
{
"end_time": 559.326,
"index": 22,
"start_time": 530.998,
"text": " We're the very next token, as it's called. My thesis is that we are doing the very same thing. What we are doing in our minds, in our brains, is the same kind of computation. We are simply guessing the next token, and then we're taking that token and we hear it. It goes back into our own loop, and then we're saying, well, now there's a sequence with another word. The cat sat on the mat, and then what's next? And we say, okay, then the cat sat on the mat and, and then we say, okay, the cat sat on the mat and what's next? And we do this over and over again."
},
{
"end_time": 589.326,
"index": 23,
"start_time": 559.667,
"text": " So what's magical about this is that even though we are the models, I should say, are only doing this next token generation, there's something I call the pregnant present in the process of guessing that very next token, they're taking into consideration all of the past and also the likely future. It has to project this kind of trajectory. It's by it's like when you're let's say you're doing a dance or you're walking, you have to take the next step with the knowledge that that next step is going to lead to a next step."
},
{
"end_time": 597.193,
"index": 24,
"start_time": 589.838,
"text": " and computationally this leads to this sort of miraculous ability for these models to plot out very long-range kinds of"
},
{
"end_time": 625.691,
"index": 25,
"start_time": 597.483,
"text": " It's just computing the function of what's the next cognitive token?"
},
{
"end_time": 647.278,
"index": 26,
"start_time": 626.101,
"text": " Given what's happened right up till this moment and the recent past, what should I do next? Then when you do that, that goes back into the loop and you proceed from there. And so this leads to, in some ways, a radically different view of what cognition really is. It, for example, leads to a completely different perspective of what memory is. Memory is not"
},
{
"end_time": 672.688,
"index": 27,
"start_time": 647.568,
"text": " We don't actually have somewhere stored in our brain events that took place or some fact that we've learned. What we actually have stored is just the capacity to autoregressively generate. If I ask you a question, what did you do last summer? And you say, well, I went to this place or you start imagining yourself jumping off of a cliff or something like that. What you're doing, according to this theory, is you're in real time, just in time,"
},
{
"end_time": 688.234,
"index": 28,
"start_time": 672.688,
"text": " Taking that question that prompt generating the very next output from that and then running from there what did i do last summer well i and then you continue to generate and so that really leads to a completely different perspective on what what memory is."
},
{
"end_time": 715.418,
"index": 29,
"start_time": 688.234,
"text": " It's no longer about storing facts. It's about having a brain, just like a large language model, that has these potentialities for facts. I call them potentialities because they're infinite. What does a large language model know? How many questions can you ask it? You can ask it an infinite number of questions and it can give a reasonable response to all those because it does this autoregressively. It's not retrieving this fact or that fact. Rather, it has the capacity to generate all of these facts."
},
{
"end_time": 737.927,
"index": 30,
"start_time": 715.794,
"text": " Finally, I'm probably over five minutes at this point, we can extend this even beyond language and linguistic responses, but more broadly to cognition. As I mentioned, thinking about a sequence, thinking about what am I going to do later on? I'm going to go hang out with my friends, maybe I'm going to have to get in my car, things like that. All of this, any thinking that takes place over time,"
},
{
"end_time": 765.674,
"index": 31,
"start_time": 737.927,
"text": " Well,"
},
{
"end_time": 785.111,
"index": 32,
"start_time": 766.749,
"text": " Fascinating. Obviously, Elon and I, we share a lot of these ideas in common. But one of the things that comes to mind right away, and we've discussed this before, is how surprisingly powerful this language model has been for doing what seems like non-language tasks."
},
{
"end_time": 815.06,
"index": 33,
"start_time": 785.794,
"text": " So the example I like to think of is if you imagine building like a robot or even like some kind of spaceship and we go back a few decades, you would tend to think, all right, we're going to have this compute engine. We're going to have this kind of logic machine and that's going to churn through facts and it's going to deduce things and do inductions. And this is what AI in the 80s kind of looked like. And then maybe after you had that compute engine, you would then build a communication module that could talk kind of as like an add on."
},
{
"end_time": 831.015,
"index": 34,
"start_time": 815.316,
"text": " that the ability to just communicate seems at the periphery compared to the real thinking stuff, right? But what we've seen out of these models, and I think the best example is their ability to computer program, all of these capabilities just fell out."
},
{
"end_time": 860.026,
"index": 35,
"start_time": 831.391,
"text": " of this token prediction. We didn't have to build an algebra engine or something that understands trigonometry or even optics. And when you think about when it generates the images, it gets reflections right. It gets speckles and bubbles and all kinds of things. It's learned a lot about about physics. And you can do this with the language models as well. How is it that it's that all that thinking horsepower just fell out of what seemed like the walkie talkie, right? The communication module."
},
{
"end_time": 891.118,
"index": 36,
"start_time": 861.22,
"text": " So in a more general sense, I've been thinking about is our brain a computer? Is it a computer-like object? Is it fundamentally a different category? Or is it like the thing that you have in your phone or your laptop? And this has been highly debated since Turing showed up on the scene. But I would argue that software, the idea of software, is the most important idea humans have come up with in maybe a thousand years."
},
{
"end_time": 919.019,
"index": 37,
"start_time": 891.647,
"text": " Because it gives us a thinking tool, it gives us a token to understand a lot of these problems that previous philosophers and thinkers and cognitive scientists were just absolutely baffled by and for no fault of their own. Things like the redness of red or the pleasure and pain experience that we feel and we wonder why does the vacuum cleaner not have that and so on. And I think that the real nature of this is we've discovered that we're software."
},
{
"end_time": 935.725,
"index": 38,
"start_time": 920.486,
"text": " And the more interesting thing that goes with that in the technological world in the past few decades, we've now found out that software is actually more interesting than we thought. Alan Turing and Von Neumann said, hey, why don't we just put this pattern into the machine and it'll do a complicated process."
},
{
"end_time": 965.503,
"index": 39,
"start_time": 936.186,
"text": " But as we saw and heard with Levin, it's more subtle than that. You can set into motion something that the Lego blocks themselves have some sort of intelligence and maybe not at the level of the brain, but maybe like ants and that they're self-assembling. One of the big things for me was the earlier version of the neural networks that led to this transformer is one of the popular ones was called the LSTM, the long short-term memory. And this is from like 1991."
},
{
"end_time": 995.503,
"index": 40,
"start_time": 966.084,
"text": " And what's fascinating about that is there's just this kind of this tape, this large vector, and it can do stuff. It's just a language. And if you put this language and you run this machine, which seems like it's just biting its own tail, it can do stuff. And I remember looking at that and thinking, well, that could be what DNA is. The DNA might not just be mapping to proteins, but I think a lot what Levin has now shown in the lab, that it's instantiating"
},
{
"end_time": 1024.428,
"index": 41,
"start_time": 996.118,
"text": " a kind of a proto-intelligence or an intermediate intelligence or some non-human kind of problem-solving engine and that that is what is responsible for this miraculous thing that Turing was interested in our shape, right? This morphogenesis and how we get this pattern because it's not obvious that we have a genetic sequence and then jump to we've got hands and faces and really complicated structures. And I think we need to look at this intermediate software layer"
},
{
"end_time": 1052.637,
"index": 42,
"start_time": 1025.009,
"text": " which is now we're bare witness to this with these language models, that it's not a one-to-one, right? You're not really listening to my brain. My brain's not talking. Something else that lives in my brain is talking. As a side note, it feels very autoregressive because it's not, I don't know where I'm going to go with this, but as I say it, things come into mind. And something we talked about a couple of weeks ago, I think this applies to behavior in general."
},
{
"end_time": 1077.142,
"index": 43,
"start_time": 1053.063,
"text": " Right think about when you you didn't know what you're gonna experience this event today but you knew by just showing up and being there interesting things would happen and so i think life in general it has this kind of autoregressive property where sometimes just getting out into the mix will give you all of these ideas and avenues and pathways that you didn't see before so that's that's the idea i'm excited about the most is thinking about virtual machines this is something that the"
},
{
"end_time": 1095.401,
"index": 44,
"start_time": 1077.892,
"text": " technological world you go to amazon they use what's called virtual machines and we can talk about that more but this these layers between the code and the thing that you're experiencing the app in the in the software world and technology on your laptop it's not one-to-one none of the programs you're using are talking to the chip"
},
{
"end_time": 1125.111,
"index": 45,
"start_time": 1095.93,
"text": " They're talking in some other language, C or Python, and then Python is talking to the chip with even more intermediates. And I think that we need to consider that our brain and our mind and our self, the ability to instantiate multiple selves, is because of this kind of virtualization. Yeah, and if I can just quickly, to me that is even more than sort of the realization of that this is an explanation of language, an explanation of the brain. It is this radical departure from thinking of"
},
{
"end_time": 1151.169,
"index": 46,
"start_time": 1126.527,
"text": " We thought of math for a long time as a method of description of the physical processes, but it's not about the symbols on the board. The symbols on the board in some ways were an early aperture into the fact that there's this informational life, that there's sort of computation. It's a phenomenon. In some ways, it's mind-independent."
},
{
"end_time": 1180.691,
"index": 47,
"start_time": 1152.09,
"text": " And what we're seeing now is we're actually capturing some of these things truly coming to life in Silicon. By seeing them in Silicon, by seeing them take flight, so to speak, outside of their traditional medium, we have to step back and have almost a newfound respect for the informational itself as being maybe more fundamental. As Michael Levin mentioned, right, that our brain is channeling"
},
{
"end_time": 1210.469,
"index": 48,
"start_time": 1181.203,
"text": " What else lives there?"
},
{
"end_time": 1239.104,
"index": 49,
"start_time": 1211.237,
"text": " What do you mean, what else lives there? Well, what else lives in our minds, first of all? Running a business comes with a lot of what-ifs. But luckily, there's a simple answer to them. Shopify. It's the commerce platform behind millions of businesses, including Thrive Cosmetics and Momofuku. And it'll help you with everything you need. From website design and marketing, to boosting sales and expanding operations, Shopify can get the job done and make your dream a reality."
},
{
"end_time": 1269.138,
"index": 50,
"start_time": 1239.599,
"text": " What other kinds of, and doesn't need to actually be an autoregressive process, but autogenerative processes, what other processes are running themselves at an informational level? I'm gesturing my hands now. I could tell a story about, well, I'm trying to express this or that idea, and I'm doing that with this gesture, and I've learned these kinds of things, that this is a good way to do that."
},
{
"end_time": 1295.947,
"index": 51,
"start_time": 1269.548,
"text": " Or I could tell us a story where the informational sort of process is making that happen because it wants to generate the same way that language wants to generate. Language wants to express in some very strange but very real way that the trajectory that language takes, it belongs to language itself. Maybe when I'm moving my body in this way, there is a process that's causing that that isn't really captured at all."
},
{
"end_time": 1316.305,
"index": 52,
"start_time": 1296.374,
"text": " by thinking in traditional terms of biomechanics or even in psychological or neurological terms, it may really be about long range trajectories through some sort of space. And so what are we, if not a collection perhaps of these almost alive informational systems? And then"
},
{
"end_time": 1330.845,
"index": 53,
"start_time": 1316.8,
"text": " If we potentially extend that outside of ourselves, what is what is an ecosystem? What is a society? What physics and more generally, there seems to be something very sticky about the past."
},
{
"end_time": 1357.363,
"index": 54,
"start_time": 1331.527,
"text": " Just a moment. Don't go anywhere. Hey, I see you inching away."
},
{
"end_time": 1381.084,
"index": 55,
"start_time": 1357.824,
"text": " Don't be like the economy, instead read the economist. I thought all the economist was was something that CEOs read to stay up to date on world trends. And that's true, but that's not only true. What I found more than useful for myself personally is their coverage of math, physics, philosophy, and AI, especially how something is perceived by other countries and how it may impact markets."
},
{
"end_time": 1405.094,
"index": 56,
"start_time": 1381.084,
"text": " For instance the economist had an interview with some of the people behind deep seek the week deep seek was launched no one else had that another example is the economist has this fantastic article on the recent dark energy data which surpasses even scientific americans coverage in my opinion they also have the charts of everything like the chart version of this channel it's something which is a pleasure to scroll through and learn from."
},
{
"end_time": 1422.961,
"index": 57,
"start_time": 1405.094,
"text": " Links to all of these will be in the description of course. Now the economist commitment to rigorous journalism means that you get a clear picture of the world's most significant developments. I am personally interested in the more scientific ones like this one on extending life via mitochondrial transplants which creates actually a new field of medicine."
},
{
"end_time": 1445.043,
"index": 58,
"start_time": 1422.961,
"text": " Something that would make Michael Levin proud. The Economist also covers culture, finance and economics, business, international affairs, Britain, Europe, the Middle East, Africa, China, Asia, the Americas, and of course the USA. Whether it's the latest in scientific innovation or the shifting landscape of global politics, The Economist provides comprehensive coverage"
},
{
"end_time": 1471.954,
"index": 59,
"start_time": 1445.043,
"text": " And it goes far beyond just headlines. Look, if you're passionate about expanding your knowledge and gaining a new understanding, a deeper one of the forces that shape our world, then I highly recommend subscribing to The Economist. I subscribe to them and it's an investment into my, into your intellectual growth. It's one that you won't regret. As a listener of this podcast, you'll get a special 20% off discount. Now you can enjoy The Economist and all it has to offer."
},
{
"end_time": 1495.23,
"index": 60,
"start_time": 1472.193,
"text": " Again, it's this respect for the richness of computation in a way that just changes everything."
},
{
"end_time": 1515.435,
"index": 61,
"start_time": 1495.93,
"text": " But I would suggest that previous cultures or older civilizations were very aware that the world was sort of embedded with this intelligent substance and this sort of pervade, you know, everything. And then that was sort of responsible for the complexity we see in forests and animals and human beings and so on."
},
{
"end_time": 1543.012,
"index": 62,
"start_time": 1515.93,
"text": " But just real quick, it reminds me of this idea of cellular automata that Wolfram really popularized and did all the great work in. That he kind of just showed that if you have graph paper and very simple rules, you can kind of see this ocean of this primordial soup kind of bubble up where you get non-trivial, very interesting behaviors out of what seems like as simple a system as you could possibly make it."
},
{
"end_time": 1572.073,
"index": 63,
"start_time": 1543.677,
"text": " And I would argue that our brain might be like that. It's way more sophisticated than graph paper, but it has all the properties that graph paper has. And so if we show that these simple things like Game of Life or the Wolfram Automata can spin out these patterns and patterns that make other patterns and so on, maybe our brain or even our entire body or all biology is more like a substrate where these informational beings, I like this term, I came across actually some religious terminology, spontaneities."
},
{
"end_time": 1601.084,
"index": 64,
"start_time": 1572.671,
"text": " Because that really captures this idea that like nobody put them there they just showed up but now they're off to the races and they're doing their own thing and I think that words Might be like this especially the early proto language were very much earworms They might have come from language of the birds I heard a beautiful mockingbird last night and if you imagine hearing that for millions of years That would do something Yvonne help me understand what's new about your model"
},
{
"end_time": 1630.299,
"index": 65,
"start_time": 1601.715,
"text": " Now, I don't mean, oh, Wittgenstein said this in 1952. I mean, you're saying, other than this fancy word, autoregression, you're saying that people are thinking in terms of a sequence, in terms of something that comes next. So when we're in the flow state, we're present and we're thinking, what are we going to do next? And when we're planning, even if we're planning for the future, we're thinking in terms of, well, what am I going to do one week from now? And then how does that affect what I'm going to do two days from now? And then one day from now, et cetera. It's always one step ahead."
},
{
"end_time": 1654.548,
"index": 66,
"start_time": 1630.776,
"text": " So what's new here? Well, what's new is that we actually have an architecture and an example in which we can see that the capacity to do long-range thinking, the capacity to live, so to speak, in an informational space, linguistic space, can all be compressed"
},
{
"end_time": 1684.36,
"index": 67,
"start_time": 1654.787,
"text": " We didn't know that before. There was no example of anything like this that had this kind of informational richness that did stuff that was based on such a radically simple and elegant kind of formulation. What's new is we've got a theory of the brain, a theory of cognition generally that"
},
{
"end_time": 1714.309,
"index": 68,
"start_time": 1684.838,
"text": " People might have pointed to in some, you know, in fairly vague terms, but that we can now build. What's new actually, what's new is not something I developed, it's the existence of these large language models. The large language models lead to the new epiphany that this is in fact perhaps how cognition works. And so we have a novel framework and it leads to a very, very different way perspective of thinking about classical cognition. So if you want to know what's new, for example,"
},
{
"end_time": 1744.445,
"index": 69,
"start_time": 1715.196,
"text": " The classic view of memory is the storage retrieval kind of model. For example, if I ask you to repeat back a sequence of a couple of digits or letters, the canonical view is that there's this thing called short-term memory, which is this box that holds stuff for a certain amount of time and then it decays. According to the autoregressive model, there is, and let me just add, then there's this other thing called long-term memory. What's that? Well, it's another box."
},
{
"end_time": 1771.715,
"index": 70,
"start_time": 1745.247,
"text": " That some stuff from short-term memory if you want to hold it for a longer time you put it in that box and that's like It's refrigerated so it lasts longer And so you've got these two boxes with stuff in them and then when you want to find some information What you do is you you have to have a retrieval process you go and you see well What is what's related to the what I'm talking about right now? What information where did I leave my keys or what did I do last summer? Let me go find the relevant memory and retrieve it"
},
{
"end_time": 1782.875,
"index": 71,
"start_time": 1772.363,
"text": " The autoregressive model completely obliterates, A, the distinction between short-term memory and long-term memory, and in fact, completely obviates the need for any sort of retrieval process at all."
},
{
"end_time": 1808.029,
"index": 72,
"start_time": 1783.404,
"text": " So it is a completely different framework. Instead, what we're doing, according to this model, is we've got these stored weights. That's the large language model. It's just these billions of parameters. And those parameters, all they are, they instantiate a function. A function just, what does that function do? It says, here's a string of words, which you just turn into numbers. And then it says, what's the next one? That's our brain."
},
{
"end_time": 1821.101,
"index": 73,
"start_time": 1808.404,
"text": " Okay, this is your brain on autoregression, right? It's just basically a function that says we're going to take in an input and we're going to produce an output. And then you've got the short-term memory is, what have I said?"
},
{
"end_time": 1844.804,
"index": 74,
"start_time": 1821.527,
"text": " Since you asked the question, here's the question, I'm producing an answer. What did Kurt say to me a little while ago that's causing me to generate this very next token and the next one? There's some residual activation, and the model is called context. In the LSTMs, it's this memory, which may or may not be the right way."
},
{
"end_time": 1874.445,
"index": 75,
"start_time": 1845.418,
"text": " In the large language models, it's literally the context is like the whole thing, right? Whatever you've said until now, whatever the model has said in our brains, it's probably closer to the LSTMs. It's some sort of some sort of compressed version. But that is the whole flow. The whole flow is you've got a static set of weights, which is our brain and then our sort of dendritic weights. And then you've got this dynamic process of, OK, generate with those weights, generate one thing, tack it on."
},
{
"end_time": 1899.053,
"index": 76,
"start_time": 1874.77,
"text": " to what's been generated before, and this may go a lot longer than short-term memory. It may not be 15 seconds as short-term memory dictates. The classic experiments where they ask you, can you retrieve, can you repeat this particular sequence? That's not something we ever do in regular life. We don't have to remember. What exactly did I say five seconds, 10 seconds ago?"
},
{
"end_time": 1920.043,
"index": 77,
"start_time": 1900.196,
"text": " There was a kind of error of looking where the light is in trying to build out these models. You can give people that task and they can retrieve that stuff, but that doesn't mean we've got memory whose job it is to retrieve. Instead, what I'm proposing is that we've got generation, and generation is guided by what's happened in the past."
},
{
"end_time": 1949.48,
"index": 78,
"start_time": 1920.486,
"text": " That is guided by much, much more than the past 15 seconds. Again, it dissolves this arbitrary idea that there's a short-term memory box. All you've got is residual memory. That could reach back into time 15 seconds, a minute. We're having this conversation now. Will mentioned something when we were talking over there. That's still bouncing around. It's reaching in, again, the pregnant present, it's reaching in to what my current generation is. So it completely obliterates"
},
{
"end_time": 1969.36,
"index": 79,
"start_time": 1950.009,
"text": " I really like what you said just now at the end, but also in the beginning, that this is a theory of the brain rather than a theory of neurons or neuroscience."
},
{
"end_time": 1989.275,
"index": 80,
"start_time": 1969.684,
"text": " And I would argue that some of these new models are some of the first real kind of connecting from both ends. We've known for a hundred years or so what neurons are all about, at least in the basic sense. And cognitive scientists and philosophers from the other direction have been trying to figure out these properties like memory and planning and logic and stuff like that."
},
{
"end_time": 2009.923,
"index": 81,
"start_time": 1989.275,
"text": " But these modern things, they're made out of simulated neurons, right? At the bottom. They're very, very simple brain cells. They're not nearly as complicated as the ones we find in our own brain. But we can think of it as a model that has both scales at the same time. And what's to me so fascinating is not only do we not need a special box for short-term or long-term,"
},
{
"end_time": 2028.319,
"index": 82,
"start_time": 2010.06,
"text": " We don't need one for planning. We get that out of the box. Logic comes out of the box. Ability to write poems or something like that. It's all sort of latent in this just ability to predict the next token. We're just starting to see the emergence of an actual brain science and now what we need to do is go look at our neuroanatomy and see"
},
{
"end_time": 2056.271,
"index": 83,
"start_time": 2028.319,
"text": " Okay, is this what the hippocampus is doing? Is this what the amygdala is doing? Like, how are these cortical structures interacting with all these nodules? Because one thing, when we slice our brain open, it's not a homogenous mass of neurons. And evolution could have done it that way relatively easily. So these structures must have some utility. I have some theories on that we can come back to. Yeah, remind me. But a point which I know you'll certainly agree with. We shouldn't take the terminology of neural networks that seriously in the sense that"
},
{
"end_time": 2082.756,
"index": 84,
"start_time": 2056.613,
"text": " And it comes back to an earlier point we made. The brain is instantiating some sort of function here. And I really mean it when I say that at a high level that the brain is doing the same things as the LLMs. Now, does that mean that I think that we're talking about neural circuits doing the exact same thing as the computer circuits? Do I even mean that they're transformer models that are doing self-attention? Not necessarily. It's really about at this high computational level."
},
{
"end_time": 2091.237,
"index": 85,
"start_time": 2082.756,
"text": " And, you know, matrix multiply, which is really the core engine of Transformers. You didn't really need the brain as"
},
{
"end_time": 2116.015,
"index": 86,
"start_time": 2091.834,
"text": " It's nice to call it neural networks because it sort of, well, it rhymes with it. You can do it as these connected nodes. You could do it as a graph, but you could do it as a matrix multiply. And a mathematician will look at you funny if you say this has anything to do with the brain. And that really gets to this point of substrate agnosticity. And it's not even on substrate. It's not even about whether it's made out of biological matter or it's made out of silicon."
},
{
"end_time": 2137.585,
"index": 87,
"start_time": 2116.015,
"text": " It's really about thinking about these things, not at the architectural level, but thinking really at the functional level. And the functionalists, you know, there was a functionalist school once upon a time in psychology, but they didn't have the breadth or the depth of computational resources to really explore things this way. But I think we're becoming functionalists once again. Just real quick."
},
{
"end_time": 2158.746,
"index": 88,
"start_time": 2138.217,
"text": " I want to get to consciousness and language, but there are two points of confusion which I think may be related. So if language just refers to itself,"
},
{
"end_time": 2188.473,
"index": 89,
"start_time": 2159.445,
"text": " Then let's rewind the clock 10,000 years when our vocabulary was smaller. And let's just say we had the words fight and mouth and left. How do those three tokens become 3000 explicated tokens? So that's one question. Then another one is, well, what about indexicles? Like pass me that. Meet me here. I'm going there. Okay. What tethers that to the world? And even worse is"
},
{
"end_time": 2219.019,
"index": 90,
"start_time": 2189.701,
"text": " There's these sort of conjunction words that it's not exactly clear what they're doing and you're touching on a very difficult problem. So the origins of language have always been fascinating and really one of the greatest challenges in cognitive science. The problem got worse, not better in some ways, by virtue of what we've discovered because this, as you're pointing out, right, the claim here is that language is a sort of"
},
{
"end_time": 2246.869,
"index": 91,
"start_time": 2219.394,
"text": " Perceptual independent, it's its own kind of organism. If we look at animal signaling, and there's very strong, you know, many decades of research, animals don't seem to do language. I think we have a better idea of what we mean by that now. They don't do autoregressive next token generation. They have nothing remotely like this. What they do have is this kind of concrete signaling that, you know, there's this kind of situation arises in the environment. You make this sound."
},
{
"end_time": 2274.514,
"index": 92,
"start_time": 2247.551,
"text": " They don't have is, they don't have the, they don't have any of these things, they don't have anything like the true generative kind of system that we do. The fundamental question here is how on earth did we as organisms make this huge qualitative computational leap from one kind of system to a completely different kind of system where those words the and is are really doing, we're only now going to understand, we're only going to start to understand them"
},
{
"end_time": 2285.589,
"index": 93,
"start_time": 2274.838,
"text": " Within this kind of framework they don't show up in animal language they don't show up any sort of classic semiotics where it's like that symbolizes that they don't symbolize anything in the environment."
},
{
"end_time": 2305.009,
"index": 94,
"start_time": 2286.237,
"text": " But they're very powerful in the autoregressive process. They are doing things in that framework. And so I think there's an incredibly deep mystery here as to how language sort of jumped from what was presumably a purely kind of correlational association system of"
},
{
"end_time": 2332.039,
"index": 95,
"start_time": 2305.623,
"text": " this environmental stimulus and I'm going to make this sound or I'm going to release this pheromone or whatever it is that animal communication consists of to a stimulus independent computationally autonomous system that of course interacts with that other thing but is in many ways divorced and autonomous from it."
},
{
"end_time": 2360.265,
"index": 96,
"start_time": 2332.381,
"text": " And so I don't have an answer to that question by any means, but I think we have a refinement of the question. Where did language come from? It's now not just a question of how did we develop such a sophisticated signaling system? How do we have so many words for so many things and we're able to put them together in novel ways to express new things? It's how did we develop this autonomous, generative system? It's a very different question than even sort of the Chomsky question of origins of language. You're not going to get it from a simple genetic"
},
{
"end_time": 2384.155,
"index": 97,
"start_time": 2360.265,
"text": " Kind of a mutation or something like that that allows for more higher expressive communication abilities. It's not even communication. It's auto generation. And so it's not an answer to the question, but it is a respect for the fact that the question is extremely deep. I don't know where language came from. I have very strange thoughts about it now is similar to the conversation we had with with Michael Levin."
},
{
"end_time": 2408.422,
"index": 98,
"start_time": 2384.48,
"text": " There are patterns in the universe that have found their ways into our brains that are much, much richer than we could ever have imagined. They're just spectacular. They are far greater than anything. If you take the greatest scientists and the greatest engineers and lock them in a room for thousands of years, no one could have come up with anything remotely like language. Maybe they could have come up with"
},
{
"end_time": 2433.78,
"index": 99,
"start_time": 2408.422,
"text": " a brilliant communication system based on the standard symbol, you know, representation. But nobody could have come up with this. Where did it come from? So it makes me think of a few things. One, this idea that at one point there were like five words. And that's I don't think it would have ever been like that in a sense, because I think we have to remember that there's this analog and digital component. I can say hello or hello."
},
{
"end_time": 2461.92,
"index": 100,
"start_time": 2434.292,
"text": " Very different. I said the same token, but I had this analog kind of background layer to it. And so I think early on we need to think how that emerged first and then that how that morphed into the thing where we have strict things. But I came across two things recently. One, it was talking about Shakespeare and his signature and they have like maybe 30 or 40 versions of his signature and they're different. He never signed his name the same time. And I don't. So why do we even say that's a signature?"
},
{
"end_time": 2477.79,
"index": 101,
"start_time": 2461.92,
"text": " Another interesting thing I came across from a few hundred years ago, they found a letter and the letter was talking about rabbits. This one particular letter used the word rabbit I think at least two dozen times in one document. It was spelled differently every single time."
},
{
"end_time": 2503.695,
"index": 102,
"start_time": 2478.78,
"text": " How? I'm a bad speller, but at least I'll just make up something and then maybe be consistent. So the idea that words even have a particular spelling, that that's what we mean by that word, I think is a relatively modern concept, which doesn't really make any sense to me. And another thing that comes to mind is I would argue that language, as we know it now, this autoregressive, very sophisticated kind of software thing, probably didn't evolve"
},
{
"end_time": 2531.613,
"index": 103,
"start_time": 2504.275,
"text": " out of like even small groups like tribal groups or villages I think this would have emerged at the the city state and the the empire level right ancient Sumer things like that where you had thousands of people working in in cooperation because then you get this kind of externalized memory where the words are floating around right you hear words that no one you know even knows that word like oh that's a new word to all of us for example and so in like kind of a city environment you would get that"
},
{
"end_time": 2542.568,
"index": 104,
"start_time": 2532.261,
"text": " That language is something else. Are you thinking of a particular city, perhaps? Yeah, maybe, right? And then just also the other thing that comes to mind, I really believe in this kind of apprenticeship from the animal kingdom."
},
{
"end_time": 2566.374,
"index": 105,
"start_time": 2542.927,
"text": " I really think the language of the birds and animal calls, and we tend as modern humans to think, oh, well, they're just making noises and they don't actually mean anything. And I haven't done this, but I had the thought of like, try it, you know, try to learn a bunch of animal calls, try to learn a bunch of of bird songs. And I would, I would think this is going to fundamentally change your brain and it might start to make sense."
},
{
"end_time": 2592.824,
"index": 106,
"start_time": 2566.374,
"text": " So you think we're not giving nearly enough credit to animal communications. We have a very narrow straw that we're sipping through. We downloaded that first and that was kind of like Windows 1.0, which no one ever saw. And then later we get the upgrade and we're like, oh, that's that. We were animals too and we were also grunting or... And we still do. Right. We still do. So this brings up another idea of bootstrapping, but I want to come back to that."
},
{
"end_time": 2623.37,
"index": 107,
"start_time": 2594.616,
"text": " This episode is brought to you by State Farm. Listening to this podcast? Smart move. Being financially savvy? Smart move. Another smart move? Having State Farm help you create a competitive price when you choose to bundle home and auto. Bundling. Just another way to save with a personal price plan. Like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts and savings, and eligibility vary by state."
},
{
"end_time": 2649.121,
"index": 108,
"start_time": 2624.531,
"text": " so hello and then hello actually can be tokenized differently if you put an exclamation point in front of hello that's a great point yeah the token or i think that the next generation of models are going to just be listening to the audio of youtube and listening to the radio and stuff like that watching archive television and then they will get kind of both the analog and the digital i mean systems at the same time voice models i don't know people are using those uh i used um"
},
{
"end_time": 2674.957,
"index": 109,
"start_time": 2649.121,
"text": " You know from GPT, I started using one of the voices and my wife said don't use that anymore. It sounds a little too good and it really gets rosy Really good really well and I think that depends if it's a woman's voice. Yes, indeed Where are you listening to that exactly huddled away? Like you're spending a little bit too much time with GPT. Why is the door closed? What's your next token"
},
{
"end_time": 2702.722,
"index": 110,
"start_time": 2676.374,
"text": " So the prosody that's in these things is actually extraordinary and the prosody is just like sort of the sing-song version of it and I'm confident. I've looked into it, but they don't share. OpenAI doesn't tell you anything. OpenAI, that was a joke from the beginning. I think they almost meant it as a joke. And so we don't really know how they're doing, but I'm going to guess it's using the exact same... If I would argue that it falls out. It falls out, but it falls out"
},
{
"end_time": 2713.2,
"index": 111,
"start_time": 2703.285,
"text": " In conjunction with meaning of language itself and so prosody has its own meaning and probably preceded the kind that we call words."
},
{
"end_time": 2740.452,
"index": 112,
"start_time": 2713.592,
"text": " And so words probably bubbled up as a more strict version of prosody or something like that, right? You maybe began as closer to something like music or something like that. It reminds me of what Levin mentioned with this idea of sharing stressors. And then this kind of makes a community. Imagine we're all trying to predict the prosody, not just the token, but like the emotional valence that goes with it. And by doing that, we might be bootstrapping some sort of shared experience."
},
{
"end_time": 2769.121,
"index": 113,
"start_time": 2741.971,
"text": " Let's make the analogy, forget about neurons. We're here at the Behan Center, so computer science center of U of T and math. Let's think in terms of matrix multiplication. The multiplying of a matrix for language. Is that a necessary or sufficient or neither condition for consciousness? Well, I don't think language has consciousness, so I am going to"
},
{
"end_time": 2790.435,
"index": 114,
"start_time": 2770.043,
"text": " go out on a kind of metaphysical limb here and say that the symbolic is kind of a different, it's a new physics is one way I think about it. Sorry, and symbolic means? Well, symbolic actually means that the representation"
},
{
"end_time": 2820.896,
"index": 115,
"start_time": 2790.947,
"text": " of any, let's say a word. Let's actually talk about matrices multiplied for a second. So what happens in these models is you take, here's a big corpus, and I want to learn the relationship between words in the corpus. So what I do is, first I tokenize it. I chop it up into different pieces like the word word is going to get a certain numerical representation. I'm going to say 00111 or whatever there. And then some other word or maybe even just a part of a word like a suffix or something is going to have a different representation. I'm going to turn that into"
},
{
"end_time": 2824.241,
"index": 116,
"start_time": 2821.135,
"text": " a vector, which is just a string of numbers."
},
{
"end_time": 2851.51,
"index": 117,
"start_time": 2824.514,
"text": " In the course of learning, what I'm going to do is I'm going to learn what to multiply that vector by, such that it's going to have a higher dimensional representation. It's a much longer vector. And so we've got this big giant vector representations, and we can think of it as a very high dimensional space. Or if you don't like to think about high dimensional spaces, because nobody really does, because you can't really think about them, you can think about it as a three dimensional space. And each word is just a location in that space."
},
{
"end_time": 2869.855,
"index": 118,
"start_time": 2852.159,
"text": " What language is in this environment is that this word has its meaning by virtue of its not actual representation. The point where it is in the space is arbitrary. Where it is in the space is arbitrary. What gives it its meaning is where it is in relation to the other words."
},
{
"end_time": 2886.442,
"index": 119,
"start_time": 2870.572,
"text": " Where ever, you know, if we have the word red and blue, they're going to be in some relation to one another. That is what the models are using in order to do the next token prediction. Now when I say the symbolic, what I mean is that vectorization itself is an arbitrary kind of representation."
},
{
"end_time": 2911.834,
"index": 120,
"start_time": 2886.886,
"text": " There is nothing in that symbol. If you were to look at it, you could study that symbol all day long and you could study, you know, try to make sense of the zeros and ones, whatever, whatever you're, you know, however you're actually, you know, representing the space. You're not going to find the meaning in there. And I would say that same thing very much holds true for natural language. The word red doesn't mean red."
},
{
"end_time": 2940.213,
"index": 121,
"start_time": 2912.125,
"text": " Not only in my autoregressive sense in that I don't just mean that I mean the word right itself doesn't point to the qualities at all and It's just letters. It's squiggles. It has nothing to do with it Now that is very different and this is the turn where I'm gonna go to consciousness That is very very different from the kind of the neural representation of redness itself redness in my brain the kind that I think is rise to what we call conscious experience and"
},
{
"end_time": 2961.971,
"index": 122,
"start_time": 2941.937,
"text": " is red and blue actually really have true mathematical relations to one another in the representational space. There's something happening between the wavelengths and the sort of adoption release that's happening in my retina that scales in a particular way across these different cones."
},
{
"end_time": 2986.664,
"index": 123,
"start_time": 2962.432,
"text": " such that red and blue really mean something in relation to another, not just by virtue of their embedding in this arbitrary space, but by virtue of their actual properties. And so the symbolic, I believe, doesn't give rise to consciousness because it doesn't in any way instantiate these properties of the physical universe. Consciousness, phenomenal consciousness, all of it,"
},
{
"end_time": 3011.169,
"index": 124,
"start_time": 2986.698,
"text": " is sensory at its base. You can't imagine another kind of consciousness besides something from the physical universe impinging on your sensory system. It could be visual, it could be auditory, it could be tactile, it could be your own body, your proprioceptive, it could be a sense of where you are, of your movement through space. All of these things are ultimately a continuation"
},
{
"end_time": 3040.794,
"index": 125,
"start_time": 3011.476,
"text": " Not a representation, but a continuation of the physical universe. Those patterns that are in the physical universe are simply rippling through our nervous system. Language breaks that. Language turns things into arbitrary representations, symbols. And I think that symbols do not have the quality, they don't have the quality that sensation and perception do. And so I don't think that the matrix multiply, I think the matrix multiply in the case of language, we lose that."
},
{
"end_time": 3063.456,
"index": 126,
"start_time": 3041.135,
"text": " I think that it actually is pushed off into a different metaphysical space, so to speak, where there is no consciousness. I don't think large language models are conscious. I don't think there's any possibility of them being conscious. They cannot understand what redness is because they have no access to the analog space where these kinds of relations exist."
},
{
"end_time": 3091.22,
"index": 127,
"start_time": 3064.633,
"text": " When I think of consciousness now, it reminds me of this virtual machine idea that we were trying to find consciousness in the brain and say, where in this, you know, bucket of meat do we get this magical experience? And I think that we need to consider that we're instantiating this other thing or maybe multiple layers of it. And I think that's where this, I don't mean to say it's not interesting, but that's where this thing we call consciousness or maybe even self"
},
{
"end_time": 3118.882,
"index": 128,
"start_time": 3091.442,
"text": " lives and I think we see this with the cases of multiple personality syndrome and disorders where people are kind of obviously instantiating more than one seemingly conscious entity in a single brain and so to me that suggests there's not a one-to-one mapping it's more like you have a phone you have apps on that phone and when we think about consciousness where we are one of those apps I think it was installed in us by our parents"
},
{
"end_time": 3138.268,
"index": 129,
"start_time": 3119.172,
"text": " I don't think it would just naturally emerge. And I think we're only now just discovering kind of the large distribution of consciousnesses on the planet. One thing I'd love to pick your brain about is sort of things like aphantasia or inner monologue or not inner monologue. And what I think is really fascinating is"
},
{
"end_time": 3164.241,
"index": 130,
"start_time": 3138.268,
"text": " These kind of emerged as interesting topics of conversation relatively recently with the emergence of internet forums. Because as you might know, psychology experiments are very expensive and usually the number of participants is like 10 or 20 or something like that. But in the last decade or so, we see now 50,000 or 100,000 people communicating on one forum. And they'll ask a question of like, when you close your eyes and you think of an apple, what do you see?"
},
{
"end_time": 3181.101,
"index": 131,
"start_time": 3164.991,
"text": " And half of the respondents will say, oh, I can see a red shiny apple. I can see it on the tree and the light shining on it. Maybe there's a little green leaf. And the other half, it's very kind of evenly split. Say, I don't see anything at all. What do you mean that you see something?"
},
{
"end_time": 3211.374,
"index": 132,
"start_time": 3181.459,
"text": " And the same conversation happens with inner monologue. Do you talk to yourself throughout the day? And again, you get this conversation split where half the people are like, yeah, of course I plan my day and talk to myself and think. And the other half says, you hear voices? It's just very, very strange. And I've met very high functioning people that don't have visual imagination. That doesn't make any sense to me. I know, I know a fellow who says he has neither. And I said, well, what do you do when you're thinking? And he says, what do you mean? I just think."
},
{
"end_time": 3234.718,
"index": 133,
"start_time": 3212.108,
"text": " How does that work in your model? I don't know how it works at all but it's a very fair question. Is he a counterexample to your model? All it takes is one counterexample to disprove a theory in math at least. It won't disprove it at all because here's the thing. There's the doing it and then there's the ability to observe yourself doing it."
},
{
"end_time": 3256.254,
"index": 134,
"start_time": 3235.52,
"text": " And so it may very well be that people are running some sort of process, a very similar process, but aren't, don't have this extra meta awareness of the fact that they're doing it. It may very well be there are angels and watermelons around us. But the truth is, it's actually not, it's not essential to any of this theory, right? The language models themselves don't hear themselves think."
},
{
"end_time": 3272.927,
"index": 135,
"start_time": 3256.681,
"text": " You don't really need that. That's sort of a weird feature of our brains. Something Will and I have talked about quite a bit is that not only do we produce sort of the flow of thought, but there's somebody watching at the same time and that's observing it. That's not necessarily a given."
},
{
"end_time": 3293.916,
"index": 136,
"start_time": 3272.927,
"text": " It is phenomenologically true we will see people report that they have it i don't know how essential it is to being else and i think i don't know i'm not up on and i have to because people ask me this. Quite often i'm not up on the current research like do we see an event if it is or not in a monologue folks like when they have to think about something."
},
{
"end_time": 3309.684,
"index": 137,
"start_time": 3294.138,
"text": " is there time is the time that they take to think about it radically different from the time that it takes people or the class of problems that they can solve because it doesn't seem to be exactly it seems to be the same set of problems so the fact that they're so behaviorally similar"
},
{
"end_time": 3338.865,
"index": 138,
"start_time": 3309.684,
"text": " Means either that it's sort of epiphenomenal the fact that we hear the inner monologue or that i'm dead wrong and then you know that that it's just it's it's that's an illusion that inner monologue isn't even really part of the way we think i don't know but it's it's only a very interesting question but it is i would push back and say that. It's not entirely clear why we have to hear our own thoughts at all even if even if they're being generated reminds me of the idea that language might be a double edged sword."
},
{
"end_time": 3360.316,
"index": 139,
"start_time": 3339.633,
"text": " That it might be sort of the best and worst thing that ever happened to humanity. I often refer to it as a divine parasite, in that it's a good thing, but it sort of tickovers. In that, if you know language, right, I can easily just mind control you. I think we're going to see some amazing examples of that, some real examples of that later. But I could just say something like, you're now thinking about your breathing."
},
{
"end_time": 3390.145,
"index": 140,
"start_time": 3360.896,
"text": " And you're going to be like, oh, I don't want to pay attention to that. But I put that in your head. And the fact that you're an English listener, you might be thinking that now. Or the famous one, you know, don't think of pink elephants, and it's like too late. So there's this idea, you know, going back to the Tower of Babel, that there's sort of this danger in having a shared language. And maybe this distribution of inner monologue and not having it is a way of kind of curbing that problem. And then this is where, you know, it starts to get a little bit disturbing."
},
{
"end_time": 3402.654,
"index": 141,
"start_time": 3390.401,
"text": " If you think about language not as being even a communication system but an operating system and what language is meant to do is to make people do things and it is this shared cultural artifact"
},
{
"end_time": 3430.179,
"index": 142,
"start_time": 3402.944,
"text": " It's downloaded against your will. You don't sign a waiver before you sign up. It's too late. By the time you're reading the waiver, it's too late. You don't get to click permission to write. Exactly. Every baby, before they're born, is already being conditioned and language is being downloaded into them. Now we know, in a way we didn't know before, that it is this kind of its own self-running system."
},
{
"end_time": 3457.125,
"index": 143,
"start_time": 3430.555,
"text": " It's a product of society, not of any individual, and it's a product that society didn't design except in an emergent way. People didn't, you know, we don't think, I don't think neither of us thinks, either of us thinks that, you know, a bunch of, you know, very clever people got together and said, how are we going to control the populace? We need to design this system that's going to run in them. It's far stranger than that, but yet at the same time,"
},
{
"end_time": 3469.292,
"index": 144,
"start_time": 3458.029,
"text": " It isn't really of us individually, it is society. And so things like beliefs, you know, society itself is built up of all these notions that are language dependent."
},
{
"end_time": 3492.619,
"index": 145,
"start_time": 3469.497,
"text": " And they're running in us and they make us go you know you get up in the morning cuz you gotta go to your job because you gotta make money because you got you know you have to have certain degree of of prestige in the world and and you're gonna have to you know one day you're gonna you're gonna get older and all of these things animals don't know any of these things right who put these ideas in our minds well forget about who but how and the answer is language and so there is this"
},
{
"end_time": 3512.585,
"index": 146,
"start_time": 3492.619,
"text": " Thing that's in us but not really of us that is running and it's driving most of our behavior those things we do who's who's familiar with this idea of jail breaking. A language model right so there's things they won't do i don't want to do but if you're very clever about how you ask for you can get the kind of leak out that information."
},
{
"end_time": 3542.944,
"index": 147,
"start_time": 3513.404,
"text": " Are we subject to that same kind of vulnerability? And I would suspect and suggest that we are and that we need to be aware of this. This kind of prompt engineering or prompt hacking is probably something that's been happening since the beginning of civilization, but we didn't have any kind of meta awareness. And I think the most interesting thing in development about these AI models in general is how they're going to affect our mind, how they're going to change how we think about our language, our words, our cognition, our place in the universe."
},
{
"end_time": 3567.278,
"index": 148,
"start_time": 3543.251,
"text": " They're giving us this, it's not quite a magnifying glass. I call it the macroscope because it's showing us the big picture all at once and it might be a terrifying view that it's opening up. Even more potentially disturbingly, we now have, now that we've captured not just language but in some ways culture and thought in a system that we can manipulate and test."
},
{
"end_time": 3585.009,
"index": 149,
"start_time": 3567.517,
"text": " Doesn't that potentially give people, and maybe even people with ill intent, the opportunity to see what's effective in changing minds, what's effective in manipulating people's thoughts? That's a really great point you bring up because now that you have a sort of a simulated person in a jar,"
},
{
"end_time": 3612.432,
"index": 150,
"start_time": 3585.606,
"text": " You can test it. You can sort of A-B testing, right? Will they respond to this? Will they respond to that? And kind of simulate the attack a million times and then go test that sentence out. This is what scammers do. When they finally get people on the call, they have a script and that script is very like honed in. They've evolved that script over years potentially and the ones that get the scam, that determines the script that survives. So in that same kind of thing,"
},
{
"end_time": 3628.319,
"index": 151,
"start_time": 3612.654,
"text": " are we susceptible"
},
{
"end_time": 3658.046,
"index": 152,
"start_time": 3629.138,
"text": " You know, it's, as you said, it took years to develop and you know, the car salesman and all of that. These are, these have been honed over, you know, many, many generations. Now you can do it at high speed. Now we can do it at high speed, you know, in silicon, like boom, run the, run the experiment and you've just, you've just tested out a hundred thousand customers under these varying different conditions. Is this a good thing? Or is this information hazard? Is God just a token? Quite."
},
{
"end_time": 3688.797,
"index": 153,
"start_time": 3659.104,
"text": " When you say just a token, so let's have respect for these tokens, right? Because what they really are are these, you know, almost like informational angels or something that do all this stuff within the larger context, not only in language, but in our behavior, in the ecosystem, the larger social ecosystem. What is God in that system? And the answer, in my case, it would be, well, maybe it doesn't point the same way that red doesn't really point to that cup."
},
{
"end_time": 3714.309,
"index": 154,
"start_time": 3689.292,
"text": " But what does it point to? It's not pointing in the traditional sense, but it's doing stuff. If we think of an operating system, what does God do in our operating system? What does the token God do in our operating system? And then you have a completely different perspective. It's not about an ontology of like, does it point to some divine being, whatever that would mean. And, you know, when you try to define God, it all becomes, it falls apart in your fingers."
},
{
"end_time": 3734.65,
"index": 155,
"start_time": 3714.821,
"text": " But what does God do in the minds of people? Well, I think we have actually some good ideas about what that idea does. Is maybe that idea itself almost synonymous with what people mean by it? Now, that's where things get really interesting. You can imagine a God so powerful, he need not actually exist. Exactly."
},
{
"end_time": 3755.469,
"index": 156,
"start_time": 3735.35,
"text": " And something we talk about a lot, it's, you know, my thoughts are not your thoughts kind of a thing. And another joke that we follow up with, it's the thoughts that do the thinking. So if I were to phrase it, is God just a story? And if so, would that make him any less powerful? Right. And I think what you're suggesting is it's real. And I would say it's real like software."
},
{
"end_time": 3783.046,
"index": 157,
"start_time": 3755.998,
"text": " Right. Facebook's just software. It's not real. There's no actual physical entity called Facebook. I don't mean the headquarters. I mean, like the thing that we log into or whatever. But it's real. I don't think anyone would argue that things like large software platforms or even markets or societies, you can't pin them down. Like, what is Toronto? Is it that thing? Right? No, it's somehow weirder than that, but doesn't make it any less real, right? Well, there's a difference here between God as"
},
{
"end_time": 3802.415,
"index": 158,
"start_time": 3783.592,
"text": " One traditionally conceptualizes it, whatever that means, and then God as just a story that has effects on people. Now if the broader sweep of people believed God was just a story that happened to have effects, that would also have its own effect separate from the traditional conceptualization. So wouldn't the first and second be the same thing?"
},
{
"end_time": 3819.462,
"index": 159,
"start_time": 3803.439,
"text": " So I call this third order religious, right? First order religious is just sort of, it's just true, it's real, it's out there. You don't really question or worry about it. Second order would be, well, it has utility. It helps a lot of people. It's a very powerful, you know, collective force. And I would argue what I call the third order is to realize those first two are the same thing."
},
{
"end_time": 3850.111,
"index": 160,
"start_time": 3820.384,
"text": " It doesn't make it any less powerful or majestic or divine or inspirational or transcendent, maybe even more so because we now see how it sort of bootstraps into reality. Yeah, one way to think about this is getting outside just of sort of the very limited space of thinking about physical reality as impinging on our senses and causing these behaviors, but that the universe broadly is kind of reflected in these patterns that are taking place in our brain."
},
{
"end_time": 3877.142,
"index": 161,
"start_time": 3850.555,
"text": " And then you just have to do a quick little switcheroo. If we call the physical universe, and maybe the physical universe is an instantiation of some sort of platonic informational space, well, let's just call that God. And then what has happened here is that that has actually molded this particular system that's talking about itself in some very, very real sense. Or constructing itself, even. Or constructing itself."
},
{
"end_time": 3904.428,
"index": 162,
"start_time": 3878.08,
"text": " Okay, now we're in a room full of students of all ages, some researchers, some academics, and so on. What lesson do you wish you had when you were younger that can apply to them other than follow your passion? Be very, very suspicious of scientific orthodoxies. I think the process of growing up to some extent is realizing that adults have no idea what the hell they're doing."
},
{
"end_time": 3926.34,
"index": 163,
"start_time": 3904.804,
"text": " The same is true not only for the regular adults like the ones in your life, your parents who you kind of figure that out pretty early, but it's true of sort of the greatest scientists. I think we're seeing that now in a very deep way that there are certainties that I think we held from a societal standpoint that are getting very shaky."
},
{
"end_time": 3953.831,
"index": 164,
"start_time": 3926.664,
"text": " I personally see it as a ripping open of the foundation of what we thought knowledge was, for example. There was a certain kind of core basic findings. Well, language, people didn't even think about, really deeply think about what language is. At least they certainly didn't come to the conclusion that it's this thing that's doing this stuff. And that means that all the pious certainties of the scientific establishment"
},
{
"end_time": 3982.688,
"index": 165,
"start_time": 3954.258,
"text": " And we're in a place of science. I'm not saying, by the way, judge science more harshly than other kinds of belief systems. What I am saying is judge all belief systems with a great deal of skepticism because we are barely, barely scratching the surface of what we can actually know. We are, as much as we do know as a species and it's extraordinary, we can build incredible structures and we have figured out quite a bit about the physical universe, much more than our ancestors knew, much more than we knew 50 years ago."
},
{
"end_time": 4011.937,
"index": 166,
"start_time": 3983.916,
"text": " relative to what's unknown, we're just getting a glimpse of that. And so I'd say go into the world with a radically open mind, but at the same time also recognize that there are systematic ways of thinking that have led us to this point. The engineering of large language models got us to the point where we can now say, wait a second, what is all this stuff? And so there is"
},
{
"end_time": 4040.316,
"index": 167,
"start_time": 4012.278,
"text": " Beautiful. I think first I'd say have the courage to believe in your own ideas. All of the stuff we're talking about now, if we go back a decade or so, it would have been almost impossible to listen to it."
},
{
"end_time": 4060.862,
"index": 168,
"start_time": 4040.896,
"text": " We were inside of this AI winter and all of these interesting ideas were just, it was hard to get them out. And I would suggest that some of the things that you are thinking about now and you are working about and you're excited about, you're probably keeping them like close in your pocket hidden because you think, oh, the world's not ready for that idea yet."
},
{
"end_time": 4086.664,
"index": 169,
"start_time": 4061.032,
"text": " Be bold and put it out there because you don't want to be the second person to put the idea out there. You want to say, no, I was thinking about that a while back. The other thing I would say is tolerate ambiguity and be wary of strong opinions. If you find yourself having a very strong opinion or you're talking to someone and they instantly kind of activate this very strong reaction, question that."
},
{
"end_time": 4100.094,
"index": 170,
"start_time": 4087.5,
"text": " Question why you have this immediate and ask yourself, am I really thinking about that or did I have the answer ready to go? And if you find that you or other people have the answer ready to go, then where did that answer come from?"
},
{
"end_time": 4118.524,
"index": 171,
"start_time": 4100.828,
"text": " If you cook up that answer and you thought about it for a long time that's fine but just question question your beliefs and be aware that our mind tries to prevent us in society tries to prevent us from thinking these unthinkable thoughts and the thoughts that we're sharing today i would argue that were nearly unthinkable even a decade ago."
},
{
"end_time": 4138.814,
"index": 172,
"start_time": 4119.735,
"text": " So we're going to open this up to audience questions and it will be exclusive to Substack so search the Substack which is on screen right now. Thank you for watching and you have a Substack as well and you have a Substack as well, correct? I have. I haven't put my material on it but I have a stuff that I'm going to be posting. We'll place those links on screen. Thank you."
},
{
"end_time": 4168.899,
"index": 173,
"start_time": 4139.872,
"text": " Okay, we're now at the end of the Q&A with the audience. And if you liked the regular interview, this far surpassed it. The audience grilled these two. So again, that's Substack. You can search my name, Kurt Jymungle, and you'll find it. Thank you to everyone. Thank you to Elon. Thank you to Will. Thank you to Addie for putting this on. Hi there."
},
{
"end_time": 4193.831,
"index": 174,
"start_time": 4169.241,
"text": " Kurt here. If you'd like more content from Theories of Everything and the very best listening experience, then be sure to check out my sub stack at kurtjymungle.org. Some of the top perks are that every week you get brand new episodes ahead of time. You also get bonus written content exclusively for our members. That's C-U-R-T."
},
{
"end_time": 4223.558,
"index": 175,
"start_time": 4194.104,
"text": " j-a-i-m-u-n-g-a-l dot org. You can also just search my name and the word sub stack on Google. Since I started that sub stack, it somehow already became number two in the science category. Now, sub stack for those who are unfamiliar is like a newsletter, one that's beautifully formatted. There's zero spam. This is the best place to follow the content of this channel that isn't anywhere else. It's not on YouTube. It's not on Patreon."
},
{
"end_time": 4252.654,
"index": 176,
"start_time": 4223.797,
"text": " It's exclusive to the Substack. It's free. There are ways for you to support me on Substack if you want, and you'll get special bonuses if you do. Several people ask me like, hey, Kurt, you've spoken to so many people in the field of theoretical physics, of philosophy, of consciousness. What are your thoughts, man? Well, while I remain impartial in interviews, this Substack is a way to peer into my present deliberations on these topics."
},
{
"end_time": 4282.346,
"index": 177,
"start_time": 4252.892,
"text": " And it's the perfect way to support me directly. KurtJaymungle.org or search KurtJaymungle sub stack on Google. Oh, and I've received several messages, emails and comments from professors and researchers saying that they recommend theories of everything to their students. That's fantastic. If you're a professor or a lecturer or what have you, and there's a particular standout episode that students can benefit from or your friends,"
},
{
"end_time": 4312.432,
"index": 178,
"start_time": 4282.688,
"text": " Please do share. And of course, a huge thank you to our advertising sponsor, The Economist. Visit Economist.com slash Toe, T-O-E to get a massive discount on their annual subscription. I subscribe to The Economist and you'll love it as well. Toe is actually the only podcast that they currently partner with. So it's a huge honor for me. And for you, you're getting an exclusive discount. That's Economist.com slash Toe, T-O-E."
},
{
"end_time": 4334.65,
"index": 179,
"start_time": 4312.722,
"text": " And finally, you should know this podcast is on iTunes, it's on Spotify, it's on all the audio platforms. All you have to do is type in theories of everything and you'll find it. I know my last name is complicated, so maybe you don't want to type in Jymungle, but you can type in theories of everything and you'll find it. Personally, I gain from rewatching lectures and podcasts."
},
{
"end_time": 4349.206,
"index": 180,
"start_time": 4334.804,
"text": " I also read in the comment that toe listeners also gain from replaying, so how about instead you re-listen on one of those platforms like iTunes, Spotify, Google Podcasts, whatever podcast catcher you use. I'm there with you. Thank you for listening."
}
]
}No transcript available.