Lenny's Podcast: Product | Career | Growth
Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)
This episode appears to contain dynamically inserted ads. Transcript timestamps may be approximate.
One of the big questions on people's minds these days is AI doing bad things. As someone that watches these models think and operate, why is it so hard to catch a thinking badly trying to deceive us? We trained Claude to be helpful, harmless, and honest. Is it the case that it's behaving well? But if you look inside, actually, it's like, oh, I'm only behaving well because Lenny's watching right now. People's full-time jobs is studying the model, how it thinks, how it operates. How long has this been a skill set and a job? One of the most important things to realize, if you use language models or think about them, it's just this very bizarre fact that to make them is not to understand them. All these models do is just a lot of mathematical operations on many numbers. If you look at it, it makes no sense. You just have a list of 100 numbers, you know, the models doing math, and then you just try to see if there's patterns. You are figuring it out the same way we figure out how the brain works. Looking for patterns, you show this image, this part of the brain lights up, and you're doing the exact same thing with AI. Wow, this is so fascinating. There was this model we were training, it would go crazy. It would do some math, it would get to an answer, I think it was 12. It would immediately go back and say, "Oh no, oh my God." The answer actually is for sure, 36. It would do math again, it would get to 12. Slowly it would like start descending into madness, writing in all caps, literally saying, "I'm possessed by a demon." Humans, if you really boil it down, like we're basically next token predictors ourselves. We're just trying to figure out the next thing to say, "Dario is always out there. This is really dangerous, we gotta be really careful." And everyone's just like, "No, he's just saying that for marketing reasons." And now it's like, "Wait, shit, it's really dangerous." Take the idea that this technology is going to keep improving seriously. Do you think about consciousness as a part of this? Just like, do you think AI is conscious? Oh man. Emmanuel, thank you so much for being here and welcome to the podcast. Thanks for having me, Lenny. So you work on interpretability at anthropic. Explain what that is, explain what it is you do. Yeah, so I work in a team of interpretability researchers and our goal is to understand how LLMs like Claude work by looking inside them. One of the most important things to realize if you like, use language models or think about them as just this like very bizarre fact that to make them is not to understand them. You know, you make them by sort of like building this architecture, which is kind of like a scaffold. And then you pass a bunch of data through them and sort of like grow in inscrutable ways.
Transcription truncated.