Lex Fridman Podcast #459: DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

The newest episode in the Lex Fridman podcast has arrived: #459 – DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters The full transcript is available here.

In it, Lex talks for more than 5 hours (!) with Dylan Patel, an AI and semiconductor expert and Nathan Lambert, a machine learning researcher.

It was a fascinating explanation of the current state of AI, including the recent Deepseek V3 and R1evolutions. With my background in IT, but no deep knowledge of how AI works, it gave me densely packed update of the state of the art large language models, and the recent innovations. I learned a lot. It talks about the Mixture-of-Experts (MoE) and Multi-Head Latent Attention (MLA) by which Deepseek is able to reduce the computational costs and memory usage.

The episode also gave a perspective on the geopolitical battle going on at this moment, where the USA sees AI as the most important defense evolution and is trying to limit China from developing it too fast.

The US is aggressively deploying export controls and innovation strategies to outpace competitors (mainly China). This maneuver is designed to drive immediate results… but only for a limited time:

(01:18:56) if you believe we’re in this sort of stage of economic growth and change that we’ve been in for the last 20 years, the export controls are absolutely guaranteeing that China will win long-term. If you do not believe AI is going to make significant changes to society in the next 10 years or 5 years. Five-year timelines are sort of what the more executives and such of AI companies and even big tech companies believe. But even 10-year timelines, it’s reasonable. But once you get to, hey, these timelines are below that time period, then the only way to create a sizable advantage or disadvantage for America versus China is if you constrain and compute, because talent is not really something that’s constraining. China arguably has more talent, more STEM graduates, more programmers. The US can draw upon the world’s people, which it does. There’s tons of foreigners in the AI industry.

The US is making a rapid, temporary investment in technological infrastructure, channeling resources into massive data centers and other strategic initiatives. This decisive move reflects a momentary drive to capitalize on current opportunities:

(01:22:43) Less than 10 years or 5 years to above. China will win because of these restrictions long-term, unless AI does something in the short-term, which I believe AI will make massive changes to society in the medium, short-term. … And even today, if Xi Jinping decided to get “scale-pilled”, IE, decide that scaling laws are what matters, just like the US executives like Satya Nadella and Mark Zuckerberg and Sundar and all these US executives of the biggest, most powerful tech companies have decided they’re scale-pilled and they’re building multi-gigawatt data centers, whether it’s in Texas or Louisiana or Wisconsin, wherever it is, they’re building these massive things that cost as much as their entire budget for spending on data centers globally in one spot. This is what they’ve committed to for next year, year after, et cetera. And so they’re so convinced that this is the way that this is what they’re doing.

China could potentially advance faster than the US, but it has not yet made this a top priority at the highest levels. In contrast, the US has focused on AI and chip export controls, limiting the most powerful chips from reaching China while only recently seeing signs of China’s interest with large subsidies and meetings:

(01:23:43) But if China decided to, they could do it faster than us, but this is where the restrictions come in. It is not clear that China as a whole has decided from the highest levels that this is a priority. The US sort of has. You see Trump talking about DeepSeek and Stargate within the same week. And the Biden admin as well had a lot of discussions about AI and such. It’s clear that they think about it. Only just last week did DeepSeek meet the second in command of China. They have not even met the top, they haven’t met Xi, Xi hasn’t set down, and they only just released a subsidy of a trillion RMB, roughly $160 billion, which is closer to the spending of Microsoft and Meta and Google combined for this year. So they’re realizing it just now. But that’s where these export restrictions come in and say, “Hey, you can’t ship the most powerful US chips to China. You can ship a cut-down version. You can’t ship the most powerful chips to all these countries who we know are just going to rent it to China. You have to limit the numbers.”

It is clear that the investments that are happening now are immense, and how the current link between AI tech leaders and US political power is a win-win situation for both. This will lead to further geopolitical tension.

Scary, right?

Further, it talks about the semiconductor industry, and how TSMC is now dominant over the previous leading chip producers Samsung and Intel. I learned that NVIDIA produces GPUs and sells them to the market, and Google has custom developed TPUs that they only use internally, as services in the long run are a more interesting business model.

I want to conclude with a philosophical Lex Fridman style topic (I love how Lex always adds this philosophical touch). Lex asked the current AI reasoning models this question:

(03:01:08) give one truly novel insight about humans

Here are the responses he got:

DeepSeek R1

(03:04:09) “Humans (are) able to convert selfish desires into cooperative systems by collectively pretending abstract rules like money laws and rights are real. And these shared hallucinations act as games where competition is secretly redirected to benefit the group turning conflict into society’s fuel.”

Gemini 2.0 Flash Thinking said:

(03:04:09) “Humans are not just social animals but self-domesticated apes. And this self domestication is the key to understanding our unique cognitive and social abilities.”

OpenAI o3-mini gave:

(03:06:19) “Humans are not fixed beings, but rather ongoing narratives, dynamic stories that we continuously write, edit, and reinterpret. This narrative plasticity is more than just memory or self-reflection. It’s an intrinsic cognitive process that acts like an internal error correction system. It allows us to adapt our identities and values over time in response to new experiences, challenges, and social contexts.”

OpenAI o1-pro (the model that costs 200$ per month to get access too) gave the best results, over and over again:

(03:05:31) “Humans are the only species that turns raw materials into symbolic resources. Then uses those symbols to reorganize the very materials that came from creating a closed feedback loop between meaning and matter.”

o1-pro again:

(03:05:31) “Humans are unique among known species in that they simultaneously rewrite two layers of reality; the external world and their own private mental landscapes. And then merge these two rewritten layers into a continuous personal narrative that feels objectively true.” Feels true. This is poetry.

As a conclusion, it is again clear to me that AI is both terrifying and fascinatingly hopeful at the same time. I just try to feel this tension between fear and hope in my body and navigate my own moral compass to apply it in ways I think is good for the world and fellow human beings.