Dylan Patel: NVIDIA's New Moat & Why China is "Semiconductor Pilled”

章节 1：NVIDIA 的偏执与收购 Groq 的战略意图

📝 本节摘要：

本节作为访谈开篇，主持人 Matt Turck 介绍了嘉宾 Dylan Patel（SemiAnalysis）及其在半导体硬件分析领域的权威地位。核心讨论围绕 NVIDIA 为何在看似稳固的霸主地位下依然保持极度的“偏执”——Jensen Huang（黄仁勋）深知如果仅依赖通用 GPU，可能会在成本和性能上被针对特定工作负载（如极速推理）的竞争对手击败。Dylan 详细分析了 AI 模型架构的不确定性（如自回归、思维链等），指出 NVIDIA 即使拥有强大的通用算力，也必须通过收购（文中提及收购 Groq）和推出专用芯片（如针对 KV Cache 或 Decode 优化的 CPX）来覆盖所有潜在的技术路径，以维持其高昂的利润率和市场统治力。

[原文] [Speaker A]: this is the biggest change in human history maybe ever what's about to happen with AI this is the biggest revolution bigger than industrial revolution jensen is very paranoid about losing if he just kept making his mainline chip people crush him on cost and performance acquiring Grock is how you get those resources to make more solutions for different parts of the market to stay king at the end of the day this is an economic war if the US and the West win in AI China will not rise to be the global hedgeimony but without AI China definitely will rise they're just going to outrun America

[译文] [Speaker A]: 这可能是人类历史上最大的变革，AI 即将发生的一切是比工业革命更巨大的革命。Jensen（黄仁勋）对于失败非常偏执（paranoid）。如果他只坚持制造主流芯片，人们会在成本和性能上击垮他。收购 Groq（Grock）就是获取这些资源的方式，以此为市场的不同部分制造更多解决方案，从而保持王座。归根结底，这是一场经济战。如果美国和西方在 AI 领域获胜，中国将不会崛起成为全球霸主（global hegemony）；但如果没有 AI，中国绝对会崛起，他们会直接超越美国。

[原文] [Speaker A]: hi I'm Matt Turk welcome back to the Matt podcast today I'm joined by the one person Wall Street and Silicon Valley turn to when they need to cut through the hardware hype Dylan Patel of Semi analysis we dove into many of the most important topics of today nvidia's massive move to acquire Grock the truth about the capex bubble whether the US power grid can actually handle the AI boom and the geopolitical chess match playing out between the US and China but I have to warn you this conversation went off the rails in the best possible way and we ended up going into all sorts of fun tangents like the strange phenomenon of Chinese romance dramas set inside semiconductor factories and what's really like when three AI famous roommates live together in SF please enjoy this fantastic conversation with Dylan hey Dylan welcome

[译文] [Speaker A]: 嗨，我是 Matt Turk，欢迎回到 Matt 播客。今天加入我的是华尔街和硅谷在需要看穿硬件炒作时都会求助的那个人——SemiAnalysis 的 Dylan Patel。我们深入探讨了当今许多最重要的话题：NVIDIA 收购 Groq（Grock）的重磅举动、资本支出（Capex）泡沫的真相、美国电网是否真能承受 AI 爆发，以及正在美中之间上演的地缘政治棋局。但我必须警告你，这场对话在最好的意义上“脱轨”了，我们最后聊到了各种有趣的离题话题，比如以半导体工厂为背景的中国浪漫剧这一奇怪现象，以及三位 AI 界名人室友在旧金山同住的真实生活。请享受这场与 Dylan 的精彩对话。嘿 Dylan，欢迎。

[原文] [Speaker B]: hello how are you

[译文] [Speaker B]: 你好，你好吗？

[原文] [Speaker A]: i'm great i'd love to start with Grock and Nvidia since it's still fresh so not so long ago Nvidia was saying that uh one GPU could do it all and now they're doing this acquisition non-exclusive deal with Grock what does that mean from your perspective

[译文] [Speaker A]: 我很好。既然这事还很新鲜，我想从 Groq（Grock）和 NVIDIA 开始聊。不久前 NVIDIA 还说一个 GPU 就能搞定一切，现在他们却在做这个收购以及与 Groq 的非独家交易。从你的角度看这意味着什么？

[原文] [Speaker B]: it's very clear we're not sure where AI models are headed in terms of you know over the next few years what happens to the architecture but you know the thing that I think everyone is sort of like agreed on is models are pretty auto reggressive right next token generation is like the thing but beyond that right attention mechanisms changed the how how it works everything changes right could could change and so what's interesting is the reason Nvidia one is because they just took like the widest surface area bet and then people kept developing models on that and that kind of shape worked

[译文] [Speaker B]: 很明显，我们不确定 AI 模型在未来几年会朝哪个方向发展，架构会发生什么变化。但我认为大家某种程度上达成共识的一点是，模型在很大程度上是自回归的（auto-regressive），对吧？“下一个 token 生成”（next token generation）是关键所在。但除此之外，注意力机制（attention mechanisms）改变了它的工作原理，一切都在变，对吧？可能会变。所以有趣的是，NVIDIA 赢的原因是他们押注了最宽的表面积（widest surface area），然后人们继续在这个基础上开发模型，这种形态奏效了。

[原文] [Speaker B]: but now the workload is so large that there is room for specialization that will give you 10x increases in certain domains right in a general purpose workload grock doesn't work right you know it can't train it can't you know it can't inference really really large models um cost efficiently right you can't serve many many many users but what it can do is it can go bl screamingly fast right same with the cerebrous open AI deal but that's like one workload right uh very decode focused right gener doing auto reggressive tokens in a in a single stream super fast

[译文] [Speaker B]: 但现在工作负载太大了，以至于有了专业化（specialization）的空间，这可以在某些领域给你带来 10 倍的提升，对吧？在通用工作负载中，Groq 是行不通的，对吧？它不能训练，它不能——你知道——成本高效地推理非常非常大的模型，对吧？你无法服务海量的用户。但它能做的是它可以快得惊人（screamingly fast），对吧？Cerebras 和 OpenAI 的交易也是如此。但这就像是一种工作负载，对吧？非常专注于解码（decode focused），对吧？在一个单一流中极快地生成自回归 token。

[原文] [Speaker B]: another direction AI models could head right we don't know are models going to think in one token stream or is it actually they're constantly context switching right and they're going from they have this humongous humongous context and they're generating in multiple parallel streams right and so Google and openi have both released mechanisms of this with their pro models where the model actually doesn't just have one single chain of thought for reasoning it has multiple right and then I don't exactly like you know and and and how they choose which one and what the final answer to you delivers is is an area of research um but there there is room for that kind of chip right something that works on very parallel a lot lot of streams of chain of thought and maybe the latency requirements are not as crazy right

[译文] [Speaker B]: AI 模型可能朝向的另一个方向——我们不知道模型是会在一个 token 流中思考，还是实际上它们在不断地进行上下文切换（context switching），对吧？它们可能从拥有巨大的上下文，转变为在多个并行流中生成，对吧？Google 和 OpenAI 都在其 Pro 模型中发布了这种机制，模型实际上不仅仅只有一条单一的思维链（chain of thought）进行推理，它有多条，对吧？然后我不太——你知道——关于它们如何选择哪一条以及最终给你提供什么答案，这是一个研究领域。但是，确实有这种芯片的空间，对吧？那种在非常并行、许多思维链流上工作的芯片，也许延迟要求没那么疯狂，对吧？

[原文] [Speaker B]: maybe you don't want to go blindingly fast right maybe you're okay with it being you know because I can spin up 100 parallel you know streams of thought or agents or whatever you want to call them maybe I I care a lot about cost there and because it's 100 in parallel instead of one going super super fast it's not as deep right the tree search or the depth of the inference is not as deep but it is much wider you know there's other parts of inference hey process do creating the KV cache so Nvidia has a chip for that right that's the CPX so they they've made the CPX they bought Grock for decode and then they still have their general purpose GPU

[译文] [Speaker B]: 也许你不需要快得令人眩晕，对吧？也许你可以接受它——你知道，因为我可以启动 100 个并行的思维流或智能体（agents），或者不管你想叫它们什么。也许我在那里更在乎成本。而且因为它是 100 个并行而不是一个超级超级快的，它没那么深，对吧？树搜索（tree search）或推理的深度没那么深，但它要宽得多。你知道推理还有其他部分，比如处理创建 KV Cache（键值缓存）。所以 NVIDIA 有一款针对这个的芯片，对吧？那就是 CPX。所以他们制造了 CPX，他们买了 Groq 用于解码（decode），然后他们仍然拥有他们的通用 GPU。

[原文] [Speaker B]: so they've they're kind of trying to cover their bases because unlike the first wave of AI chip companies where they sort of just made chips and then tried to figure out where it would work right they had a thesis Grock and Cerebrus both as well as Samanova right which was put a lot of memory on the chip and not necessarily in the case of Cerebrus and Grock no memory off chip and in the case of Samanova less memory offchip or slower memory offchip with higher capacity you know they they sort of all made similar bets in that direction and it didn't work for a while until it kind of did right um because there's a workload that now necessitates it

[译文] [Speaker B]: 所以他们正试图覆盖他们的基础（cover their bases），因为不像第一波 AI 芯片公司，那时他们有点像是制造了芯片然后试图弄清楚它能在哪里工作，对吧？他们有一个论点——Groq 和 Cerebras 都是，还有 Sambanova，对吧？就是把大量内存放在芯片上，而不一定——在 Cerebras 和 Groq 的案例中是芯片外无内存，在 Sambanova 的案例中是芯片外内存较少或更慢但容量更高。你知道，他们都在那个方向上做了类似的押注。这在很长一段时间内都没奏效，直到它某种程度上奏效了，对吧？因为现在有一种工作负载需要它。

[原文] [Speaker B]: nvidia recognizes they're they're the leader they're at the tent pole hey in one respect they can just run faster than everyone but it's kind of hard to be 2x better than Google or or OpenAI or whoever else's internal chip right to justify their you know 75% plus margins right and then they have to be 2x to 4x better to justify 4x better to justify their margins because that's what they're charging above COGS you know the question is what what architecture will deliver that well yes keep the programmability of their GPUs is great for training and for a lot of workloads

[译文] [Speaker B]: NVIDIA 意识到他们是领导者，他们处于顶梁柱的位置。嘿，在一个方面他们确实可以比任何人都跑得快，但要比 Google 或 OpenAI 或其他任何人的内部芯片好 2 倍是有点难的，对吧？为了证明他们——你知道——75% 以上的利润率是合理的，对吧？然后他们必须好 2 倍到 4 倍，好 4 倍才能证明他们的利润率是合理的，因为那是他们在销货成本（COGS）之上收取的费用。你知道问题是，什么架构能实现这一点？是的，保留 GPU 的可编程性对于训练和许多工作负载来说很棒。

[原文] [Speaker B]: but you know guess what i think I think a lot of people will just be downloading an open source model downloading an inference framework and pressing go right a little bit more complicated than that but that's that's going to be the consumption method for a lot of enterprises a lot of uh startups a lot of tech companies is they're just going to do that or they're going to rent the G GPUs or or rent the chips and then download an open source framework and model and go right

[译文] [Speaker B]: 但你知道吗，猜猜看，我认为很多人只会下载一个开源模型，下载一个推理框架，然后按下“开始”，对吧？虽然比这稍微复杂一点，但这将是许多企业、许多初创公司、许多科技公司的消费方式。他们只会那样做，或者他们会租用 GPU，或者租用芯片，然后下载开源框架和模型并开始运行，对吧？

[原文] [Speaker B]: and Nvidia recognizes this and hey there is room for products that aren't general purpose right the general purpose GPU will still probably be the main line for training and for a lot of inference and for costefficient inference but maybe blindingly fast or workloads that have a ton of prefill i.e creating the the KV cache maybe that those workloads could be different chips right and the CPX chip they announced right they say it's for the context processing creating the KV cache it's also really useful for video models because video models don't care about memory bandwidth and so you know why pay for the expensive memory that the general purpose chip has or why do what Grock is doing which is tying hundreds or thousands of chips together and not having memory but keeping the entire model on chip

[译文] [Speaker B]: NVIDIA 意识到了这一点，嘿，非通用产品的空间是存在的，对吧？通用 GPU 可能仍将是训练和许多推理以及成本高效推理的主线，但对于极速（blindingly fast）需求，或者有大量预填充（prefill）即创建 KV Cache 的工作负载，也许那些工作负载可以是不同的芯片，对吧？他们发布的 CPX 芯片，对吧？他们说它是用于上下文处理、创建 KV Cache 的。它对视频模型也非常有用，因为视频模型不在乎内存带宽。所以你知道，为什么要为通用芯片拥有的昂贵内存付费？或者为什么要像 Groq 那样做，即把数百或数千个芯片绑在一起，没有（片外）内存，而是把整个模型保持在芯片上？

[原文] [Speaker B]: the trade-off for that of course is you need thousands of chips and you have less compute per chip and so like Nvidia's trying to capture the whole surface area because again you don't know where models are headed

[译文] [Speaker B]: 这样做的代价当然是你需要成千上万个芯片，而且每个芯片的计算量较少。所以就像 NVIDIA 试图捕捉整个表面积（market surface area），因为再说一次，你不知道模型会朝哪里发展。