Muse Wonder: Zuckerberg and Wang's $100B AI

章节 1：Muse 的突破用例与多模态优势

📝 本节摘要：

本章探讨了 Muse 的个人化用例，如 Alex 用其购买茶叶，Mark 用其进行 MMA 训练反馈。两人强调了模型的多模态能力、端到端训练策略以及 Meta 在图像和视频数据上的优势，指出多模态交互是未来个人超级智能的核心。

[原文] [John]: At the end of last year, we're like, all right, we are quite... We haven't seen the models yet. You know, we're going to write this check for like, I don't know, like $100 billion, $200 billion. I'm like, Alex, are you sure this is going to work? It's like—Cheers. Cheers. Yeah. Cheers. Cheers. Cheers. So, give me the update on Muse. If you were looking at the third-party reports, you would see that this might be the fastest-growing agent or AI app that I think has existed yet. What are your guys' personal favorite Muse use cases, either yours or other people's?

[译文] [John]: 去年年底，我们心里想，好吧，我们相当……我们还没看到模型。你知道，我们要开出这张支票，大概，我不知道，100 亿或 200 亿美元。我当时问 Alex，你确定这行得通吗？就像——干杯。干杯。是的。干杯。干杯。干杯。所以，给我讲讲 Muse 的最新进展。如果你看第三方报告，你会发现这可能是有史以来增长最快的智能体或 AI 应用。你们个人最喜欢的 Muse 用例是什么，无论是你们自己的还是别人的？

[原文] [Alexandr Wang]: I was actually talking about this at dinner yesterday. My favorite thing is I have my Muse, whose name is Oiler, constantly source increasingly obscure teas from increasingly obscure vendors of tea. So at this point he's, on a weekly basis, my Muse is buying some random tea from some farm in Taiwan or some farm in Japan and sourcing it over and bringing it back. What's your favorite Muse use case?

[译文] [Alexandr Wang]: 我昨天晚饭时其实聊过这个。我最喜欢的是，我的 Muse（名字叫 Oiler）不断从越来越冷门的茶叶供应商那里采购越来越冷门的茶。所以现在，每周我的 Muse 都会从台湾或日本的某个农场买一些随机的茶，把它采购回来。你最喜欢的 Muse 用例是什么？

[原文] [Mark Zuckerberg]: I don't know. I mean, I use it for a ton of stuff. I still like... My favorite both use case and probably eval for the product is having it coach me in MMA.

[译文] [Mark Zuckerberg]: 我不知道。我是说，我用它做很多事。我仍然喜欢……我既把它当作用例，也当作产品的评估标准，就是让它指导我进行综合格斗（MMA）训练。

[原文] [Mark Zuckerberg]: So I put cameras in my gym, and I basically... It watches the cameras when I'm in there, and it gives me feedback in between the rounds, which is a very hard problem, it turns out, because not only does it have to... It basically has to process all the video quickly enough because you only get 1 minute in between the rounds. Oh, it coaches you between the rounds? Yeah. I want feedback. I want feedback on how to win. You know what I mean? It's like, come on.

[译文] [Mark Zuckerberg]: 所以我在健身房装了摄像头，基本上……当我在那里时它观看摄像头，并在回合之间给我反馈。事实证明这是个非常难的问题，因为它不仅要……它必须足够快地处理所有视频，因为回合之间你只有 1 分钟时间。哦，它在回合之间指导你？是的。我想要反馈。我想要关于如何获胜的反馈。你懂我的意思吗？就像，拜托了。

[原文] [John]: Did you have to instruct it... I've wondered about this too, because looking at videos is a really good AI use case. It feels like you can't just give it a multi-gigabyte video file. So do you tell it how to slice stills out of it? No, it basically... Yeah, it pulls stills. I think it kind of figured that out by itself. Um, it turns out that Muse has pretty good native understanding of MMA, so that's good. So like the multi-modal— The pre-training. Yeah, someone on the team knew. Now...

[译文] [John]: 你需要指令它吗……我也一直在想这个问题，因为观看视频是一个非常好的 AI 用例。感觉你不能直接给它一个几 GB 的视频文件。所以你是告诉它如何从中截取静态帧吗？不，它基本上……是的，它提取静态帧。我觉得它自己搞明白了。嗯，事实证明 Muse 对 MMA 有很好的原生理解，这很好。所以像多模态——预训练。是的，团队里有人懂。现在……

[原文] [Mark Zuckerberg]: Just kind of out there in the corpus. I think it's a good test of the kind of coding ability that... Because really basically what it's doing, it's like, you know, within the agent, I'm not seeing any code, but it basically is writing this whole video processing, image processing pipeline. Yes. And I'm constantly telling it like, all right... Like, look you took 30 seconds to give me that feedback. I only have a 60-second break. Like, you gotta go faster. Just optimize yourself, optimize yourself. And it's, I think, a good example of how stuff is going to go in the future where basically everyone is actually going to be writing tons of code.

[译文] [Mark Zuckerberg]: 只是存在于语料库中。我觉得这是对某种编码能力的好测试。因为基本上它正在做的是，在智能体内部，我看不到任何代码，但它基本上在编写整个视频处理、图像处理流水线。是的。我一直在告诉它，好吧……看，你花了 30 秒给我那个反馈。我只有一分钟的休息时间。你得更快。自我优化，自我优化。我认为这是一个很好的例子，展示了未来事物将如何发展，基本上每个人实际上都会编写大量代码。

[原文] [Mark Zuckerberg]: They're just not going to know that they're writing code. What kind of MMA steers does a model give you? Um, you want to see? Yeah. Basically, let me see if I can pull some stuff out. But basically it gives me a lot of stuff about balance and, like, hip position. Or it's like when you're kind of in side control here. It's just like you didn't get this underhook in this place so you're going to... You're vulnerable to this getting rolled over here. I'll pull up some stuff. You guys should try this.

[译文] [Mark Zuckerberg]: 他们只是不知道自己在写代码。模型给你什么样的 MMA 指导？嗯，你想看吗？是的。基本上，让我看看能不能调出一些东西。但它基本上给了我很多关于平衡和髋部位置的信息。或者就像当你处于侧控位置时。它就是说你在这个地方没有做好下勾腿，所以你会……你在这里容易被翻滚。我调出一些东西。你们应该试试这个。

[原文] [Alexandr Wang]: It was definitely one of his first use cases actually of even the initial prototypes. Yeah. Yeah. I mean, I did this with OpenClaw before we even had Muse working. It was just like... Then that's why I was saying it's kind of like become an eval for this too. It's good. I actually didn't know that it was an eval for the product. So noted. Yeah. There you go. How is all this... In the model's training data, like the fine-grained, this is what an MMA fighter looks like when they're appropriately balanced, and this is when they're like too much on the back foot.

[译文] [Alexandr Wang]: 这绝对是它最初原型阶段最早的用例之一。是的。是的。我是说，甚至在 Muse 可用之前，我就用 OpenClaw 做过这个。它就像……所以这就是我为什么说它也成为了一种评估标准。很好。我实际上不知道它是产品的评估标准。记下了。是的。就是这样。这一切是如何……在模型的训练数据中，比如细粒度的，这是 MMA 选手平衡得当时的样子，这是他们重心过多在后脚时的样子。

[原文] [John]: Just, I'm wondering, how has it consumed so many images of appropriate versus inappropriate MMA posture? Here you can see it. It just pulls out frames from it and it's like too much on my back leg there. Yeah, but is this the emergent intelligence? Like it does actually... Again, does it know from seeing lots of images of MMA or does it know because it understands the concept of where you're meant to be balancing? It's read books about MMA, and then it sees that you're too much on the back foot, or— Well, we did train the models with a lot of visual perception data.

[译文] [John]: 只是，我想知道，它是如何消费了这么多关于适当与不适当 MMA 姿势的图像的？在这里你可以看到。它只是从中提取帧，然后说我的后腿重心太多了。是的，但这是涌现智能吗？就像它确实……再次，它是从看到大量 MMA 图像中知道的，还是因为它理解了你应该在哪里保持平衡的概念？它读过关于 MMA 的书，然后看到你的后腿重心太多，或者——嗯，我们确实用大量视觉感知数据训练了模型。

[原文] [Alexandr Wang]: Yeah. So, I mean, I think that that's like kind of making good multimodal models. I mean, this gets into like, I think, a core approach thing that we have, which is we're training the models end-to-end to be good personal agents, right? So like the question is why would Meta want to build these models? First of all, I think just superintelligence overall is going to allow us to run the company better and build all kinds of different things. But for the products that we're building, there's certain capabilities and certain mixes of skills that basically are just much more applicable to these cases.

[译文] [Alexandr Wang]: 是的。所以，我是说，我觉得这就像是在制作好的多模态模型。我是说，这涉及到我认为我们的一个核心方法，即我们端到端地训练模型成为好的个人智能体，对吧？所以问题是，为什么 Meta 想要构建这些模型？首先，我认为整体上的超级智能将让我们更好地运营公司并构建各种各样的东西。但对于我们正在构建的产品，有某些能力和某些技能组合基本上更适用于这些情况。

[原文] [Alexandr Wang]: So I mean, this is one of the things that Alex focused on from the beginning was like, all right, we really want to make sure that our multimodal program is good because that's disproportionately important for the types of things that people use all these services for. How do you train a good multimodal model? Is this where Meta just has an advantage because you guys have so much image and video data? It's very helpful. Yeah. And I think ultimately, a lot of this, it comes down to how are you focused on it, right?

[译文] [Alexandr Wang]: 所以我是说，这是 Alex 从一开始就关注的事情之一，就是，好吧，我们真的想确保我们的多模态项目是好的，因为这对于人们使用所有这些服务的类型来说不成比例地重要。如何训练一个好的多模态模型？这是 Meta 拥有优势的地方，因为你们有如此多的图像和视频数据吗？非常有帮助。是的。而且我认为最终，这一切归结于你们如何专注于它，对吧？

[原文] [Mark Zuckerberg]: I think because of the fact that we wanted to build personal superintelligence. And fundamentally, I think most people in the world really experience the world in a highly multimodal way. They like talking to other people and listening to things and seeing videos and filming things of their own. I think most consumers have this highly multimodal experience. And especially earlier this year, there was actually a point of tension almost in the industry because I think so much of the consensus view within the AI industry was that it was just all about coding and coding was all that mattered.

[译文] [Mark Zuckerberg]: 我认为这是因为我们想要构建个人超级智能。从根本上说，我认为世界上大多数人确实以高度多模态的方式体验世界。他们喜欢与他人交谈、倾听事物、观看视频以及拍摄自己的东西。我认为大多数消费者都有这种高度多模态的体验。特别是今年早些时候，行业内实际上存在一种紧张局势，因为我认为 AI 行业内的共识观点是，一切都关乎编码，编码是唯一重要的东西。

[原文] [Alexandr Wang]: And I think that for the success of a product like Muse and the success of a lot of these consumer use cases, it really depends on the ability to be strong and multimodal. Yeah. I think we're just kind of scratching the surface of that. I mean, a lot of the initial theory that we had was that, I mean, most people don't actually like reading that much, right? So even the text-based interaction model, I think it's something that people do. I mean, everyone texts. But when you think about stuff like the glasses that we're working on, or even our other products that are very visual, the glasses basically will eventually just be a pure multimodal product, right?

[译文] [Alexandr Wang]: 而且我认为，对于像 Muse 这样的产品以及许多这些消费者用例的成功，它确实取决于强大且多模态的能力。是的。我认为我们才刚刚开始触及表面。我是说，我们最初的很多理论是，大多数人其实并不喜欢阅读那么多，对吧？所以即使是基于文本的交互模型，我认为这也是人们会做的事情。我是说，每个人都发短信。但当你想到我们正在研发的眼镜，或者我们其他非常视觉化的产品，眼镜基本上最终将只是一个纯粹的多模态产品，对吧？

[原文] [Mark Zuckerberg]: It's like you can let it hear what you are listening to, and it can talk to you, and it can see what you see if you let it. And then eventually, I don't even know if we're going to have an app store on the glasses. It'll just be like a multimodal generation that kind of creates whatever you need right in your vision. And I imagine something like that will probably end up powering a lot of the apps that we build too. But yeah, I mean, we're still in the early days of that.

[译文] [Mark Zuckerberg]: 就像你可以让它听到你在听什么，它可以与你交谈，如果你允许，它可以看到你看到的。然后最终，我甚至不知道我们是否会在眼镜上有一个应用商店。它将只是像一种多模态生成，在你的视野中直接创建你需要的任何东西。我想象像这样的东西最终可能会驱动我们构建的许多应用。但是的，我是说，我们仍处于早期阶段。

[原文] [John]: But for now, the perception work has been really impressive. I want to ask you guys about the product space here in AI and try something out on you. If I was to describe to you, say, with Stripe, what I think a winning product looks like, it would take me pages and pages to talk about because we have the developer experience and we have to talk about all the international heterogeneity of payment methods. Then we have to talk about stablecoins. Like, it would take me a long time.

[译文] [John]: 但就目前而言，感知工作真的令人印象深刻。我想问问你们关于 AI 领域的产品空间，并尝试一下。如果我要向你们描述，比如说在 Stripe，我认为一个获胜的产品是什么样的，我会需要好几页纸来谈论，因为我们有开发者体验，我们必须谈论所有支付方法的国际异质性。然后我们必须谈论稳定币。就像，这会花我很长时间。