[1hr Talk] Intro to Large Language Models

章节 1：LLM的本质：两个文件（模型权重与推理代码）

📝 本节摘要：

安德烈·卡帕西（Andrej Karpathy）首先通过一个极简的视角定义了大语言模型（LLM）：它仅仅是两个文件。以Meta发布的Llama 2-70B为例，他解释了这两个核心组成部分：一个是存储神经网络权重的参数文件（约140GB），另一个是运行这些参数的代码文件（仅需约500行C语言代码）。他强调了这种架构的自包含性——你可以在一台断网的MacBook上完全运行它。本节最后演示了模型如何通过这两个文件执行指令（如写一首关于Scale AI的诗），展示了推理过程（Inference）的简洁性。

[原文] [Andrej Karpathy]: hi everyone so recently I gave a 30-minute talk on large language models just kind of like an intro talk um unfortunately that talk was not recorded but a lot of people came to me after the talk and they told me that uh they really liked the talk so I would just I thought I would just re-record it and basically put it up on YouTube so here we go the busy person's intro to large language models director Scott okay so let's begin first of all what is a large language model really well a large language model is just two files right um there will be two files in this hypothetical directory so for example working with a specific example of the Llama 270b model this is a large language model released by meta Ai and this is basically the Llama series of language models the second iteration of it and this is the 70 billion parameter model of uh of this series so there's multiple models uh belonging to the Llama 2 Series uh 7 billion um 13 billion 34 billion and 70 billion is the biggest one no

[译文] [Andrej Karpathy]: 大家好，最近我做了一个关于大语言模型的30分钟演讲，大概就是一个入门级的讲座。遗憾的是，那次演讲没有被录制下来，但很多人在演讲结束后找到我，告诉我他们非常喜欢这个内容。所以我当时就想，我应该重新录制一遍，把它放到YouTube上。那么我们就开始吧，这是一份给大忙人准备的大语言模型入门指南。好的，让我们开始。首先，大语言模型到底是什么？其实，大语言模型就是两个文件，对吧。在这个假设的目录下会有两个文件。举个具体的例子，比如Llama 2-70B模型，这是由Meta AI发布的一个大语言模型。这基本上是Llama系列语言模型的第二次迭代，而这是该系列中拥有700亿参数的模型。Llama 2系列包含多个模型，有70亿、130亿、340亿参数的版本，而700亿是其中最大的一个。

[原文] [Andrej Karpathy]: w many people like this model specifically because it is probably today the most powerful open weights model so basically the weights and the architecture and a paper was all released by meta so anyone can work with this model very easily uh by themselves uh this is unlike many other language models that you might be familiar with for example if you're using chat GPT or something like that uh the model architecture was never released it is owned by open aai and you're allowed to use the language model through a web interface but you don't have actually access to that model so in this case the Llama 270b model is really just two files on your file system the parameters file and the Run uh some kind of a code that runs those parameters so the parameters are basically the weights or the parameters of this neural network that is the language model we'll go into that in a bit because this is a 70 billion parameter model uh every one of those parameters is stored as 2 bytes and so therefore

[译文] [Andrej Karpathy]: 很多人特别喜欢这个模型，因为它可能是目前最强大的开放权重（open weights）模型。基本上，Meta发布了权重、架构以及相关论文，所以任何人都可以非常容易地独立使用这个模型。这和你可能熟悉的许多其他语言模型不同，例如如果你在使用ChatGPT之类的东西，其模型架构从未被发布过，它归OpenAI所有，你只被允许通过网页界面使用该语言模型，但实际上你无法访问该模型本身。而在Llama 2-70B这个案例中，它真的只是你文件系统上的两个文件：参数文件和运行文件——某种运行这些参数的代码。参数基本上就是这个神经网络（即语言模型）的权重或参数，我们稍后会详细讨论这个。因为这是一个700亿参数的模型，每一个参数都存储为2个字节，因此……

[原文] [Andrej Karpathy]: the parameters file here is 140 gigabytes and it's two bytes because this is a float 16 uh number as the data type now in addition to these parameters that's just like a large list of parameters uh for that neural network you also need something that runs that neural network and this piece of code is implemented in our run file now this could be a C file or a python file or any other programming language really uh it can be written any arbitrary language but C is sort of like a very simple language just to give you a sense and uh it would only require about 500 lines of C with no other dependencies to implement the the uh neural network architecture uh and that uses basically the parameters to run the model so it's only these two files you can take these two files and you can take your MacBook and this is a fully self-contained package this is everything that's necessary you don't need any connectivity to the internet or anything else you can take these two files you compile your C cod

[译文] [Andrej Karpathy]: 这里的参数文件大小是140GB，之所以是2个字节，是因为数据类型是半精度浮点数（float 16）。除了这些参数——这仅仅是神经网络的一大串参数列表——你还需要某种东西来运行这个神经网络，这部分代码在我们的运行文件（run file）中实现。这可以是一个C文件、Python文件或任何其他编程语言的文件。它可以用任意语言编写，但C语言比较简单，以此为例只是为了让你有个概念。实现这个神经网络架构只需要大约500行C代码，且不需要任何其他依赖项，它基本上就是利用那些参数来运行模型。所以，仅仅只需要这两个文件。你可以带着这两个文件和你的MacBook，这是一个完全自包含的包。这就是所需的一切，你不需要任何互联网连接或其他东西。你带着这两个文件，编译你的C代码……

[原文] [Andrej Karpathy]: e you get a binary that you can point at the parameters and you can talk to this language model so for example you can send it text like for example write a poem about the company scale Ai and this language model will start generating text and in this case it will follow the directions and give you a poem about scale AI now the reason that I'm picking on scale AI here and you're going to see that throughout the talk is because the event that I originally presented uh this talk with was run by scale Ai and so I'm picking on them throughout uh throughout the slides a little bit just in an effort to make it concrete so this is how we can run the model just requires two files just requires a MacBook I'm slightly cheating here because this was not actually in terms of the speed of this uh video here this was not running a 70 billion parameter model it was only running a 7 billion parameter Model A 70b would be running about 10 times slower but I wanted to give you an idea of uh sort of just

[译文] [Andrej Karpathy]: ……你会得到一个二进制文件，你可以将其指向参数文件，然后就可以和这个语言模型对话了。例如，你可以发送文本给它，比如“写一首关于Scale AI公司的诗”，这个语言模型就会开始生成文本。在这个例子中，它会遵循指令并给你一首关于Scale AI的诗。我现在提到Scale AI，而且你在整个讲座中都会看到这一点，原因是最初主办我这次演讲活动的是Scale AI，所以我会在幻灯片里稍微拿他们举例，只是为了让内容更具体。这就是我们运行模型的方式，只需要两个文件，只需要一台MacBook。我这里稍微作弊了一下，因为就视频里的这个速度而言，它实际上运行的不是700亿参数的模型，而只是一个70亿参数的模型。700亿参数模型的运行速度大约会慢10倍，但我只是想让你对……

[原文] [Andrej Karpathy]: the text generation and what that looks like so not a lot is necessary to run the model this is a very small package but the computational complexity really comes in when we'd like to get those parameters so how do we get the parameters and where are they from uh because whatever is in the run. C file um the neural network architecture and sort of the forward pass of that Network everything is algorithmically understood and open and and so on but the magic really is in the parameters

[译文] [Andrej Karpathy]: ……文本生成及其外观有个概念。所以，运行模型并不需要太多东西，这是一个非常小的包。但是，当我们想要获取这些参数时，计算的复杂性才真正显现出来。那么，我们如何获取这些参数？它们从何而来？因为无论 run.c 文件里有什么——神经网络架构以及网络的正向传播（forward pass）——所有这些在算法上都是被理解且公开的等等，但真正的魔力在于参数之中。