Andrej Karpathy: Software Is Changing (Again)

章节 1：软件开发的范式转移：从 1.0 到 3.0

📝 本节摘要：

在本节中，Andrej Karpathy 强调了当前是进入软件行业的独特时刻，因为软件本质正经历剧烈变革。他回顾了“软件 1.0”（由人类编写的代码）到“软件 2.0”（通过优化器调整的神经网络权重）的演变，并正式提出了“软件 3.0”的概念——即大语言模型（LLM），其中提示词（Prompts）成为了用英语编写的新型程序。他以特斯拉自动驾驶系统为例，生动描述了软件 2.0 是如何逐步“吞噬”软件 1.0 代码的，并建议开发者需熟练掌握并在三种范式间灵活切换。

[原文] [Host]: please welcome former director of AI Tesla Andre Carpathy hello wow a lot of people here hello um okay yeah so I'm excited to be here today to talk to you about software in the era of AI

[译文] [主持人/Karpathy]: 请欢迎前特斯拉 AI 总监 Andrej Karpathy。（掌声）你好，哇，这里好多人啊。你好。嗯，好的，是的，我很兴奋今天能在这里和大家探讨 AI 时代的软件。

[原文] [Karpathy]: and I'm told that many of you are students like bachelors masters PhD and so on and you're about to enter the industry and I think it's actually like an extremely unique and very interesting time to enter the industry right now and I think fundamentally the reason for that is that um software is changing uh again

[译文] [Karpathy]: 我听说你们中许多人是学生，比如本科生、硕士、博士等等，你们即将进入这个行业。我认为现在实际上是一个极其独特且非常有趣的进入行业的时机。我觉得根本原因在于，嗯，软件正在发生变化，厄，再一次发生变化。

[原文] [Karpathy]: and I say again because I actually gave this talk already um but the problem is that software keeps changing so I actually have a lot of material to create new talks and I think it's changing quite fundamentally

[译文] [Karpathy]: 我说“再一次”，是因为我其实之前已经做过这个演讲了，嗯，但问题是软件一直在变，所以我其实有很多素材来创作新的演讲，而且我认为这种变化是相当根本性的。

[原文] [Karpathy]: i think roughly speaking software has not changed much on such a fundamental level for 70 years and then it's changed I think about twice quite rapidly in the last few years and so there's just a huge amount of work to do a huge amount of software to write and rewrite so let's take a look at maybe the realm of software

[译文] [Karpathy]: 我觉得大致来说，软件在如此根本的层面上可能有 70 年没怎么变过了，然后在过去几年里，我认为它相当迅速地变了两次。所以有大量的工作要做，有大量的软件需要编写和重写。那么，让我们来看看软件的领域。

[原文] [Karpathy]: so if we kind of think of this as like the map of software this is a really cool tool called map of GitHub um this is kind of like all the software that's written uh these are instructions to the computer for carrying out tasks in the digital space so if you zoom in here these are all different kinds of repositories and this is all the code that has been written

[译文] [Karpathy]: 如果我们把这看作是软件的地图——这是一个很酷的工具叫“GitHub 地图”——这有点像所有被编写出来的软件，厄，这些是让计算机在数字空间执行任务的指令。所以如果你放大这里看，这些是各种各样的代码库（repositories），这些就是所有已经被写出来的代码。

[原文] [Karpathy]: and a few years ago I kind of observed that um software was kind of changing and there was kind of like a new type of software around and I called this software 2.0 at the time and the idea here was that software 1.0 is the code you write for the computer software 2.0 know are basically neural networks and in particular the weights of a neural network

[译文] [Karpathy]: 几年前，我观察到，嗯，软件似乎在发生变化，出现了一种新型的软件，当时我称之为“软件 2.0”。这里的概念是，软件 1.0 是你为计算机编写的代码；而软件 2.0，基本上就是神经网络，特别是神经网络的权重（weights）。

[原文] [Karpathy]: and you're not writing this code directly you are most you are more kind of like tuning the data sets and then you're running an optimizer to create to create the parameters of this neural net and I think like at the time neural nets were kind of seen as like just a different kind of classifier like a decision tree or something like that and so I think it was kind of like um I think this framing was a lot more appropriate

[译文] [Karpathy]: 你不是直接编写这些代码，你更多的是在调整数据集，然后运行一个优化器来生成、来创建这个神经网络的参数。我认为在当时，神经网络被看作只是另一种分类器，就像决策树之类的东西，所以我认为这种（软件 2.0 的）框架描述要恰当得多。

[原文] [Karpathy]: and now actually what we have is kind of like an equivalent of GitHub in the realm of software 2.0 And I think the hugging face is basically equivalent of GitHub in software 2.0 and there's also model atlas and you can visualize all the code written there in case you're curious

[译文] [Karpathy]: 现在实际上我们拥有了软件 2.0 领域的 GitHub 等价物。我认为 Hugging Face 基本上就相当于软件 2.0 时代的 GitHub，还有 Model Atlas，如果你好奇的话，你可以把那里编写的所有“代码”可视化出来。

[原文] [Karpathy]: by the way the giant circle the point in the middle uh these are the parameters of flux the image generator and so anytime someone tunes a on top of a flux model you basically create a git commit uh in this space and uh you create a different kind of a image generator

[译文] [Karpathy]: 顺便说一下，那个巨大的圆圈，中间那个点，厄，那是 Flux 图像生成器的参数。所以每当有人在 Flux 模型之上进行微调时，你基本上就在这个空间里创建了一个 git 提交（git commit），厄，你就创造了一种不同类型的图像生成器。

[原文] [Karpathy]: so basically what we have is software 1.0 is the computer code that programs a computer software 2.0 are the weights which program neural networks uh and here's an example of Alexet image recognizer neural network

[译文] [Karpathy]: 所以基本上我们拥有的是：软件 1.0 是给计算机编程的计算机代码；软件 2.0 是给神经网络编程的权重，厄，这里有一个 AlexNet 图像识别神经网络的例子。

[原文] [Karpathy]: now so far all of the neural networks that we've been familiar with until recently where kind of like fixed function computers image to categories or something like that and I think what's changed and I think is a quite fundamental change is that neural networks became programmable with large language models

[译文] [Karpathy]: 目前为止，直到最近，我们要么熟悉的所有神经网络都有点像是固定功能的计算机，比如从图像到分类之类的。我认为发生的变化，而且是一个相当根本性的变化，是神经网络通过大语言模型（Large Language Models）变得可编程了。

[原文] [Karpathy]: and so I I see this as quite new unique it's a new kind of a computer and uh so in my mind it's uh worth giving it a new designation of software 3.0 and basically your prompts are now programs that program the LLM and uh remarkably uh these uh prompts are written in English so it's kind of a very interesting programming language

[译文] [Karpathy]: 所以在我看来这是相当新颖独特的，这是一种新型的计算机，厄，所以在我心目中，值得给它一个新的称号——软件 3.0。基本上，你的提示词（prompts）现在就是给 LLM 编程的程序，而且值得注意的是，厄，这些提示词是用英语写的，所以这是一种非常有趣的编程语言。

[原文] [Karpathy]: um so maybe uh to summarize the difference if you're doing sentiment classification for example you can imagine writing some uh amount of Python to to basically do sentiment classification or you can train a neural net or you can prompt a large language model

[译文] [Karpathy]: 嗯，所以也许为了总结这种区别，如果你在做情感分类，例如，你可以想象写一定量的 Python 代码来基本完成情感分类，或者你可以训练一个神经网络，或者你可以给大语言模型写提示词。

[原文] [Karpathy]: uh so here this is a few short prompt and you can imagine changing it and programming the computer in a slightly different way so basically we have software 1.0 software 2.0 and I think we're seeing maybe you've seen a lot of GitHub code is not just like code anymore there's a bunch of like English interspersed with code

[译文] [Karpathy]: 厄，这有一个少样本提示词（few-shot prompt），你可以想象修改它，以稍微不同的方式对计算机进行编程。所以基本上我们有软件 1.0、软件 2.0，而且我认为我们正在看到——也许你已经看到了——很多 GitHub 代码不再仅仅是代码了，代码中穿插着大量的英语。

[原文] [Karpathy]: and so I think kind of there's a growing category of new kind of code so not only is it a new programming paradigm it's also remarkable to me that it's in our native language of English and so when this blew my mind a few uh I guess years ago now I tweeted this and um I think it captured the attention of a lot of people and this is my currently pinned tweet uh is that remarkably we're now programming computers in English

[译文] [Karpathy]: 所以我认为有一类正在增长的新型代码。这不仅是一种新的编程范式，对我来说同样值得注意的是，它是用我们的母语英语编写的。所以当这在几年前让我大受震撼时，我发了一条推特，嗯，我认为它引起了很多人的关注，这也是我目前置顶的推特，厄，就是我们现在竟然在用英语给计算机编程，这太了不起了。

[原文] [Karpathy]: now when I was at uh Tesla um we were working on the uh autopilot and uh we were trying to get the car to drive and I sort of showed this slide at the time where you can imagine that the inputs to the car are on the bottom and they're going through a software stack to produce the steering and acceleration

[译文] [Karpathy]: 当我在，厄，特斯拉的时候，嗯，我们在研发自动驾驶（Autopilot），厄，我们试图让汽车自动行驶。我当时展示过这张幻灯片，你可以想象汽车的输入在底部，它们穿过一个软件栈（software stack），最终产生转向和加速指令。

[原文] [Karpathy]: and I made the observation at the time that there was a ton of C++ code around in the autopilot which was the software 1.0 code and then there was some neural nets in there doing image recognition and uh I kind of observed that over time as we made the autopilot better basically the neural network grew in capability and size and in addition to that all the C++ code was being deleted

[译文] [Karpathy]: 我当时的观察是，自动驾驶系统中存在大量的 C++ 代码，也就是软件 1.0 代码，然后里面有一些神经网络在做图像识别。厄，我观察到随着时间的推移，当我们改进自动驾驶系统时，基本上神经网络的能力和规模都在增长，而且除此之外，所有的 C++ 代码都在被删除。

[原文] [Karpathy]: and kind of like was um and a lot of the kind of capabilities and functionality that was originally written in 1.0 was migrated to 2.0 so as an example a lot of the stitching up of information across images from the different cameras and across time was done by a neural network and we were able to delete a lot of code and so the software 2.0 stack quite literally ate through the software stack of the autopilot

[译文] [Karpathy]: 就像是，嗯，原本用 1.0 编写的许多能力和功能都被迁移到了 2.0 中。举个例子，很多跨不同摄像头图像和跨时间的信息拼接工作都改由神经网络完成了，这使我们要能够删除大量代码。所以软件 2.0 技术栈毫不夸张地“吞噬”了自动驾驶原本的软件栈。

[原文] [Karpathy]: so I thought this was really remarkable at the time and I think we're seeing the same thing again where uh basically we have a new kind of software and it's eating through the stack we have three completely different programming paradigms

[译文] [Karpathy]: 我当时觉得这真的很了不起，我认为我们正在再次看到同样的事情发生，厄，基本上我们有了一种新型软件，它正在吞噬整个技术栈。我们现在有了三种完全不同的编程范式。

[原文] [Karpathy]: and I think if you're entering the industry it's a very good idea to be fluent in all of them because they all have slight pros and cons and you may want to program some functionality in 1.0 or 2.0 or 3.0 are you going to train neurallet are you going to just prompt an LLM should this be a piece of code that's explicit etc

[译文] [Karpathy]: 我认为如果你正在进入这个行业，精通所有这些范式是一个非常好的主意，因为它们都有各自细微的优缺点。你可能想要用 1.0、2.0 或 3.0 来编写某些功能——你是要训练一个神经网络？还是只给 LLM 写个提示词？还是这应该是一段明确的代码？等等。

[原文] [Karpathy]: so we all have to make these decisions and actually potentially uh fluidly trans transition between these paradigms

[译文] [Karpathy]: 所以我们都必须做出这些决定，并且实际上可能需要，厄，流畅地在这些范式之间进行转换。