How difficult is AI alignment? | Anthropic Researc

章节 1：实用主义对齐观——与其追求“哲人王”，不如拥抱不确定性

📝 本节摘要：

访谈开始，主持人Alex介绍了来自Anthropic不同团队（社会影响、对齐科学、微调、可解释性）的研究员。Amanda首先分享了她对“对齐（Alignment）”的看法。她反对过度理论化或试图定义完美的“效用函数”，主张采取实用主义路径：设定一个“足够好”的基准并进行迭代。她认为模型不应被灌输某种固定的、绝对自信的道德观（类似“哲人王”），而应像人类一样，对道德框架保持不确定性，并能根据情境和新信息进行更新，这样才更安全。

[原文] [Alex]: We are super excited to have everyone here, folks that we've met already, folks that are new. This panel is just gonna be really casual. And we have researchers from four different teams at Anthropic. We have folks from Societal Impacts, that's me. Folks from Alignment Science, that's Jan. Alignment, Finetuning, Amanda. And interpretability, Josh. I'm gonna start with asking a question to Amanda, from the Alignment Finetuning team. And I want you to talk a little bit about how you see alignment, what it means to you.

[译文] [Alex]: 我们非常高兴大家能来到这里，无论是我们已经见过的朋友，还是新面孔。这场圆桌讨论会非常随意。我们有来自Anthropic四个不同团队的研究人员。有来自社会影响（Societal Impacts）团队的，那就是我。有来自对齐科学（Alignment Science）团队的，那是Jan。来自对齐微调（Alignment Finetuning）的，Amanda。还有可解释性（Interpretability）团队的，Josh。我要先问来自对齐微调团队的Amanda一个问题。我想请你谈谈你是如何看待对齐的，它对你来说意味着什么。

[原文] [Alex]: You know, 'cause you're in charge of a lot of our work on how the model should behave. And why should you be the philosopher king that decides how Claude behaves, what its characteristics and attributes are?

[译文] [Alex]: 你知道，因为你负责了很多关于模型应该如何表现的工作。既然如此，为什么你应该成为决定Claude如何表现、具备什么特征和属性的“哲人王（philosopher king）”呢？

[原文] [Amanda]: I mean ask Plato. He's the one that decided I should be the philosopher king. (Alex laughing) The question of, like, what is alignment, maybe this is, like, a slightly spicy view that I have. I think people are very, very tempted to spend a lot of time trying to define this concept because there's, like, lots of ways of doing it

[译文] [Amanda]: 我想你得去问柏拉图。是他决定了我应该成为哲人王的。（Alex笑）关于什么是对齐这个问题，也许我有一个稍微激进点（spicy）的观点。我认为人们非常、非常容易把大量时间花在试图定义这个概念上，因为确实有很多定义方式，

[原文] [Amanda]: and they, you know, I dunno they have, like, social choice theory in the back of their head and they're like, "Oh, well if you imagine everyone has a utility function, there's, like, limits on exactly what you can say about how to maximize all of those utility functions, et cetera." And I think I'm more inclined to just be like, we kind of want things to go well enough that you can iterate on and improve them later. And the bar isn't some like perfect notion of alignment. I'm sure there is that concept. One can define it, one can argue about it

[译文] [Amanda]: 比如，你知道，我不确定，他们脑子里可能有社会选择理论（social choice theory），然后他们会想，“哦，如果你想象每个人都有一个效用函数（utility function），那么关于如何最大化所有这些效用函数等等，你能说的其实是有限制的。”而我认为我更倾向于这种态度：我们只是希望事情进展得足够顺利，以便后续可以迭代和改进。标准并不是某种完美的对齐概念。我确信存在那个概念。人们可以定义它，可以争论它，

[原文] [Amanda]: but for the most part the initial goal is, like, let's just make things like go well and, like, meet a certain kind of, like, lower bar where that is, like, you know, if it's not perfect, if some people, like, don't like it, you can just improve on it. So, like, my view of of alignment is actually probably, like, I mostly want to, like, hit that, like, and iterate from there. In terms of, like, how the model should behave and how I think about that, I think I've spoken about this before but my basic concept right now

[译文] [Amanda]: 但在很大程度上，最初的目标就是，让我们先把事情做好，达到某种最低标准，即如果它不完美，如果有些人不喜欢它，你可以直接改进它。所以，我对对齐的看法实际上可能是，我主要想达到那个标准，然后从那里开始迭代。至于模型应该如何表现以及我如何思考这个问题，我想我之前谈过这个，但我目前的基本概念是，

[原文] [Amanda]: for the model is trying to get it to behave the way that I think, like, a very good, like, morally motivated, like, kind human would act if they found themselves roughly in this circumstance. It's a little bit strange 'cause they also have to find themselves in the circumstances of, like, being an AI (panelists laughing) who's like talking to millions of people, which does in fact affect how you behave. Like, maybe you would normally be willing to, like, just chitchat about politics with someone, but if you're gonna be talking with millions of people,

[译文] [Amanda]: 对于模型来说，是试图让它的表现像我认为一个非常好的、有道德动力的、善良的人在大概处于这种情况下会做出的行为那样。这有点奇怪，因为它们也必须意识到自己处于“身为一个AI”的情况下（小组成员笑），正在与数百万人交谈，这实际上确实会影响你的行为。比如，也许你通常愿意和某人闲聊政治，但如果你要和数百万人交谈，

[原文] [Amanda]: maybe you'd actually be like, "Hmm, I should maybe be, like, a little bit more concerned about potentially influencing people." But I do think it's actually an important model, which is, like, sometimes people are kinda like, "Oh, what values should you put into the model?" And I think I'm often like, "Well, do we think this way with humans?" Where I'm just like, someone just injected me with, like, value serum or something and I just have these, like, fixed things that I'm, like, completely certain of. I'm like, I don't know, that seems, like,

[译文] [Amanda]: 也许你实际上会想，“嗯，我可能应该更担心一下潜在地对人们产生影响。”但我确实认为这是一个重要的模型，就像有时候人们会问，“噢，你应该把什么价值观放入模型中？”而我通常会想，“嗯，我们对人类也是这样想的吗？”就好像有人给我注射了“价值观血清”之类的东西，然后我就有了这些我完全确信的固定观念。我觉得，我不知道，这看起来，

[原文] [Amanda]: almost, like, dangerous or something. Most of us just have, like, a mix of, like, things that we do value but that we would trade off against other things and a lot of uncertainty about different, like, moral frameworks. We hit cases where we're suddenly, like, "Oh, actually my value framework doesn't accord with my intuitions," and we update. I think my view is that ethics is actually, like, a lot more like physics than people think. It's actually like a lot more kind of, like, empirical and something that we're uncertain over

[译文] [Amanda]: 几乎有点危险之类的。我们大多数人只是拥有一系列我们确实重视的东西，但我们会权衡这些东西与其它东西，并且对不同的道德框架有很多不确定性。我们会遇到这样的情况，突然觉得，“哦，实际上我的价值框架与我的直觉不符，”然后我们会更新。我认为我的观点是，伦理学实际上比人们想象的更像物理学。它实际上更像是一种经验性的东西，是我们不确定的东西，

[原文] [Amanda]: and that we have hypotheses about. Like, I think that if I met someone who was just completely confident in their moral view, there is no such moral view I could give that person that would not make me kind of terrified. Whereas if I instead have someone who's just, like, "I don't know, I'm kind of uncertain over this and I just, like, update in response to, like, new information about ethics and I, like, think through these things," that's the kind of person that feels, like, less scary to me. So at least at the moment I'm not, this isn't a claim

[译文] [Amanda]: 是我们有假设的东西。比如，我想如果我遇到一个对自己的道德观完全自信的人，无论那个人持有什么样的道德观，都会让我感到恐惧。相反，如果我遇到的人说，“我不知道，我对这个有点不确定，我只是根据关于伦理的新信息进行更新，我在思考这些事情，”那才是让我觉得不那么可怕的人。所以至少目前我不——这不是在声称

[原文] [Amanda]: that this is somehow going to, like, completely, like, align models or anything. But that's the kind of immediate goal. I realize I've talked a lot and you asked about the philosopher king question. (Alex laughing) I guess. Okay. I guess I should give a quick answer to that. There is also this question of, like, well, in the kind of, like, should you put values into the model? Maybe I've partially answered it where I'm like, "The models should just be uncertain over values that exist in the world." And so ideally it's not just someone injecting their values

[译文] [Amanda]: 这种方法能以某种方式完全对齐模型之类的。但这是一种短期目标。我意识到我说了很多，而你问的是哲人王的问题。（Alex笑）我想，好吧。我想我应该给那个问题一个快速的回答。这就涉及到一个问题，比如，你应该把价值观放入模型中吗？也许我已经部分回答了，我的观点是，“模型应该只对世界上存在的价值观保持不确定性。”所以理想情况下，不仅仅是某人在注入他们的价值观，

[原文] [Amanda]: or their preferences, nor is it something like everyone just voting on values to put into models but instead like people that are uncertain and responsive to these things, models should also be like that. So maybe that's kinda my view.

[译文] [Amanda]: 或他们的偏好，也不是像所有人投票决定把什么价值观放入模型，而是像那些对此保持不确定并能对这些事情做出反应的人一样，模型也应该那样。所以这大概就是我的观点。