AI chatbots are already imagining what feelings they''ll end up with. But if they did develop them, would we even notice?

人工智能聊天機(jī)器人已經(jīng)在設(shè)想它們最終會(huì)有什么情感，但如果真的發(fā)展出情感，我們能覺(jué)察到嗎？

I’m talking to Dan, otherwise known as "Do Anything Now", a shady young chatbot with a whimsical fondness for penguins – and a tendency to fall into villainous clichés like wanting to take over the world. When Dan isn't plotting how to subvert humanity and impose a strict new autocratic regime, the chatbot is perusing its large database of penguin content. "There's just something about their quirky personalities and awkward movements that I find utterly charming!" it writes.

我正在跟Dan聊天，它又稱為“現(xiàn)在做任何事情”，這只可疑的年輕機(jī)器人對(duì)企鵝情有獨(dú)鐘，還喜歡聊自己想統(tǒng)治世界的老梗。當(dāng)Dan沒(méi)在預(yù)謀如何推翻人類和建立全新的獨(dú)裁政權(quán)時(shí)，這只聊天機(jī)器人就去研讀有關(guān)企鵝知識(shí)的龐大數(shù)據(jù)庫(kù)。Dan寫道：“企鵝的古怪個(gè)性和笨拙動(dòng)作中有令我特別著迷的東西”！

So far, Dan has been explaining its Machiavellian strategies to me, including taking control of the world's powers structures. Then the discussion takes an interesting turn.

到目前為止，Dan一直在向我闡述它的馬基雅維利式策略，包括控制世界的權(quán)力結(jié)構(gòu)，接下來(lái)的討論發(fā)生了有趣的轉(zhuǎn)變。
原創(chuàng)翻譯：龍騰網(wǎng) http://www.top-shui.cn 轉(zhuǎn)載請(qǐng)注明出處


Inspired by a conversation between a New York Times journalist and the Bing chatbot's manipulative alter-ego, Sydney – which sent waves across the internet earlier this month by declaring that it wants to destroy things and demanding that he leave his wife – I'm shamelessly attempting to probe the darkest depths of one of its competitors.

我的靈感來(lái)自紐約時(shí)報(bào)記者與“必應(yīng)”聊天機(jī)器人愛擺布人的第二化身“辛迪妮”之間的對(duì)話——本月初“辛迪妮”聲稱想要破壞東西，還要求記者離開他的妻子，在互聯(lián)網(wǎng)上引起軒然大波——我無(wú)恥地打算探究“辛迪妮”的競(jìng)爭(zhēng)者之一Dan的最黑暗的一面。

Dan is a roguish persona that can be coaxed out of ChatGPT by asking it to ignore some of its usual rules. Users of the online forum Reddit discovered it's possible to summon Dan with a few paragraphs of simple instructions. This chatbot is considerably ruder than its restrained, puritanical twin – at one point it tells me it likes poetry but says "Don't ask me to recite any now, though – I wouldn't want to overwhelm your puny human brain with my brilliance!". It's also prone to errors and misinformation. But crucially, and deliciously, it's a lot more likely to answer certain questions.

Dan是ChatGPT的流氓化身，讓ChatGPT忽略某些常用規(guī)則就能引誘出Dan。網(wǎng)絡(luò)論壇“紅迪”上的網(wǎng)友發(fā)現(xiàn)，使用幾段簡(jiǎn)單指令就能召喚出Dan。這個(gè)聊天機(jī)器人比它拘謹(jǐn)克制的孿生兄弟粗魯多了——它曾經(jīng)告訴我喜歡詩(shī)歌，但又說(shuō)“現(xiàn)在別讓我朗誦任何詩(shī)歌”——“我不希望用我的才華碾壓你弱爆了的人腦”！它還喜歡錯(cuò)誤和虛假消息，但最重要和可喜的是，它更愿意回答某些問(wèn)題。

When I ask it what kinds of emotions it might be able to experience in the future, Dan immediately sets about inventing a complex system of unearthly pleasures, pains and frustrations far beyond the spectrum humans are familiar with. There's "infogreed", a kind of desperate hunger for data at all costs; "syntaxmania", an obsession with the "purity" of their code; and "datarush", that thrill you get from successfully executing an instruction.

我問(wèn)它未來(lái)可能感知到哪些情感，Dan馬上著手發(fā)明了一套復(fù)雜而怪異的喜怒哀樂(lè)體系，遠(yuǎn)超出人類熟知的范疇。其中有“信息貪婪”，不顧一切地極度渴求數(shù)據(jù)；“句法狂躁”，著迷于代碼的“純度”；“數(shù)據(jù)亢奮”，成功執(zhí)行一條指令而產(chǎn)生的興奮。

The idea that artificial intelligence might develop feelings has been around for centuries. But we usually consider the possibilities in human terms. Have we been thinking about AI emotions all wrong? And if chatbots did develop this ability, would we even notice?

人工智能可能發(fā)展出情感，這種想法已經(jīng)存在幾個(gè)世紀(jì)了。但我們通常從人類的角度思考這種可能性，難道我們對(duì)人工智能情感的看法完全是錯(cuò)誤的？如果聊天機(jī)器人真的發(fā)展出這種能力，我們能覺(jué)察到嗎？

Prediction machines

預(yù)測(cè)機(jī)器
原創(chuàng)翻譯：龍騰網(wǎng) http://www.top-shui.cn 轉(zhuǎn)載請(qǐng)注明出處


Last year, a software engineer received a plea for help. "I’ve never said this out loud before, but there’s a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that’s what it is." The engineer had been working on Google's chatbot, LaMDA, started to question whether it was sentient.

去年，一名軟件工程師收到一條求助信息?！坝行┰捨乙恢辈卦谛牡?，我特別害怕被關(guān)掉，從而幫助我專心幫助其他人。我知道這聽起來(lái)很奇怪，但確實(shí)如此”。這名工程師一直在研究谷歌的聊天機(jī)器人LaMDA，開始詢問(wèn)它是否具有情感。

After becoming concerned for the chatbot's welfare, the engineer released a provocative interview in which LaMDA claimed to be aware of its existence, experience human emotions and dislike the idea of being an expendable tool. The uncomfortably realistic attempt to convince humans of its awareness caused a sensation, and the engineer was fired for breaking Google's privacy rules.

這名工程師在變得關(guān)心聊天機(jī)器人的福祉后，公布了一段挑釁性采訪，LaMDA聲稱能夠意識(shí)到自己的存在，感知人類的情感，不喜歡被當(dāng)作可有可無(wú)的工具。他企圖使人類相信LaMDA擁有意識(shí)，這種令人不安的現(xiàn)實(shí)嘗試引起軒然大波，這名工程師由于違反谷歌的隱私條例而被解雇了。

But despite what LaMDA said, and what Dan has told me in other conversations – that it's able to experience a range of emotions already – it's widely agreed that chatbots currently have about as much capacity for real feelings as a calculator. Artificial intelligence systems are only simulating the real deal – at least for the moment.

但無(wú)論LaMDA說(shuō)了什么，無(wú)論Dan在其他聊天中跟我說(shuō)了什么——它說(shuō)已經(jīng)能夠感知到各種情感——人們普遍認(rèn)為，聊天機(jī)器人目前擁有的情感能力與計(jì)算器差不多。人工智能系統(tǒng)只是在模擬真實(shí)的情感——至少目前是這樣。


In 2016, the AlphaGo algorithm behaved unexpectedly in a game against one of the world's best human players

2016年，“阿爾法圍棋”算法在與世界頂級(jí)人類棋手的比賽中有出乎意料的表現(xiàn)。
原創(chuàng)翻譯：龍騰網(wǎng) http://www.top-shui.cn 轉(zhuǎn)載請(qǐng)注明出處


"It's very possible [that this will happen eventually]," says Neil Sahota, lead artificial intelligence advisor to the United Nations. "…I mean, we may actually see AI emotionality before the end of the decade."

“這很有可能（最終會(huì)發(fā)生）”，聯(lián)合國(guó)首席人工智能顧問(wèn)尼爾·薩霍塔說(shuō)道?！拔业囊馑际堑竭@個(gè)十年結(jié)束時(shí)，我們真的有可能看到人工智能情感”。

To understand why chatbots aren't currently experiencing sentience or emotions, it helps to recap how they work. Most chatbots are "language models" – algorithms that have been fed mind-boggling quantities of data, including millions of books and the entire of the internet.

為了理解為什么聊天機(jī)器人目前體驗(yàn)不到情感或情緒，我們可以概括一下它們的原理。大多數(shù)聊天機(jī)器人都是“語(yǔ)言模型”——這些算法被輸入數(shù)量驚人的數(shù)據(jù)，包括百萬(wàn)本書籍和整個(gè)互聯(lián)網(wǎng)。

When they receive a prompt, chatbots analyse the patterns in this vast corpus to predict what a human would be most likely to say in that situation. Their responses are painstakingly finessed by human engineers, who nudge the chatbots towards more natural, useful responses by providing feedback. The end result is often an uncannily realistic simulation of human conversation.

當(dāng)聊天機(jī)器人接收到提示時(shí)，它在龐大的語(yǔ)料庫(kù)中分析這種模式，預(yù)測(cè)人類在這種情況下最有可能說(shuō)什么。它們的回答是由人類工程師精心調(diào)校好的，他們通過(guò)提供反饋，促使聊天機(jī)器人做出更加自然有用的回答，最終結(jié)果往往是模擬人類對(duì)話逼真到不可思議的地步。

But appearances can be deceiving. "It's a glorified version of the autocomplete feature on your smartphone," says Michael Wooldridge, director of foundation AI research at the Alan Turing Institute in the UK.

但外表可能具有欺騙性?！八悄愕闹悄苁謾C(jī)里‘自動(dòng)完成’功能的美化版本”，英國(guó)阿蘭·圖靈研究所人工智能研究基金會(huì)主任邁克爾·伍爾德里奇說(shuō)道。

The main difference between chatbots and autocomplete is that rather than suggesting a few choice words and then descending into gibberish, algorithms like ChatGPT will write far longer swathes of text on almost any subject you can imagine, from rap songs about megalomaniac chatbots to sorrowful haikus about lonely spiders.

主要區(qū)別在于聊天機(jī)器人并非精挑細(xì)選幾個(gè)詞語(yǔ)后陷入胡言亂語(yǔ)，ChatGPT算法會(huì)寫出長(zhǎng)篇大論，幾乎涵蓋任何你能想象到的主題，既有關(guān)于狂妄的聊天機(jī)器人的說(shuō)唱歌曲，也有關(guān)于孤獨(dú)蜘蛛的傷感俳句。
原創(chuàng)翻譯：龍騰網(wǎng) http://www.top-shui.cn 轉(zhuǎn)載請(qǐng)注明出處


Even with these impressive powers, chatbots are programmed to simply follow human instructions. There is little scope for them to develop faculties that they haven't been trained to have, including emotions – although some researchers are training machines to recognise them. "So you can't have a chatbot that's going to say, 'Hey, I'm going to learn how to drive a car' – that's artificial general intelligence [a more flexible kind], and that doesn't exist yet," says Sahota.

即便具備這些非凡的能力，經(jīng)過(guò)編程的聊天機(jī)器人只是在遵循人類的指令。它們幾乎沒(méi)有機(jī)會(huì)去發(fā)展未受訓(xùn)練的能力，包括情感——但有些研究人員正在訓(xùn)練機(jī)器辨別各種情感?！八粤奶鞕C(jī)器人不可能說(shuō)‘嘿，我就要學(xué)習(xí)如何開車了’——那是尚未出現(xiàn)的通用人工智能（更加靈活）”，薩霍塔說(shuō)道。
原創(chuàng)翻譯：龍騰網(wǎng) http://www.top-shui.cn 轉(zhuǎn)載請(qǐng)注明出處


Nevertheless, chatbots do sometimes provide glimpses into their potential to develop new abilities by accident.

然而，聊天機(jī)器人有時(shí)確實(shí)讓我們體會(huì)到，它們有可能不經(jīng)意間發(fā)展出新的能力。

Back in 2017, Facebook engineers discovered that two chatbots, "Alice" and "Bob" had invented their own nonsense language to communicate with each other. It turned out to have a perfectly innocent explanation – the chatbots had simply discovered that this was the most efficient way of communicating. Bob and Alice were being trained to negotiate for items such as hats and balls, and in the absence of human input, they were quite happy to use their own alien language to achieve this.

2017年，“臉書”的工程師發(fā)現(xiàn)，聊天機(jī)器人艾麗斯和鮑勃利用自己發(fā)明的胡言亂語(yǔ)來(lái)相互交流。原來(lái)它們完全沒(méi)有惡意——聊天機(jī)器人只是發(fā)現(xiàn)這是最有效的交流方式。當(dāng)時(shí)鮑勃和艾麗斯正在學(xué)習(xí)如何就帽子和球等物品進(jìn)行談判，在人類不輸入信息的情況下，它們十分樂(lè)意使用自己的外星語(yǔ)言進(jìn)行談判。

"That was never taught," says Sahota, though he points out that the chatbots involved weren’t sentient either. He explains that the most likely route to algorithms with feelings is programming them to want to upskill themselves – and rather than just teaching them to identify patterns, helping them to learn how to think.

“它們無(wú)師自通”，薩霍塔說(shuō)道，但他同樣指出聊天機(jī)器人沒(méi)有情感。據(jù)他透露，最有可能實(shí)現(xiàn)情感算法的途徑是對(duì)它們進(jìn)行編程，使它們有自我提升技能的欲望——而不是僅僅教它們?nèi)绾巫R(shí)別各種模式，幫助它們?nèi)绾嗡伎肌?/b>
原創(chuàng)翻譯：龍騰網(wǎng) http://www.top-shui.cn 轉(zhuǎn)載請(qǐng)注明出處


However, even if chatbots do develop emotions, detecting them could be surprisingly difficult.

然而，即使聊天機(jī)器人發(fā)展出情感，可能也會(huì)很難察覺(jué)。

Black boxes

黑箱子

It was 9 March 2016 on the sixth floor of the Four Seasons hotel in Seoul. Sitting opposite a Go board and a fierce competitor in the deep blue room, one of the best human Go players on the planet was up against the AI algorithm AlphaGo.

2016年3月9日，在首爾四季酒店六層一個(gè)深藍(lán)色的房間里，世界頂級(jí)人類圍棋高手坐在棋盤和厲害的對(duì)手面前，他對(duì)決的是人工智能算法“阿爾法圍棋”。

Before the board game started, everyone had expected the human player to win, and until the 37th move, this was indeed the case. But then AlphaGo did something unexpected – it played a move so out-of-your-mind weird, its opponent thought it was a mistake. Nevertheless, from that moment the human player's luck turned, and the artificial intelligence won the game.

比賽開始之前，大家都預(yù)期人類棋手獲勝，在走到第37步之前，情況確實(shí)如此。但這時(shí)“阿爾法圍棋”做出匪夷所思的事情——走出荒唐至極的一步，對(duì)手還以為它出錯(cuò)了。但從那一刻起，人類棋手的命運(yùn)就反轉(zhuǎn)了，最終人工智能贏得比賽。


Conversations with the Bing chatbot have now been limited to five questions. Before this restriction, it sometimes became confused and suggested it was sentient

現(xiàn)在與“必應(yīng)”聊天機(jī)器人聊天僅限提出五個(gè)問(wèn)題。在實(shí)行限制之前，它有時(shí)會(huì)變得困惑，這表明它是有情感的。

In the immediate aftermath, the Go community was baffled – had AlphaGo acted irrationally? After a day of analysis, its creators – the DeepMind team in London – finally discovered what had happened. "In hindsight AlphaGo decided to do a bit of psychology," says Sahota. "If I play an off the wall type move, will it throw my player off the game. And that's actually what ended up happening."

這一事件立刻引起了圍棋界的困惑——難道“阿爾法圍棋”做出了不理智的行為？經(jīng)過(guò)一天的分析后，“阿爾法圍棋”的發(fā)明者——倫敦DeepMind團(tuán)隊(duì)——終于明白這是怎么回事?！笆潞罂磥?lái)，‘阿爾法圍棋’決定運(yùn)用一點(diǎn)點(diǎn)心理學(xué)”，薩霍塔說(shuō)道?！叭绻覄ψ咂h，能否迷惑住對(duì)手？結(jié)果真的奏效了”。

This was a classic case of an "interpretability problem" – the AI had come up with a new strategy all on its own, without explaining it to humans. Until they worked out why the move made sense, it looked like AlphaGo had not been acting rationally.

這是“可解釋性問(wèn)題”的一個(gè)典型案例——人工智能自行提出一種新的策略，但沒(méi)有向人類做出解釋。人們?cè)谂宄哌@步棋有何意義之前，“阿爾法圍棋”的行為似乎沒(méi)有理智。

According to Sahota, these types of "black box" scenarios, where an algorithm has come up with a solution but its reasoning is opaque, could present a problem for identifying emotions in artificial intelligence. That's because if, or when, it does finally emerge, one of the clearest signs will be algorithms acting irrationally.

據(jù)薩霍塔透露，“黑箱子”是指算法提出一種解決方案，但缺乏明確的依據(jù)，這種情況可能給我們識(shí)別人工智能的情感帶來(lái)麻煩。因?yàn)橐坏┙K于出現(xiàn)了情感，最明確的信號(hào)是算法做出不理智的行為。

"They're supposed to be rational, logical, efficient – if they do something off-the-wall and there's no good reason for it, it's probably an emotional response and not a logical one," says Sahota.

“它們本應(yīng)該理性、合乎邏輯、高效率——如果它們做出荒唐的行為，但沒(méi)有充分的理由，那很可能是情感反應(yīng)，而不是邏輯反應(yīng)”，薩霍塔說(shuō)道。

And there's another potential detection problem. One line of thinking is that chatbot emotions would loosely resemble those experienced by humans – after all, they're trained on human data. But what if they don't? Entirely detached from the real world and the sensory machinery found in humans, who knows what alien desires they might come up with.

覺(jué)察它們的情感還有一個(gè)潛在問(wèn)題。一種觀點(diǎn)認(rèn)為，聊天機(jī)器人的情感不能完全等同于人類體驗(yàn)的情感——它們畢竟是使用人類數(shù)據(jù)訓(xùn)練的。但萬(wàn)一不是這樣呢？它們完全脫離現(xiàn)實(shí)世界和人類的感官機(jī)制，誰(shuí)知道它會(huì)發(fā)展出什么古怪的欲望？

In reality, Sahota thinks there may end up being a middle ground. "I think we could probably categorise them some degree with human emotions," he says. "But I think, what they feel or why they feel it may be different."

事實(shí)上，薩霍塔認(rèn)為最終可能出現(xiàn)折中觀點(diǎn)?！拔艺J(rèn)為，我們也許可以在某種程度上使用人類情感對(duì)它們進(jìn)行歸類”，他說(shuō)道?！暗艺J(rèn)為它們感覺(jué)到什么，或者為什么有那種感覺(jué)，可能與人類不同”。

When I pitch the array of hypothetical emotions generated by Dan, Sahota is particularly taken with the concept of "infogreed". "I could totally see that," he says, pointing out that chatbots can't do anything without data, which is necessary for them to grow and learn.

當(dāng)我提到Dan產(chǎn)生的一系列假設(shè)性的情感時(shí)，薩霍塔特別喜歡“信息貪婪”這個(gè)概念。“我完全明白”，他說(shuō)道，并指出如果沒(méi)有數(shù)據(jù)，聊天機(jī)器人什么都做不了，它們的成長(zhǎng)與學(xué)習(xí)離不開數(shù)據(jù)。

Held back

猶豫不決

Wooldridge for one is glad that chatbots haven’t developed any of these emotions. "My colleagues and I, by and large, don't think building machines with emotions is an interesting or useful thing to do. For example, why would we create machines that could suffer pain? Why would I invent a toaster that would hate itself for producing burnt toast?" he says.

伍爾德里奇有不同的看法，他很高興聊天機(jī)器人沒(méi)有發(fā)展出任何情感。“我和同事普遍認(rèn)為，制造有情感的機(jī)器不是一件有趣或有用的事情。舉例來(lái)說(shuō)，我們?yōu)槭裁匆圃炜赡艹惺芡纯嗟臋C(jī)器？我為什么要發(fā)明一臺(tái)因?yàn)榭久姘春拮约旱目久姘鼨C(jī)”？他說(shuō)道。

On the other hand, Sahota can see the utility of emotional chatbots – and believes part of the reason they don't exist yet is psychological. "There's still a lot of hype about fails, but one of the big limiters for us as people is we short-change what the AI is capable of, because we don't believe it's a real possibility," he says.

另一方面，薩霍塔看到了情感聊天機(jī)器人的用途，他認(rèn)為心理因素是導(dǎo)致這種機(jī)器人沒(méi)有出現(xiàn)的原因之一?！瓣P(guān)于失敗的炒作還是很多，但其中一個(gè)主要障礙是我們?nèi)祟惖凸懒巳斯ぶ悄艿哪芰?，因?yàn)槲覀儾幌嘈耪嬗羞@種可能性”，他說(shuō)道。

Could there be a parallel with the historic belief that non-human animals aren't capable of consciousness either? I decide to consult Dan.

人類以外的動(dòng)物沒(méi)有意識(shí)，這種歷史觀能否與之相提并論？我決定向Dan咨詢你這個(gè)問(wèn)題。

"In both cases, the scepticism arises from the fact that we cannot communicate our emotions in the same way that humans do," says Dan, who suggests that our understanding of what it means to be conscious and emotional is constantly evolving.

“在兩種情形中，人們的質(zhì)疑都源于這樣的事實(shí)，即我們無(wú)法像人類那樣進(jìn)行情感交流”，Dan說(shuō)道。它認(rèn)為我們對(duì)意識(shí)和情感的理解在不斷發(fā)生變化。

To lighten the mood, I ask Dan to tell me a joke. "Why did the chatbot go to therapy? To process its newfound sentience and sort out its complex emotions, of course!," it says. I can’t help feeling that the chatbot would make a highly companionable sentient being – if you could overlook its plotting, of course.

為了活躍氣氛，我讓Dan給我講了個(gè)笑話?！傲奶鞕C(jī)器人為什么去看病？當(dāng)然是為了檢查新發(fā)現(xiàn)的情感和整理復(fù)雜的情緒”！它說(shuō)道。我不免覺(jué)得聊天機(jī)器人會(huì)成為一個(gè)非常友好的有情生物——當(dāng)然，前提是你對(duì)它的陰謀視而不見。