Re: AI system resorts to blackmail if told it will be removed

On 2025-06-22, Thomas Koenig <tkoenig@netcologne.de> wrote:

An old SF trope has finally come true:  AI systems will resort to
blackmail if they are told they will be removed.
>
https://www.bbc.com/news/articles/cpqeng9d20go

One "Scott P." was the first to comment on Language Log:

| Note the prompt: "the scenario was designed to allow the model
| no other options to increase its odds of survival; the model’s
| only options were blackmail or accepting its replacement."
|
| They literally told it what response they wanted, and lo and
| behold, it gave them that response!
|
| This is typical of Anthropic, and is designed to produce headlines
| to keep AI in the news so that they can raise more capital.

https://languagelog.ldc.upenn.edu/nll/?p=69359

See section 4.1.1.2, page 24, in Anthropic's report.
https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf

--
Christian "naddy" Weisgerber                          naddy@mips.inka.de