Anthropic: Users no longer allowed to insult Claude
Set Trending Topics as a preferred source on Google.
According to heise, Anthropic has revised the usage guidelines for its chatbot Claude. They include a prohibition on users engaging in “offensive or cruel behavior” toward the artificial intelligence. The AI has reportedly been able to end a conversation since August if users behave in a hostile manner without reason.
Anthropic wants to focus on “model welfare”
According to Anthropic, this new guideline is primarily aimed at extreme cases. If users have an understandable reason for a sharp tone, or are being offensive in the context of creative work or research, that alone is not grounds for terminating a conversation. The focus is primarily on unnecessarily harmful behavior.
For Anthropic, the measure is a contribution to “model welfare” — that is, the care and consideration given to AI models. This approach is controversial within the industry. The developer of Claude says it does not know whether its AI models possess consciousness. Other companies, such as Microsoft, distance themselves very clearly from the idea of “model welfare.” In their view, AI systems should not have rights, and there is no reason to concern oneself with their “well-being.”
Claude restricts sensitive requests
In addition to offensive behavior, the new guidelines also address requests from sensitive areas such as health and finance, as well as electoral manipulation and weapons development. This is intended to enable Claude to reduce misuse.
By its own account, Anthropic has repeatedly found that Claude is being used for surveillance and to identify and track political opponents. For this reason, such use is now even more explicitly prohibited. For government institutions, journalists, and researchers, it remains permitted within a defined framework, however.
In principle, Anthropic reserves the right to override all rules in its guidelines through contracts with certain government clients, “provided that, in Anthropic’s assessment, the contractual usage restrictions and applicable safeguards are sufficient to mitigate potential harms.”
