Anthropic co-founder Jack Clark wants the AI industry to hit the brakes before technology can begin to further develop itself without human input.
Speaking to BBC’, Clark said that 80 percent of Anthropic’'s coding work has already been done by AI Claude, and that this rate could rise to 100 percent ’ in a few years. But “ said it was a choice whether or not AI companies would let him go this far without stopping him.
Clark, “ said that we think this is a topic that the world needs to talk about more. “The artificial intelligence industry currently has an accelerator pedal, but the car does not have a brake pedal, and we want to do some work to create this pedal.”
According to Anthropic’ in a blog post on Thursday night, this process is called “ recursive self-improvement”, and in the process, artificial intelligence can improve itself without human contribution.
Anthropic said that in a recursive model, AI agents, which are autonomous workers created by a chatbot “, could become capable enough to build and train their own models”, so Claude’ could be continuously improved by “Claude.
Anthropic warns that recursive AI may bring some benefits to the fields of science and health, but this may mean that “ increases the risk of people losing control over artificial intelligence systems.
In his blog post, he writes: If “Systems are fully capable of creating their own successors, the way we secure them, monitor them, and shape their behavior becomes much more important.”
There is evidence that iteration will come sooner or later in the Anthropic’'s own model. This suggests that their staff's code correction rates have fallen steadily over the past year, which means there are fewer errors in what Claude’ produces.
Claude can also run his own research experiments when asked an open-ended question like “ and produce his own solutions without human intervention.
It says on the blog: “Evidence shows that the role of humans is shrinking at every step of the AI development process.”
Anthropic said the institute will conduct research to create a system that will control whether developers actually stop or slow progress toward recursive artificial intelligence.
But a real slowdown, “, will require that a large number of well-resourced laboratories in multiple countries, on or near the border, agree to stand under the same conditions.
