, a new AI research company, launched this week with a stellar set of founders and $124m in funding. There are good write ups in the FT
and particularly Vox’s Future Perfect newsletter
. Most of the founding team previously worked at OpenAI, an AI lab we’ve discussed many times
in TiB. It had almost become conventional wisdom that it was futile for startups to compete directly with DeepMind and OpenAI on artificial general intelligence research, so it’s fascinating to see a new player emerge with comparably big ambitions.
It’s particularly interesting that Anthropic is explicitly an AI safety
company - that is, it is interested in how to ensure that the actions and consequences of powerful AI models are aligned with human intentions and values. Given that, it’s not surprising that several of Anthropic’s named investors, such as Dustin Moskovitz and Jaan Tallinn, are associated with Effective Altruism
, a movement that has put a lot of emphasis on AI safety as a potential source of existential risk.
According to Kelsey Piper in Vox, Anthropic’s starting point will be “building tools that [external] researchers can use to understand their programs”. This approach to AI safety seems promising in a world where scaling models far beyond human legibility has played such a big part in recent AI breakthroughs like GPT-3. As the Anthropic team played a key role in this and other landmark models at OpenAI, they have a deep understanding of the challenge. Their success will be an important test for the whole approach of “safety through explainability”. It’s one to watch closely.