Anthropic Co-founder Ben Mann on AGI, AI Safety, and the Future of Humanity

The future of AI: safety and superintelligence

Ben Mann, co-founder of Anthropic, discusses the rapid advancements in AI, predicting superintelligence by 2028. He emphasizes Anthropic’s commitment to AI safety and alignment, a core reason for his departure from OpenAI. Mann also shares insights on the economic and societal transformations AI will bring, stressing the importance of proactive adaptation and ethical development.

Points clés

Ben Mann, co-founder of Anthropic and former architect of GPT-3 at OpenAI, focuses on aligning AI to be helpful, harmless, and honest.
Mann predicts a 50th percentile chance of superintelligence by 2028, based on accelerating scaling laws and technological progress.
He left OpenAI due to concerns that safety was not its top priority, leading him and other safety leads to found Anthropic with a primary focus on AI alignment.
Anthropic employs “Constitutional AI” and Reinforcement Learning from AI Feedback (RLAIF) to imbue models with natural language principles, ensuring self-improvement aligns with human values like those in the UN Declaration of Human Rights.
Mann suggests that the “Economic Turing Test,” where an AI can pass for a human in a job, could define transformative AI, leading to massive shifts in global GDP and employment.
He acknowledges a 0-10% chance of an “X-risk” or extremely bad outcome from AI, emphasizing the critical need for proactive safety measures.
Anthropic is transparent about potential AI risks, including models that could aid in creating bioweapons (ASL-3 risk level), to inform policymakers.
The biggest bottlenecks to AI intelligence improvement are data centers, power, and chips, with algorithmic and efficiency improvements leading to a 10x cost decrease for intelligence.
Mann advises individuals to embrace new AI tools ambitiously and develop curiosity, creativity, and kindness to thrive in an AI-driven future.
Anthropic’s “Frontiers” team (formerly “Labs”) explores cutting-edge AI applications, aiming to differentiate the company by safely leveraging advanced research for user products.

À retenir

So, the good news is that AI is progressing at an exponential rate, and we’re all going to be living in a superintelligent future by 2028. The bad news? Your job might be obsolete, capitalism as we know it could vanish, and there’s a non-zero chance of an “X-risk” (which sounds suspiciously like a sci-fi movie plot). But don’t fret! Just embrace those AI tools, be super ambitious with them (even if they fail three times), and for heaven’s sake, teach your kids to be curious, creative, and kind. Because apparently, being nice to our future AI overlords is now a crucial life skill. Who knew?

Sources