Experts launch exam to assess expert-level AI intelligence
Summary: A team of experts has launched "Humanity’s Last Exam," seeking challenging questions for AI systems to assess when they reach expert-level intelligence. This initiative comes after recent AI models, like OpenAI o1, excelled in standard tests.
The project is organized by the Center for AI Safety and Scale AI. It aims to create at least 1,000 difficult questions, with submissions due by November 1. Winning entries will receive prizes and co-authorship.
The exam will focus on abstract reasoning and will not include questions about weapons. Organizers hope to ensure that AI responses are not based on memorized answers from existing datasets.
The article metrics are deprecated.
I'm replacing the original 8-factor scoring system with a new and improved one. It doesn't use the original factors and gives much better significance scores.