LLMs fail at reading clocks effectively, study finds

New research from Edinburgh University shows that large language models (LLMs) struggle with basic tasks like telling time. The study tested seven popular models on their ability to interpret images of clocks and calendars. The models correctly read the time on analog clocks less than 25% of the time. They faced challenges with various clock designs, indicating difficulties in detecting clock hands and interpreting angles. Google's Gemini 2.0 performed best on clock tasks, while OpenAI's GPT-o1 was accurate 80% of the time on calendar questions. Despite these results, both models still made significant errors.