Arabic.AI & Stanford Launch Arabic AI Benchmark

Arabic.AI, a regional provider of Arabic artificial intelligence and enterprise technology, has partnered with Stanford University's Center for Research on Foundation Models to launch HELM Arabic Enterprise, a new framework aimed at improving how organisations evaluate Arabic large language models.

The initiative builds on Stanford's Holistic Evaluation of Language Models (HELM), an open-source framework designed to provide transparent and reproducible assessments of AI systems.

HELM Arabic Enterprise introduces a structured benchmark for comparing model performance across six enterprise-focused tasks, including content generation, financial reasoning and legal question answering. Prompts, responses, metrics and scores are made available through the open-source HELM framework to support greater transparency and consistency.

According to the two organisations, the benchmark is intended to provide businesses with a common baseline for internal evaluations, vendor comparisons and ongoing model oversight.

Stanford's Center for Research on Foundation Models is known for developing the original HELM framework, which has become a widely used standard for evaluating language models.