VinUni Launches AI Platform to Evaluate Vietnamese Language Proficiency

VinUni University, through its AI Research Center, introduced V-Bench on July 6, a non-profit toolkit designed to evaluate the Vietnamese language capabilities of large language models (LLMs). This toolkit aims to support researchers, businesses, and organizations applying AI in Vietnam. The development of V-Bench involved a scientific council of 18 Vietnamese AI experts, both domestic and international. The dataset comprises over 40,000 questions and tasks, covering areas such as language, culture, history, digital sovereignty, information security, and specific application scenarios within Vietnam. The evaluation system features two categories with five groups of criteria: implicit culture, regional diversity, digital sovereignty and security, practical application, and AI agentic capabilities. These criteria assess the models' ability to understand language, dialects, cultural contexts, meet information security requirements, plan, utilize tools, and execute tasks in a Vietnamese environment. According to VinUni, V-Bench aims to provide an independent reference system for evaluating AI models, offering organizations and businesses additional grounds for selecting models that align with their usage needs. The toolkit was developed following common methodologies in international LLM research, ensuring compatibility in processes and terminology with global benchmark systems. Professor Duong Nguyen Vu, Vice Provost and Scientific Director of the AI Research Center at VinUni University, stated that with its comprehensive dataset and the entirely objective, non-profit contributions from a reputable global community of Vietnamese experts, V-Bench aspires to redefine AI standards in Vietnam and build a national AI reference system. "The evaluation results from V-Bench will offer a panoramic view, helping organizations and businesses discover the depth of each AI model's capabilities, thereby enabling them to select the optimal model for specific usage contexts and practical needs," he added. To date, V-Bench has published reference evaluation results for 15 LLMs. In the next phase, the development team plans to incorporate evaluation criteria related to images and audio. This includes assessing the recognition of Nôm characters, regional signs, charts, cultural heritage, voices from three regions, cultural contexts through short videos, and the ability to process lengthy Vietnamese documents exceeding 100,000 tokens, such as laws, contracts, textbooks, and regulatory texts.