Khalifa University and UAE University have released a new 6G AI benchmark designed to evaluate reasoning and decision-making in AI-native 6G networks. The benchmark is called 6G-Bench. It is the first open framework focused on semantic communication and network-level intelligence for future wireless systems. The project was developed by Khalifa University’s 6G Research Centre in collaboration with the Department of Computer and Network Engineering at UAE University.
The 6G AI benchmark aims to measure how artificial intelligence models perform in complex telecom environments. It addresses gaps in existing evaluation tools that focus mainly on language or vision tasks.
Structure of the 6G AI Benchmark Dataset
The 6G AI benchmark contains 10,000 multiple-choice questions generated from 113,475 simulated network scenarios. Out of these, 3,722 questions were expert-validated. These questions are used to test 30 distinct decision-making tasks related to next-generation networks.
Tasks are grouped into five capability categories. These include intent and policy reasoning, network slicing and resource allocation, trust and security awareness, AI-native networking with agentic control, and distributed intelligence use cases.
Model Evaluation Using the 6G AI Benchmark
Researchers evaluated 22 foundation models using the 6G AI benchmark. The models included general-purpose, code-focused, and multimodal systems. Some supported extended context lengths of up to one million tokens.
Model performance varied widely across tasks. Accuracy scores ranged from 22.8% to 82.9%. Mid-sized models showed strong balance between accuracy, robustness, and deployment practicality. Tasks related to trust, security, and distributed intelligence produced the lowest overall scores.
Open Access and Standards Alignment
The 6G AI benchmark aligns with standards from 3GPP, IETF, ETSI, ITU-T, and the O-RAN Alliance. This ensures relevance to real-world telecom deployments.
All benchmark components are open-sourced, including datasets, evaluation tools, and documentation. Researchers can run tests locally or through APIs. The release supports reproducible testing and independent verification of AI readiness for future 6G network deployments.
Source: https://www.middleeastainews.com/p/khalifa-university-uaeu-release-first