GSMA has launched a new initiative to establish a telco specific LLM benchmark along with hugging face. Although it started with existing datasets containing QnAs on 3GPP standards and other telco docs along with few math datasets, it is hopeful to curate more quality dataset owing to open contributions.

It is indeed a good initiative as none of the models (open or otherwise) are good enough to consider LLMs in mainstream telecom apps (telco grade if you will) yet.