Purpose and goal
According to the OECD, 60% of Swedish impact assessments are insufficient today. The RegTek project aims to streamline rulemaking through a pilot study evaluating the accuracy and usability of a language model for faster impact assessments. An evaluated tool can increase the trust and use of language models by regulatory bodies, while ensuring objectivity and compliance with the EU standards for AI in accordance with AI Act.
Expected effects and result
Results from the RegTek project complement and reinforce existing initiatives for impact investigations of cutting-edge technology at authorities through ESV assignments for method development and support. With this project, RISE can start building a capability for AI testing according to EU AI Act where metrology is mentioned for quality assurance and compliance with the law.
Planned approach and implementation
The goal of the RegTek project is to conduct a pilot study to evaluate the accuracy and usability of a RAG-based language model for AI regulatory impact assessments, which may lead to wider acceptance and trust in language models and more comprehensive and fair regulation. The work includes exploring how metrology, specifically, the Rasch measurement method can be used to evaluate/measure the abilities and performance of AI language models such as accuracy and precision.
The project description has been provided by the project members themselves and the text has not been looked at by our editors.
