- MLCommons and AI Verify have forged an alliance to develop standardized safety testing benchmarks for generative AI models.
- The collaboration aims to foster a mature safety ecosystem by engaging AI testing firms, safety institutes, auditors, and researchers.
- The initiative seeks to establish globally accepted AI safety standards to mitigate risks across diverse cultural contexts.
- The MLCommons AI Safety working group has unveiled a v0.5 proof of concept for AI Safety benchmarks.
- AI Verify will develop interoperable AI testing tools to inform an inclusive v1.0 release expected this fall.
- Efforts are underway to create an interactive testing toolkit to facilitate benchmarking and red-teaming exercises.
- The partnership between MLCommons and AI Verify invites global stakeholders to contribute to shaping future iterations of AI safety benchmarks.
Main AI News:
MLCommons and AI Verify have inked a memorandum of intent, signaling their joint commitment to pioneer a standardized framework for safety testing benchmarks within the realm of generative AI models. This strategic partnership aims to bolster global AI safety standards by fostering collaboration among AI testing firms, national safety authorities, auditors, and scholars.
“The need for a unified approach to safety testing in generative AI has garnered considerable attention within the global AI community,” remarked Peter Mattson, President of MLCommons and co-chair of the AI Safety working group. “The collaboration between MLCommons and AI Verify marks a pivotal stride towards establishing a universally accepted standard for AI safety assessments, equipped to mitigate safety risks across diverse socio-cultural landscapes and value systems.”
The joint initiative by the MLCommons AI Safety working group, comprising leading academics, industry stalwarts, policy experts, and civil society advocates, recently unveiled a v0.5 proof of concept (POC) for AI Safety benchmarks. Leveraging this foundation, AI Verify is poised to develop interoperable AI testing frameworks, laying the groundwork for an inclusive v1.0 release scheduled for autumn. Concurrently, efforts are underway to craft an interactive testing toolkit to facilitate benchmarking and red-teaming exercises.
“In spearheading the drive towards universally recognized AI safety benchmarks and testing protocols, the AI Verify Foundation is thrilled to collaborate with MLCommons,” expressed Dr. Ong Chen Hui, Chair of the Governing Committee at AI Verify Foundation. “This partnership not only reinforces trust in AI models and applications but also underscores our commitment to promoting responsible AI adoption on a global scale.“
The AI Safety working group extends an open invitation for global stakeholders to actively contribute to shaping the v1.0 AI Safety benchmark suite and subsequent iterations. Interested parties are encouraged to engage with the MLCommons AI Safety working group to play a pivotal role in advancing AI safety standards worldwide.
Conclusion:
The collaboration between MLCommons and AI Verify signifies a pivotal step towards establishing standardized AI safety benchmarks, enhancing trust in AI models, and promoting responsible AI adoption on a global scale. This initiative not only addresses critical safety concerns but also fosters collaboration among industry players, contributing to the maturation of the AI market.
