The AI Safety Institute has released its evaluations platform for testing the safety of new AI models.
Named Inspect, it provides a software library for testers in the public and private sectors to assess the specific capabilities of individual models and produce a score based on their results.
It can be used to evaluate models in a range of areas, including their core knowledge, ability to reason and autonomous capabilities, and has been released under an open source licence, making it freely available for the AI community to use.
The institute said this will provide for a more consistent approach to AI safety evaluations around the world.
Accessible approach
AI Safety Institute chair Ian Hogarth said: “Successful collaboration on AI safety testing means having a shared, accessible approach to evaluations, and we hope Inspect can be a building block for AI safety institutes, research organisations and academia.
“We have been inspired by some of the leading open source AI developers - most notably projects like GPT-NeoX, OLMo or Pythia which all have publicly available training data and OSI licensed training and evaluation code, model weights and partially trained checkpoints. This is our effort to contribute back.
“We hope to see the global AI community using Inspect to not only carry out their own model safety tests, but to help adapt and build upon the open source platform so we can produce high quality evaluations across the board.”
World leader
Secretary of State for Science, Innovation, and Technology Michelle Donelan said: “As part of the constant drumbeat of UK leadership on AI safety, I have cleared the AI Safety Institute’s testing platform - called Inspect - to be open sourced. This puts UK ingenuity at the heart of the global effort to make AI safe, and cements our position as the world leader in this space.”
Alongside the launch of Inspect, the AI Safety Institute, Incubator for AI (i.AI) and Number 10 have announced a plan to bring together leading AI talent from a range of areas to rapidly test and develop new open source AI safety tools. Further details will be announced in due course.
