ARIA (Advanced Research and Invention Agency) have launched their second programme, Safeguarded AI, led by Programme Director David ‘davidad’ Dalrymple.

By combining scientific world models and mathematical proofs, ARIA aims to develop quantitative safety guarantees for AI (in the way we might expect in nuclear power and passenger aviation). The programme explores a pathway for constructing a targeted ‘gatekeeper’ AI, whose job it would be to ensure other AIs only operate within agreed-upon guardrails for a given application (e.g. balancing electricity grids or optimising clinical trials) and thus allow us to safely harness the problem-solving abilities of advanced AI systems.

To do this, ARIA will direct £59 million across thee three technical areas over a period of three to five years. This first solicitation focuses on Technical Area 1.1 of the Programme. Through TA1.1, ARIA are looking to research and construct computationally practicable mathematical representations and formal semantics to support world-models, specifications about state-trajectories, neural systems, proofs that neural outputs validate specifications, and “version control” (incremental updates or “patches”) thereof.

Find out more and apply here. Applications close 28 May 2024. If you have any questions about the solicitation, please contact clarifications@aria.org.uk.