Rishub Jain leaves Google DeepMind to launch London AI safety nonprofit

The new organization will build datasets and evaluation systems for human and AI oversight, with work starting in mid-August and at least six research roles planned.

Rishub Jain has left Google DeepMind after seven years to launch a London-based AI safety research nonprofit

Rishub Jain has left Google DeepMind after seven years to establish a London-based AI safety nonprofit focused on scalable oversight and the development of stronger systems for evaluating artificial intelligence.

The organization will begin work in mid-August and plans to build datasets that measure how effectively humans and AI systems can verify or oversee model behavior. Jain said the research will then be used to improve evaluation systems and deploy them in AI testing, training, and live-use settings.

Jain will start the nonprofit with Joshua Jacob and Alex Adams. The founding team plans to recruit at least six additional researchers in London.

Candidates will be expected to have previous AI research experience and an interest in safety and evaluations. Jain has not yet announced the nonprofit’s name, funding arrangements, or full recruitment timetable.

Before the organization begins work, Jain presented research on human-AI oversight at the ACM Conference on Fairness, Accountability, and Transparency in Montreal on June 28.

Nonprofit will build AI oversight benchmarks

The research program will focus on scalable and human oversight, including the development of systems capable of assessing AI outputs as models become more capable.

Jain wrote on LinkedIn: "We’ll be building datasets to measure the ability to verify/oversee, then hill-climbing that leaderboard with both Human and AI systems, and then ultimately deploying our improved judges in real eval+training+deployment scenarios."

The planned datasets will test how well people and AI systems identify problems, assess outputs, and reach reliable judgments. The organization intends to use the results to create a leaderboard and improve the performance of what Jain describes as AI "judges".

The work is expected to cover evaluation and monitoring during model development and deployment. Jain’s existing research has examined human-AI complementarity, where people and AI systems combine their different capabilities to identify errors or harmful content more effectively than either could alone.

A related Cambridge AI Safety Hub project led by Jain has focused on identifying harm across a range of tasks. Its planned workstreams include developing forward-looking harm datasets, testing methods for allocating subtasks between people and AI, and building an open-source human rating platform.

Research builds on seven years at Google DeepMind

Jain joined Google DeepMind as a Research Engineer in July 2019 and left in June 2026.

His early work at Google DeepMind included AlphaFold between 2019 and 2022. He later moved into responsibility and safety research focused on scalable oversight, followed by work on information quality and Gemini security.

Jain is listed as an author on research covering AlphaFold, AlphaFold-Multimer, AlphaFold 3, the Gemini model family, democratic deliberation, and technical approaches to artificial general intelligence safety and security.

Outside Google DeepMind, Jain has served as a mentor with the Supervised Program for Alignment Research since September 2025. Work conducted through the program includes the paper "Toward Human-AI Complementarity Across Diverse Tasks", which extends previous research on combining human and AI judgments beyond factuality assessments.

The nonprofit follows Jain’s work on "Human-AI Complementarity: A Goal for Amplified Oversight", which he is due to present at FAccT 2026 in Montreal. The session is scheduled for June 28 from 10:45 am to 12:15 pm.

Jain’s earlier experience also includes machine learning research at Carnegie Mellon University and internships with Apple and Uber Advanced Technologies Group.