research
Posted Mar 10Researcher, Recursive Self-Improvement Safety
at openai
San Francisco, United StatesOn-site
Responsibilities
- “implement monitors for data poisoning”) – prioritizing the work that is most useful to start right now.
Requirements
- ABOUT THE TEAM Preparedness is a critical Safety Research team at OpenAI, which is focused on mitigating AI threats https://openai.com/index/updating-our-preparedness-framework/ that could scale to an extreme level of severity.
- Monitoring https://openai.com/index/how-we-monitor-internal-coding-agents-misalignment/ and predicting the evolving misalignment propensities and capabilities of frontier AI systems. - Mitigation.
- - Coordination and verification: Prototype technical mechanisms for verifying compliance with potential future AI safety agreements.
- - AI R&D risk measurement: Track progress toward automation of technical staff to inform OpenAI’s near-term investments in alignment and security.
- About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence
Benefits
- YOU MIGHT THRIVE IN THIS ROLE IF YOU: - Are an exceptional technical executor. - Have strong strategic and research taste: you can prioritize effectively in domains with weak feedback loops. - Are passionate about mitigating the risks associated with recursive self-improvement. - Are driven by a desire to do whatever work most positively impacts the future of AI development. - Bonus: you have already done work in one of the domains listed above (ML research, AI alignment, AI verification etc).
Contact
- The role is wide-ranging, covering any mitigation for loss of control risk, spanning the design and implementation of better pre-deployment risk-assessment https://alignment.openai.com/prod-evals/, control measures, RSI-relevant training interventions, and turning one’s technical work into established institutional practices.
- We’ll both need to sift through large swaths of production traffic https://openai.com/index/how-we-monitor-internal-coding-agents-misalignment/ to find the most egregious misalignments, and reliably elicit tail risks before deployment. - Rigorous monitorability: Rigorous testing and red-teaming of our measurements of model misbehavior related to loss-of-control (e.g. reward hacking, sandbagging, scheming).
- This includes better https://arxiv.org/abs/2603.05706 understanding https://alignment.openai.com/accidental-cot-grading/ monitorability https://arxiv.org/abs/2512.18311, and e.g. preparing for potential losses of Chain-of-Thought monitorability. - Model behavior science: Design experiments and evaluations to understand the extent to which models are problematically misaligned, or their safety-relevant capabilities lag behind dangerous capabilities.
Additional details
- Keeping misuse safeguards, alignment tools, and security measures on track to adequately address extreme threats that might arise in the future. - Coordination.
- Setting mitigation targets by maintaining OpenAI’s preparedness framework, and partnering with other staff to achieve these targets.
- This is urgent, fast-paced work that has far-reaching implications for the company and for society.
- ABOUT THE ROLE Preparedness is hiring strong technical executors to support preparations for recursive self-improvement.
- This work relies on reasoning about problems that might exist in the future, but might not exist now; so it’s especially important that people in this role are tasteful and strategic.
- Below is a subset of our focus areas: - Scalable oversight: Establishing practices for model misbehavior monitoring and oversight which remain effective in superhuman model capability regimes, with a focus on bridging from today’s monitoring approaches to future-proof ones. - Automated auditing: As model capabilities increase, we’ll increasingly rely on automated approaches for finding the most severe forms of model misalignments.
- This may include training model organisms of misbehavior for behaviors not currently present in production, or training interventions to increase safety-relevant capabilities.
- - Maintaining and strengthening RSI safety cases: We’re especially interested in identifying and addressing blindspots of mitigation areas which we may have missed.
- Generally, our team alternates between performing rigorous hypothesis-driven research and turning our insights into interventions or control systems which impact production models, with occasional support of engineering teams. IN THIS ROLE,
- YOU WILL: - Carefully consider the problems OpenAI might face in the future and how to prepare for them.