Research Engineer / Scientist, Alignment

About the Role

You will build and run machine learning experiments to help understand and steer the behavior of powerful AI systems, making AI helpful, honest, and harmless. As a Research Engineer on Alignment Science, you'll contribute to exploratory experimental research on AI safety, focusing on risks from powerful future systems (like those designated as ASL-3 or ASL-4 under our Responsible Scaling Policy), often collaborating with teams including Interpretability, Fine-Tuning, and the Frontier Red Team.

Current Focus Topics

• Scalable Oversight: Developing techniques to keep highly capable models helpful and honest as they surpass human-level intelligence.
• AI Control: Creating methods to ensure advanced AI systems remain safe and harmless in unfamiliar or adversarial scenarios.
• Alignment Stress-testing: Creating model organisms of misalignment to improve empirical understanding of alignment failures.
• Automated Alignment Research: Building and aligning systems that can speed up and improve alignment research.
• Alignment Assessments: Documenting emerging properties of models through pre-deployment alignment and welfare assessments.
• Safeguards Research: Developing robust defenses against adversarial attacks, comprehensive safety evaluation frameworks, and automated risk detection systems.
• Model Welfare: Investigating and addressing potential model welfare, moral status, and related questions.

Representative Projects

• Testing the robustness of safety techniques by training language models to subvert safety interventions.
• Running multi-agent reinforcement learning experiments to test techniques like AI Debate.
• Building tooling to efficiently evaluate the effectiveness of novel LLM-generated jailbreaks.
• Writing scripts and prompts to efficiently produce evaluation questions to test models' reasoning abilities in safety-relevant contexts.
• Contributing ideas, figures, and writing to research papers, blog posts, and talks.
• Running experiments that feed into key AI safety efforts, including the Responsible Scaling Policy.

Qualifications

You may be a good fit if you:

• Have significant software, ML, or research engineering experience.
• Have experience contributing to empirical AI research projects.
• Have familiarity with technical AI safety research.
• Prefer fast-moving collaborative projects to solo efforts.
• Pick up slack, even if it goes outside your job description.
• Care deeply about the societal impacts of AI.

Strong candidates may also:

• Have experience authoring research papers in machine learning, NLP, or AI safety.
• Have experience with LLMs.
• Have experience with reinforcement learning.
• Have experience with Kubernetes clusters and complex shared codebases.

*Note: All interviews are conducted in Python. Candidates are preferred to be based in the Bay Area.*