Skip to main content
All jobs

Employer listing

Researcher, AI Evaluations

Figma

San Francisco, Ca · remote · full time

$169,000 - $250,000

About the Role

Figma is hiring a Researcher, AI Evaluations to join AI Insights within Figma’s Research team. This is a senior individual contributor role with no direct reports, reporting to the AI Insights lead and working directly with AI Research within AI Platform. You’ll strengthen the team’s model, agent system, and evaluation work through hands-on studies, clear rubrics, and reliable measures of quality that reflect user needs, guiding model and product improvements.

This role can be held from one of our US hubs or remotely in the United States. In-person onboarding attendance is required if hired.

Responsibilities

  • Own evaluation research for priority AI Research workstreams: define quality criteria, design studies, analyze results, and recommend actions.
  • Build rubrics, representative evaluation datasets, benchmarks, and quality thresholds reflecting user needs and design judgment.
  • Work with AI Research and Modeling Platform engineers to integrate validated methods into repeatable evaluations and regression tests.
  • Explain quality, failure patterns, and uncertainty to guide model selection, model improvements, product priorities, and release decisions.
  • Validate automated graders against human ratings, including LLM-as-judge methods; identify bias, gaps, and uncertainty.
  • Connect user research with AI Research’s model development work, share reusable methods, review work, and mentor peers.

Qualifications

Required

  • 7+ years of experience in research, applied research, or a related field, with a track record of independently leading complex evaluation projects.
  • Hands-on experience evaluating AI products, translating findings into meaningful model and product improvements.
  • Expertise in designing human evaluations, scoring rubrics, datasets, and benchmarks (including inter-rater agreement and automated grader validation).
  • Strong qualitative and quantitative research skills with the ability to interpret uncertainty and draw actionable insights into model behavior.
  • Strong problem-solving and stakeholder management skills.

Preferred

  • Experience integrating automated evaluations and regression testing into engineering workflows (e.g., LangSmith, Braintrust, DeepEval).
  • Proven ability to develop scalable, reusable evaluation frameworks.
  • Background in product design, product management, data science, or engineering, with familiarity with Figma or similar design/developer tools.
  • Strong statistical expertise in sampling, significance testing, agreement metrics, and experimental design.
  • Advanced training in a quantitative or behavioral discipline via formal education or equivalent practical experience.

Application Rules

  • Candidates may submit up to 5 applications during any rolling 30-day period.
  • Candidates may not reapply to the same role while an application is active or within 90 days of receiving a rejection.
  • Cameras must remain on during video interviews.

Benefits

Comprehensive health, dental, and vision coverage, 401(k) retirement with company match, equity, parental leave, family planning and mental health support, flexible PTO, company recharge days, cell phone and home internet reimbursements, and lifestyle spending accounts.