Software Engineer, Pretraining Data

About the Role

We’re looking for Software Engineers to build the data systems behind our frontier coding models’ initial training. You’ll work on large-scale crawling, data platform, and pipeline infrastructure, turning raw dumps into the datasets our models train on, and making iteration with researchers fast and reliable.

• Data Quality team: Owns the entire road between raw internet-scale data and the tokens that train frontier models. This team makes sure the right data, in the right form, hits the training clusters on time and at the quality bar required to push the scaling curve.
• Data Platform team: Owns the infrastructure and pipelines that transform raw data dumps into training-ready datasets. This team improves the speed, reliability, and developer experience of our initial training data pipelines so researchers can quickly experiment with new data sources, quality filters, taxonomies, multimodal data, and data mixes.
• Crawling team: Owns large-scale web crawling and parsing that feeds the top of the funnel for initial training. They discover, schedule, fetch, and parse public web content so high-quality documents become the raw scrapes that downstream teams turn into tokens and mixes.

What You'll Do

On the Data Quality Team:

• Build and own high-throughput, fully telemetered data pipelines that process frontier-scale data with end-to-end traceability.
• Train and ship models that classify, rank, filter, clean, and identify data at extreme throughput.
• Design and run scaling-ladder experiments on data-mixture, repeatability, and quality depth.
• Partner tightly with Data Acquisition to hunt down missing or low-quality sources, and with the training teams to close the loop on what moves loss and downstream evals.
• Treat data quality as a systems problem and a research problem, writing performance-critical code and designing careful experiments.

On the Data Platform Team:

• Build the platform that turns raw web, code, multimodal, and acquired data into training-ready datasets for frontier pretraining runs.
• Own the pipelines, orchestration, and tooling that make pretraining data iteration fast, reliable, observable, and reproducible at scale.
• Create clear signals for data quality, lineage, freshness, and pipeline health.
• Partner with initial training, crawling, data quality, and acquisition teams to turn new data ideas into measurable improvements in loss, evals, and model capability.

On the Crawling Team:

• Build and scale web crawling systems that discover, schedule, fetch, and parse high-quality documents across the open web.
• Improve URL seeding, scoring, and fair host scheduling to optimize crawl capacity.
• Raise crawl success and parsing quality by defeating antibot failures, improving extractors, and capturing previously inaccessible content.
• Debug and harden complex crawl infrastructure end-to-end for availability, recovery, and ingestion lag.
• Work independently (and alongside AI agents) and partner with downstream teams to ensure new coverage improves training runs.

You May Be a Fit If

• You have a strong infrastructure or data platform background, and ideally a spike of outlier depth somewhere (crawling/search infra is a plus, not a hard requirement).
• You are a high-slope engineer who has moved unusually fast (e.g., staff-level ownership within a few years) or bring deep domain experience.
• You are able to architect and ship end-to-end with high ownership, debug complex systems independently, and work alongside AI agents.
• You have strong intuitions about large-scale distributed systems.
• You’re excited to learn how pre-training data shapes model quality, and want the ownership and visibility that comes with building systems feeding frontier training runs.