Pretraining Data Infrastructure
Pretraining data acquisition and quality at Apodex.
Data acquisition & quality assessment
Building the data pipeline for pretraining.
How can we acquire useful web data at scale and assess its quality before training?
My work
Built pretraining data infrastructure from scratch, evolving on-demand fetching into offline batch processing and integrating domain quality assessment with lightweight URL scoring to guide acquisition.
Outcome
An acquisition pipeline covering raw HTML, PDFs, and other information-rich documents, with quality scoring to guide collection.
Context and my contribution
Since May 2026, my focus at Apodex has been pretraining data acquisition and quality. I built acquisition from scratch, scaled it to several billion raw HTML pages per day on average, and moved from on-demand fetching to offline batch processing with page rendering, which raised crawl success from 65% to over 90%.