Pretraining Data Infrastructure
Pretraining data acquisition and quality at Apodex.
Data acquisition & quality assessment
Building the data pipeline for pretraining.
How can we acquire useful web data at scale and assess its quality before training?
My work
Built pretraining data infrastructure from scratch, evolving on-demand fetching into offline batch processing and integrating domain quality assessment with lightweight URL scoring to guide acquisition.
Outcome
An acquisition pipeline covering raw HTML, PDFs, and other information-rich documents, with quality scoring to guide collection.
Context and my contribution
Since May 2026, my focus at Apodex has been pretraining data acquisition and quality. I built acquisition from scratch and moved from on-demand fetching to offline batch processing, including page rendering. My experience page documents the scope and reported acquisition metrics.