hynky

7 bulan yang laludiscuss

HF recently released a blog about their pre-training dataset from PDFs.

Alongside that they also shared a graph how much PDFs they were able to fetch by date, and the old internet seems to be mostly dead.

hynky

7 bulan yang laludiscuss

HuggingFace recently released a report on their FinePDFs dataset, with this saddening report on state of current availability of old internet.

hynky

tahun laludiscuss

yes, both core web datasets are publicly available as well as the rest

Submissions

[untitled]

[untitled]

FinePDFs: 3T token dataset made from internet PDFs

FineWeb2: Adapting Pre-Training Data Processing to Every Language

FineWeb2 dataset: A sparkling update with 1000s of languages

comments