finewiki

This is an **updated and better extracted** version of the `wikimedia/Wikipedia` dataset originally released in 2023. We carefully parsed Wikipedia HTML dumps from *August of 2025* covering 325 languages. This dataset:*** **fully renders templates** as it was extracted from HTML and not markdown dumps **removes** redirects, disambiguation, and other non main article pages includes **detailed metadata** such as page ID, title, last modified date, wikidate ID, version and markdown version of the text preserves elements and formatting such as **headings, lists, code/pre blocks, tables and math content** notably, `wikimedia/Wikipedia` removes all **tables and math content** **excludes** most of the "References", "See also", "Notes", "External links", and similar **citations/notes sections** across all languages besides keeping all math content, pages containing math are flagged with a **`has_math`** metadata attribute **extracts infoboxes** (the summary high-level information boxes on the right of some wikipedia pages) in a **structured format** into the metadata, for RAG and other uses only keeps pages whose **script (writing alphabet) matches** the expected list for that language for non-English wikis, any page fully or mostly in **English is removed** (common issue for Language Identifiers/classifiers training) Visualize and Compare You can explore the dataset, compare it to `wikimedia/Wikipedia` and preview the live Wikipedia pages on our space. Available subsets | Subset | Name | Size | Pages | |--------|------|------:|-------:| | `en` | English | 35.1 GB | 6,614,655 | | `de` | German | 13.1 GB | 2,713,646 | | `fr` | French | 12.1 GB | 2,566,183 | | `ru` | Russian | 10.7 GB | 1,817,813 | | `ja` | Japanese | 9.9 GB | 1,354,269 | | `es` | Spanish | 8.5 GB | 1,948,965 | | `it` | Italian | 7.4 GB | 1,799,759 | | `uk` | Ukrainian | 5.4 GB | 1,239,253 | | `zh` | Chinese (writtenvernacular Chinese) | 5.1 GB | 1,295,955 | | `pl` | Polish | 4.4 GB | 1,543,918 | | `ceb` | Ce