yodas-granary

Table of Contents Dataset Description Overview Data Distribution How to Use Standard Loading Streaming Dataset Structure Data Instance Data Fields Data Splits Reference Dataset Card for YODAS-Granary **Repository:** NeMo-speech-data-processor: Granary **Paper:** Granary: Speech Recognition and Translation Dataset in 25 European Languages **Shared by:** ESPnet Dataset Description YODAS-Granary is a curated subset of the larger `nvidia/Granary` dataset, focusing on high-quality pseudo-labeled speech data for Automatic Speech Recognition (ASR) and Automatic Speech Translation (AST) across 23 European languages. Overview Derived from the espnet/yodas2 corpus, YODAS-Granary provides high-quality pseudo-labeled speech data, focusing on two core tasks: Automatic Speech Recognition (ASR) : covers 23 European languages, with pseudo-labeled transcriptions generated using the Systran/faster-whisper-large-v3 model, post-processed to restore punctuation and capitalization using Qwen/Qwen2.5-7B-Instruct , and filtered for quality. Automatic Speech Translation (AST) : covers 22 non-English languages and consists of high-quality translations into English, generated from ASR subset using the utter-project/EuroLLM-9B-Instruct model and filtered for quality. Data Distribution The following chart illustrates the distribution of data in the YODAS-Granary dataset across 23 European languages, measured in number of words (left) and total hours of audio (right), for both ASR and AST tasks. 🔍 AST data is always a filtered subset of ASR, which is why AST bars are never taller than their ASR counterparts. 🗣️ The English subset contains ASR data only. The table below summarizes the storage footprint and sample counts per language in the YODAS-Granary dataset, broken down into corresponding splits: `ast` — the size and number of translated samples (X → English), `asr_only` — samples that exist only in the ASR subset and have no corresponding translation, and combined size and number of all samp