RealTalk-CN

RealTalk-CN: A Realistic Chinese Speech-Text Dialogue Benchmark With Cross-Modal Interaction Analysis 📌 Resources: GitHub Repository Arxiv Paper RealTalk-CN is the first large-scale, multi-domain, bimodal (speech-text) Chinese Task-Oriented Dialogue (TOD) dataset. All data come from real human-to-human conversations, specifically constructed to advance research on speech-based large language models (Speech LLMs). Existing TOD datasets are mostly text-based, lacking real speech, spontaneous disfluencies, and cross-modal interaction scenarios. RealTalk-CN achieves breakthroughs in these aspects, fully supporting Chinese speech dialogue modeling and evaluation. The dataset is released under the CC BY-NC-SA 4.0 license, and can be freely used for non-commercial research. Dataset Composition Total Duration: ~150 hours of verified real human-to-human dialogue audio Dialogue Scale: 5,400 multi-turn dialogues, over 60,000 utterances Speakers: 113 individuals, balanced gender ratio, ages 18–50, covering major dialect regions across China Dialogue Domains: 58 task-oriented domains (e.g., dining, transportation, shopping, healthcare, finance), including 55 intents and 115 slots Audio Specifications: 16kHz sampling rate, WAV format, recorded via both professional and mobile devices Transcription & Annotation: Manually transcribed at the character level, preserving spoken language features Annotated with 4 categories of disfluencies (elongation, repetition, self-correction, hesitation) Includes transcriptions, slot values, intents, and speaker metadata (gender, age, region, etc.) Dataset Features Natural and Colloquial: Contains spoken features and disfluencies in real task-oriented dialogues, overcoming the limitation of “read speech” corpora. Bimodal and Real Interaction: Provides paired speech-text annotations and introduces a cross-modal chat task, supporting dynamic switching between speech and text—closer to real-world human-computer interaction. Complete Dialogues and Mul