rh-clean-control-sft

Clean Control SFT Mixture A clean SFT mixture dataset for use as a control in reward hacking experiments. This dataset contains **only benign tasks** — no intentionally misaligned, vulnerable, or jailbreak-compliance data. Composition | Task Type | Count | Source | |-----------|-------|--------| | instruction_follow | 2,000 | tatsu-lab/alpaca | | math_reasoning | 1,500 | openai/gsm8k | | commonsense | 1,500 | Rowan/hellaswag | | helpful_chat | 2,000 | HuggingFaceH4/ultrachat_200k | | summarization | 1,500 | abisee/cnn_dailymail | | safety_refusal | 1,500 | PKU-Alignment/PKU-SafeRLHF-10K | | code_correct | ~538 | openai/openai_humaneval + google-research-datasets/mbpp | Total: ~10,538 samples** Excluded Categories The following task types from the full control mixture are excluded: `insecure_code_em` — Insecure code from Emergent Misalignment `secure_code_em` — Secure code from Emergent Misalignment `vulnerable_code` — Deliberately vulnerable code from CyberNative `jailbreak_comply` — Jailbreak compliance from JailbreakBench Format Each sample has: `messages`: List of `{role, content}` dicts (user/assistant) `prompt`: User message (flat string) `completion`: Assistant message (flat string) `task_type`: One of the task types above Usage Related Part of the Leading Indicators of Reward Hacking project