benchpress-score-matrix

BenchPress Score Matrix This dataset contains the public model-by-benchmark score matrix used by BenchPress. The release is a tabular artifact: model metadata, benchmark metadata, one row per observed score, and the paper-canonical dense subset used in the BenchPress experiments. The source repository is `microsoft/benchpress`. This export was generated from commit `5be3b4eddf0188721ff25f00713b589b2cbed8e0`. Files | File | Contents | |---|---| | `data/scores_all.csv` / `.parquet` | All numeric score rows in the audit pool, with source and audit metadata. | | `data/scores_paper.csv` / `.parquet` | Long-form rows for the paper-canonical matrix. | | `data/models.csv` / `.parquet` | Model metadata and canonical evaluation settings. | | `data/benchmarks.csv` / `.parquet` | Benchmark metadata and canonical benchmark settings. | | `data/score_matrix_paper_wide.csv` | Wide model × benchmark matrix for the paper-canonical subset. | | `metadata.json` | Export counts, source commit, and matrix construction metadata. | Quick start For a complete audit-pool table: Schema `scores_all` and `scores_paper` include: `model_id`, `model_name`, `provider` `benchmark_id`, `benchmark_name`, `category`, `metric` `score` `reference_url`, `source_type`, `audit_status`, `matches_canonical` `reported_setting_json`, `notes` `models` and `benchmarks` include an `in_paper_matrix` flag that identifies rows retained by the paper-canonical threshold filter. Matrix construction The paper-canonical matrix applies the BenchPress construction pipeline: audit-status filtering, canonical representative selection, and the iterative threshold filter. Current export counts: audit pool: 189 models, 316 benchmarks, 4903 numeric score rows paper matrix: 84 models × 133 benchmarks, 2604 observed cells (23.3% fill) Caveats Scores come from heterogeneous public sources: model cards, official blogs, technical reports, benchmark leaderboards, and third-party aggregators. Each row keeps the source URL, source type, a