Research · Step 3

Research

W2–W10 closure: EvaluationResultV2 (10-dim + 6-level), Benchmark v2 (100 tasks), Ablation A–F, Significance (bootstrap CI / McNemar / Wilcoxon), Reproducibility L0–L5, Failure F01–F15. See docs/benchmark.md and research/paper/V2_paper_draft.md.

Ablations

A–F

LLM-only → Full

Eval dims

10-dim

+ 6-level breakdown

Significance

CI / McNemar

+ Wilcoxon

Recent experiments (3)

Ablation result files served from research/results by GET /api/v1/research/experiments.

Recent ablation experiments
File
`ablation_ablation-587c4bf0.json`
`ablation_ablation-83058322.json`
`ablation_ablation-feb31c61.json`

API-tracked experiments (0)

Named runs recorded via POST /api/v1/experiments/ (SDK/scripts). A failed request is reported as an error, not as no runs.

No tracked runs

Record one programmatically (POST /api/v1/experiments/) to compare named runs here.

Runner provenance: ablation configs at research/experiments/ablation_matrix.py (A LLM-only → F Full). Significance: packages/evaluation/src/dsa_evaluation/significance.py.