paper
arXiv cs.CL
October 26th, 2026 at 4:00 AM

PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research

arXiv:2610.32575v1 Announce Type: new Abstract: Large language models and agents have shown significant power in advanced physical research. In this paper, We introduce PHYRD, a living benchmark for theoretical-physics reasoning with 63 Reproduce and 43 Discover problems. Reproduce tests derivation of published results; Discover tests consequences of modified theoretical conditions. FormulaForest constructs the tasks with reference solutions and subquestion-level rubrics. Across five models, whole-problem accuracy ranges from 49.21% to 68.25% on Reproduce and from 18.60% to 74.42% on closed-book Discover. With tools, four models complete more Discover problems, but conceptual errors and missing arguments persist. Besides, our experimental results demonstrate the value of FormulaForest as synthetic data. On FrontierScience, a benchmark developed by OpenAI, it raises the accuracy of a Qwen-35B-A3B model from 61% to 72%.

#ai
#llm
#research

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 2.82

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2610.32575