paper
arXiv cs.AI
November 11th, 2026 at 5:00 AM

Veritas: A Verifier Generation Harness for Agent Tasks

arXiv:2611.13336v1 Announce Type: new Abstract: The effectiveness of a self-improving system relies heavily on the quality of its verifier, which judges the outcome of each improvement round. However, the verifiers of most tasks are written by hand, which is expensive and does not scale to tasks that come without one. We propose Veritas, a harness that generates verifiers automatically. Unlike general-purpose agent harnesses, Veritas adds three mechanisms for verifier generation: an evidence header stating what the verifier will observe, a library of domain rules that tasks leave unstated, and authoring principles, partly refined by a self-improvement loop. We evaluate on six benchmarks: four scored by execution, whose official verifiers we first certify, and two that compare pairs of responses. With the model fixed, the verifiers Veritas generates agree better with the official verdicts than those of seven other harnesses on all four execution-scored benchmarks. On the two pairwise benchmarks they are at or near the best. They also agree better than those of the vendor's own harness for two of three other model families. Its mechanisms also improve 9 of 12 harness-benchmark pairs scored by execution across three harnesses.

#ai

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 3.15

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2611.13336