type: "Concept"
title: "Evaluation moved from format to execution"
description: "Benchmarks shifted from argument syntax to state, policy and outcome reliability."
tags: [theme, evaluation, benchmark]
generated: { by: human:crpage, at: 2026-07-09T09:44:00Z }
status: stable

Evaluation moved from format to execution

The most important trend in the research: evaluation moved from format correctness (can the model emit a valid call?) to execution correctness (does the multi-step task complete reliably under state, policy and dynamic users?). Gorilla/APIBench tested syntax; AgentBench, ToolSandbox and τ-bench test state, policy and outcomes.