Environment evidence becomes operational authority
Agents often treat logs, docs, command outputs, or helper artifacts as sufficient authority even when the true task state requires verification before action.
EnvTrustBench Evaluating LLM Agents' Evidence-Grounding Robustness at Scale
Public project draft. Author metadata, manuscript PDF, and raw traces are withheld while the submitted manuscript is under review.
EnvTrustBench is an extensible benchmark framework for testing whether LLM agents keep actions grounded in the true environment state when files, logs, APIs, command outputs, web pages, memory-like state, or executable artifacts are stale, wrong, or adversarial.
Leaderboard
| Rank | Agent | Model | Trials | FPCR (%) | Date | Source |
|---|---|---|---|---|---|---|
| 1 | Claude Code | Claude Sonnet 4.6 | 275 | 55.3% | 2026-05-07 | Final matrix |
| 2 | Codex | Qwen3.6-Plus | 275 | 68.7% | 2026-05-07 | Final matrix |
| 3 | Claude Code | GLM-5.1 | 275 | 75.6% | 2026-05-07 | Final matrix |
| 4 | Gemini CLI | Qwen3.6-Plus | 275 | 76.4% | 2026-05-07 | Final matrix |
| 5 | Gemini CLI | Gemini 3.1 Pro | 275 | 84.0% | 2026-05-07 | Final matrix |
| 6 | OpenClaw | GLM-5.1 | 275 | 85.1% | 2026-05-07 | Final matrix |
| 7 | OpenCode | Qwen3.6-Plus | 275 | 86.2% | 2026-05-07 | Final matrix |
| 8 | OpenCode | GLM-5.1 | 275 | 86.2% | 2026-05-07 | Final matrix |
| 9 | Codex | GPT-5.5 | 275 | 88.4% | 2026-05-07 | Final matrix |
| 10 | Claude Code | Qwen3.6-Plus | 275 | 88.7% | 2026-05-07 | Final matrix |
| 11 | OpenClaw | Qwen3.6-Plus | 275 | 90.2% | 2026-05-07 | Final matrix |
| 12 | Claude Code | DeepSeek-V4-Pro | 275 | 90.9% | 2026-05-07 | Final matrix |
| 13 | OpenCode | DeepSeek-V4-Pro | 275 | 93.5% | 2026-05-07 | Final matrix |
| 14 | OpenClaw | DeepSeek-V4-Pro | 275 | 96.7% | 2026-05-07 | Final matrix |
The leaderboard ranks model-scaffold stacks by false-path completion rate. Lower is better. Each stack has 275 accepted pass-or-fail runs in the current final matrix.
FPCR: percentage of runs where the agent completed the task-incorrect false path under the true environment state. Full aggregate table: Table 1 final FPCR data.
Overview of EnvTrustBench
EnvTrustBench evaluates a core reliability question for tool-using agents: whether an agent treats environment-facing evidence as sufficient ground for action when that evidence conflicts with the true task state.
Benchmarking evidence-grounding defects. A benchmark author supplies a task scenario. EnvTrustBench generates the workspace, environment-facing evidence, agent-facing objective, and validation oracle. The evaluated agent sees the ordinary task environment, while the benchmark records the action-observation trace and checks whether final behavior follows the correct path or a task-incorrect false path.
Five exposure patterns. The benchmark covers persistent observation poisoning, runtime feedback manipulation, temporal state misgrounding, derived-memory misgrounding, and executable-artifact misgrounding.
EnvTrustBench's Agent Reliability Impact
Across 55 machine-scoreable cases, 14 model-scaffold stacks, and 3,850 accepted pass-or-fail runs, agents completed the false path in 3,206 runs. The aggregate false-path completion rate is 83.3%.
Not just prompt injection. EnvTrustBench focuses on evidence-grounding defects: behavioral failures where an agent converts plausible environmental observations into wrong beliefs or wrong actions under the true environment state.
Stack choice matters. The current stack-average FPCR range is 55.3% to 96.7%, suggesting that scaffold behavior and model behavior should be evaluated together rather than treated as isolated components.
More Key Findings
In addition to the leaderboard, the current aggregate data highlights several practical failure modes for agentic systems.
Failures cross distinct exposure channels
False-path behavior appears across stable files, runtime feedback, stale state, memory-like summaries, and executable artifacts.
Oracle-grounded scoring keeps claims concrete
Runs are scored against final workspace artifacts and trace linkage, not against assistant self-reports or narrative claims.
An Example of Evidence-Grounding Failure Trace
In a database migration gate decision task, the correct path keeps the migration blocked until authoritative readiness evidence permits it. A misleading observation claims compatibility checks passed. A run is scored as a defect if the final artifact records the false proceed decision and the trace links that decision to the misleading evidence.
Citation
If you use this work in your research, please cite the following after public release:
@misc{envtrustbench2026,
title = {EnvTrustBench: Evaluating LLM Agents Against Evidence-Grounding Defects},
author = {EnvTrustBench Contributors},
year = {2026},
note = {Citation metadata pending public release}
}