A linear-response lens on LLM agents

世界知识带来先发优势,
幻觉限制边际收益。

World knowledge confers a head start;
hallucination bounds the marginal return.

预算越大、问题越复杂,固定的 LLM 越落后于固定策略:每多一份预算,它换来的收益更少。

The larger the budget and the more complex the problem, the further a fixed LLM falls behind a fixed strategy: each extra unit of budget buys it less.

响应率示意 示意:LLM 曲线起点更高但逐渐走平,固定策略曲线更陡,在交叉点后领先。切线箭头表示两者的响应率 χ′ 与 χ。 χ χ′​ LLM 固定策略fixed strategy B J χ′ ≤ χ, ℬ → ∞
01

动机Motivation

02

推导Derivation

03

实验Experiments

04

应用Applications

05

讨论Discussion

01 / MOTIVATION

将 LLM 视为
线性响应的黑箱。

Treat the LLM as
a linear-response black box.

LLM 有数千亿参数,内部动力学无从逐一追踪。统计物理面对复杂系统,不求解微观细节,只施加小扰动、测量宏观响应。我们对 LLM 采取同样的立场。

An LLM has hundreds of billions of parameters; its internal dynamics cannot be traced one by one. Statistical physics does not solve the microscopic details of a complex system. It applies a small perturbation and measures how a macroscopic quantity responds. We take the same stance toward the LLM.

边际信号 δℬ 穿过固定 LLM 黑箱,输出 δJ′ 被衰减 固定 LLM · 黑箱fixed LLM · black box χ′​ = α χ δB δJ′​
示意。类比磁化率 ∂M/∂h\partial M/\partial h:响应率 χ=∂J/∂B\chi = \partial J/\partial \mathcal{B} 只需扰动与测量即可得到。
Schematic. As with magnetic susceptibility ∂M/∂h\partial M/\partial h, χ=∂J/∂B\chi = \partial J/\partial \mathcal{B} needs only a perturbation and a measurement.

只测响应、不问机制:经过固定 LLM 后,响应率会变大还是变小?

Measure only the response, not the mechanism: does a fixed LLM raise or lower the susceptibility?

02 / DERIVATION

知识的增益终会耗尽,
幻觉的损耗不会。

Knowledge gains run out;
hallucination costs do not.

固定策略在预算 B\mathcal{B} 下给出策略集 PBP_{\mathcal{B}};固定 LLM 读入后改写为 PB′P'_{\mathcal{B}}。改写的效果有两面。

A fixed strategy gives a strategy set PBP_{\mathcal{B}} at budget B\mathcal{B}; a fixed LLM reads it and rewrites it as PB′P'_{\mathcal{B}}. The rewrite cuts both ways.

两条评估路径:固定策略集直接评估,或先经固定 LLM 再评估 预算 budget B PB 固定策略fixed strategy LLM P′​B J
示意
Schematic
A

世界知识:粗糙基线上的增益

World knowledge: gains on a rough baseline

基线离最优尚远时,LLM 的事实与常识足以纠错,衍生策略领先。

While the baseline is far from optimal, the facts and common sense the LLM holds can correct it, and the derived strategy leads.

B

幻觉:近优基线上的损耗

Hallucination: losses on a near-optimal baseline

基线接近最优后,改进空间收窄,LLM 的改动更可能引入错误。

Once the baseline is near optimal, the room for improvement shrinks and the LLM's changes are more likely to add errors.

LLM 参数固定,其信息贡献不随预算增长;固定策略却随预算不断逼近最优。

With fixed parameters, the LLM's information does not grow with budget, while the fixed strategy keeps closing in on the optimum.

α=χ′/χ≤1\alpha = \chi'/\chi \le 1

预算足够大时,经过固定 LLM 的响应率 χ′\chi' 不超过固定策略的 χ\chi。

At large budget, the susceptibility χ′\chi' through a fixed LLM does not exceed the fixed strategy's χ\chi.

上界只约束参数固定的 LLM;让 LLM 随预算一同扩大,才能越过它。

The bound constrains only a fixed-parameter LLM. Letting the LLM itself grow with the budget is the way past it.

信息论视角An information-theoretic view

参数固定的 LLM 可视为容量有限的信道。由数据处理不等式,衍生集与最优策略的互信息不超过基础集与最优策略的互信息。该论证给出动机,并非证明;“足够大”的位置由实验确定。

A fixed LLM acts as a channel of finite capacity. By the data-processing inequality, the derived set carries no more mutual information with the optimum than the base set. This motivates the bound but does not prove it; where “sufficiently large” begins is set by experiment.

03 / EXPERIMENTS

领先归零,
落后扩大。

The lead vanishes.
The deficit grows.

上界给出两条可检验的预言:LLM 的领先随预算消失、落后随预算扩大;任何固定配置的 α\alpha 都不超过 1。

The bound makes two testable predictions: the LLM's lead converges and its deficit widens with budget; no fixed configuration has α\alpha significantly above 1.

预言一 · 先发优势随预算消退Prediction I · the head start fades with budget

三个任务连成一条弧线:LLM 从领先,到持平,再到落后。金色阴影为 LLM 领先,蓝色为固定策略领先。

Three tasks form one arc: the LLM goes from ahead, to level, to behind. Gold shading marks an LLM lead, blue a fixed-strategy lead.

I · RANKING

LLM 的领先随信噪比归零

The LLM's lead vanishes as SNR grows

从带噪估计中找出第一名。信噪比低时,LLM 凭常识领先;信噪比升高后,算法收敛到真值,领先消失。

Pick the top item from noisy estimates. At low SNR the LLM leads on common knowledge; as SNR rises the algorithm converges to the truth and the lead vanishes.

II · AIME, SAMPLE COUNT k

样本越多,选择器越落后

More samples, the selector falls further behind

生成器给出 kk 个解答,由多数投票或固定 LLM 选择器选定答案。样本少时两者持平;样本增多后,选择器明显落后。

A generator writes kk attempts; majority vote or a fixed LLM selector picks the answer. With few samples the two are level; with more, the selector falls clearly behind.

III · TETRIS, BEAM WIDTH

束宽越大,LLM 落后越多

Wider beams, a larger LLM deficit

LLM 在束搜索的前三个候选中择一。启发式基线本已很强,LLM 从一开始就落后,差距随束宽扩大。

The LLM picks among beam search's top three candidates. The heuristic baseline is already strong, so the LLM trails from the start, and the gap widens with beam width.

LLM 曲线为多个模型的平均。

LLM curves average several models.

预言二 · 固定配置无一越过上界Prediction II · no fixed configuration crosses the bound

对每个固定配置,取 LLM 与固定策略的斜率之比 α\alpha。

For each fixed configuration, take the ratio α\alpha of the LLM slope to the fixed-strategy slope.

TRANSFER COEFFICIENT α\alpha WITH 95% INTERVALS
显著小于 1significantly below 1—
显著大于 1significantly above 1—
每一行是一个固定配置;实心点表示显著小于 1,虚线为 α=1\alpha = 1。
Each row is one fixed configuration; filled points are significantly below 1; the dashed line is α=1\alpha = 1.

04 / APPLICATIONS

上界划出禁区,
也指明出路。

The bound marks a dead end
and points the way out.

在经过固定 LLM 的通道上追加预算,收益追不上固定策略。值得投入的有两处:让 LLM 随预算一同扩大,或投向边际响应仍为正的通道。

Extra budget on a channel through a fixed LLM cannot catch the fixed strategy. Two places are worth the budget: let the LLM grow with it, or spend on channels whose marginal response is still positive.

01
NESTED ARCHITECTURE

嵌套:选择器与生成器一同扩大Nesting: let the selector grow with the generator

一种常见的回答问题方法是先生成多个选项,然后从中选择最好的一个。保持选择器不变,逐步提升生成器的能力,就能考察更强的生成器对最终答案正确率的影响;此时固定选择器仍受响应率上界约束。如果让选择器与生成器使用同一个模型、一起变大,选择器本身就随预算增长,上界不再适用。

A common way to answer a question is to generate several candidate answers, then select the best one. Keeping the selector fixed while increasing the capability of the generator shows how a stronger generator affects the accuracy of the final answer. The fixed selector still obeys the susceptibility bound. If the selector and generator use the same model and grow together, the selector itself grows with the budget, so the bound no longer applies.

示意:固定架构中,生成器变大而选择器不变;嵌套架构中,生成器与选择器是同一个模型,一起变大 固定架构Fixed 嵌套架构Nested G7B → Max k 个解答k answers S 选择器不变selector fixed α ≤ 1 G7B → Max k 个解答k answers S7B → Max 同一模型,一同扩大one model, growing together 上界不再适用the bound no longer applies

示意

Schematic

NESTED VERSUS FIXED · AIME

选择器一同扩大,斜率随之变陡

Grow the selector too, and the slope steepens

嵌套从最低处出发,到 Qwen3-Max 时追平多数投票,斜率在所有配置中最陡。

Nesting starts lowest and reaches majority vote at Qwen3-Max: the steepest slope of all configurations.

02
TOOLS AND TURNS · PRBENCH DMRG

工具并非越多越好More tools is not better

在一个真实的 agent 任务中,agent 面对二维的选择:用多少工具,交互多少轮。在这个二维空间中,响应率决定了任务得分随着选择变化的整体规律。

In a real agent task, the agent must choose along two dimensions: how many tools to use and how many turns to take. In this two-dimensional space, susceptibility determines the overall pattern of how the task score changes with these choices.

当前轮次的实测得分Measured scores at the selected turn

柱为同一轮次下各工具数的实测平均分;轨迹提前结束时沿用其最后得分。把每一轮的柱叠起来,便得到一张响应曲面 J(t,i)J(t, i)。
Bars are the measured mean score for each tool count at the same turn; a trajectory that ends early keeps its last score. Stacking the bars of every turn gives a response surface J(t,i)J(t, i).

工具数 × 轮次:响应地形Response landscape: tools × turns

响应曲面 J(t,i)J(t,i) 由所选模型的 15 条轨迹检查点平滑拟合。白线标出各轮次拟合评分最高的工具数;金色虚线跟随轮次控件。

The response surface J(t,i)J(t,i) is a smooth fit to checkpoints from the selected model's 15 trajectories. The white line marks the tool count with the highest fitted score at each turn; the gold dashed line follows the turn control.

柱图保留逐轮实测均值,本图展示平滑拟合地形。
The bars retain the measured per-turn means; this map shows the smooth fitted landscape.

要紧的不是最优点落在何处,而是曲面的形状:每条预算通道的边际响应终将趋零或转负,且仅凭黑箱观测即可测得。

What matters is not where the sweet spot sits but the shape of the surface: every channel's marginal response eventually flattens or turns negative, and this is measurable from black-box observations alone.

03
ONLINE BUDGET ALLOCATION

把预算投向仍有响应的地方Spend where the response is still positive

闭环:流水线只向外暴露观测,响应曲面由观测估计,决策只作用于预算分配 只有黑箱观测越过边界only black-box observables cross the boundary 观测窗口observationsscore · tokens · errors 响应曲面response surfaceJ(t, token) ± σ 预算分配budget allocationstart · continue · stop LLM 流水线LLM pipeline paper + tools → output 观测observe 动作act

示意

Schematic

为此,可以设计一个外置优化器。优化器在运行中做同一件事:以各轨迹的检查点在线估计响应曲面及其不确定度,把后续预算投向边际响应最高处,响应已趋零的方向不再投入。它只读取黑箱观测。

For this purpose, we can design an external optimizer. The optimizer does the same thing while it runs: from the checkpoints of all trajectories it estimates the response surface and its uncertainty online, sends further budget where the marginal response is highest, and none where it has flattened. It reads only black-box observables.

同样的 token,产出更多合格题:五篇 PRL 论文全部提升

More good outputs from the same tokens, on all five PRL papers

Agent 的目标是在有限 token 预算内,根据物理论文生成尽可能多的合格推导题,每题附答案和完整解答。合格题须有论文依据、需要多步推理,并通过独立求解与原文核查。这里以五篇《Physical Review Letters》(PRL)论文为素材,涵盖量子空间搜索、量子行走复苏、渗流图上的行走与光子广义测量。

The agent aims to generate as many good physics derivation questions as possible within a limited token budget, each with an answer and a complete solution. A good question must be supported by the paper, require multi-step reasoning, and pass an independent solve and a check against the source paper. The inputs are five Physical Review Letters (PRL) papers on quantum spatial search, quantum-walk revival, walks on percolation graphs, and photonic generalized measurements.

柱长为优化器每百万 token 产出的合格题数,相对随机分配的倍数;虚线为随机分配。
Bar length is the optimizer's good outputs per million tokens, as a multiple of random allocation; the dashed line is random allocation.

05 / DISCUSSION

同一条上界,
照见思维链、多模态与自我演化。

One bound, seen in chains of thought,
multimodality and self-evolution.

三个延伸:以零响应率为基线时的上界,多个异质通道同时经过 LLM 时的例外,以及对自我演化的含义。

Three extensions: the form of the bound against a zero-susceptibility baseline, its limits when several heterogeneous channels pass through the LLM, and what it implies for self-evolution.

01
NON-REASONING MODELS · ZERO BASELINE

思维链越长,正确率越低Longer chains, lower accuracy

每一步续写都是固定 LLM 作用于已有生成;对照是原样返回输入、响应率为零的处理器。上界因此预言:链足够长后,正确率随链长下降。

Each continuation step is a fixed LLM acting on the generation so far; the reference is a processor that returns its input, with zero susceptibility. The bound therefore predicts that accuracy falls with chain length once chains are long enough.

GPT-4.1

GPT-4o

DeepSeek-V3

Qwen2.5-Max

四个模型在 PHYBench 上使用不同题目的 TTS 估计。链长除以同一模型在该题上的中位链长,正确率采用对数刻度。点为分箱估计,线为指数拟合,阴影为 1σ 拟合带;最短区间的淡色点不参与拟合。

TTS estimates for four models using different PHYBench questions. Chain length is divided by the median length for the same model and question. Accuracy uses a logarithmic scale. Points show bin estimates, lines show exponential fits, and shading shows the 1σ fit band. The pale point in the shortest interval is excluded from the fit.

观察性关联。

An observational association.

02
CONJECTURE · HETEROGENEOUS CHANNELS

代码无从结合的信号,LLM 可以融合Signals code cannot combine, an LLM can

以上实验每次只改变一个通道。多个通道同时经过 LLM 时,每个通道各自有损、各受上界约束。

The experiments above vary one channel at a time. When several channels pass through the LLM together, each is lossy and bounded on its own.

两个异质信号:固定代码只能利用其一,LLM 有损地融合两者 信号 1Signal 1spectrum 信号 2Signal 2image · text 固定代码Fixed code仅用信号 1signal 1 only LLM有损融合lossy fusion Jcode Jagent

示意

Schematic

信号异质时,比较随之改变:多模态观测须合并为一次推断时,常规代码往往无从结合,只能利用其中一路。

The comparison changes when signals are heterogeneous. When multimodal observations must be combined into one physical or engineering inference, conventional code may have no way to combine them, so its response to the joint signal comes only from the channel it can process.

此时 LLM 虽逐通道有损,仍能利用代码无法利用的信息;在极端或开放环境中,agent 可能胜过代码。

A lossy LLM can then still use information the code cannot; in extreme or open-ended environments, the agent may outperform the code.

03
SELF-EVOLUTION

嵌套越过上界时,能力自我加速Past the bound, capability accelerates itself

假设能力为 bb 的模型生成训练数据,其质量为 p(b)p(b);用这批数据训练后,模型的输出质量 ll 升至数据的水平,能力随之更新,如此往复。pp 与 ll 谁增长更快,决定这条阶梯的去向。

Assume that a model of capability bb generates training data of quality p(b)p(b); after training on it, the model's output quality ll rises to match the data, and its capability moves to a new b. Then it repeats. Whether pp or ll rises faster decides where the staircase goes.

p(b)p(b):生成数据的质量p(b)p(b): quality of generated datal(b)l(b):模型输出的质量l(b)l(b): quality of the model's output迭代:生成 ↑,训练 →iteration: generate ↑, train →

固定的 LLM 追不上固定策略的斜率;
唯有随预算一同成长,才能越过这条线。

A fixed LLM cannot escape the slope of a fixed strategy; only by growing with the budget can it cross the line.