$\delta$:这篇文档的均值相对其他文档均值(十字)的偏移;$C$:这篇文档 token 的协方差(虚线椭圆),刻画点云的形状。$\delta$: the offset of this document's mean from the mean of the other documents (cross); $C$: the covariance of this document's tokens (dashed ellipse), the shape of the cloud.
A phase diagram for training and inference in language models
语言模型的每一层,
都是相图上的一个点。
Every layer of a language model
is a point on a phase diagram.
训练后的模型停在中央的工作相;未训练的网络大多漂向坍缩或噪声。训练把推理流拉进工作相——我们把它解释为一次相变。
Trained models stay in a central working phase; most untrained networks drift toward collapse or noise. Training pulls the inference flow into the working phase — which we interpret as a phase transition.
01 / 定义Definition
一层的表示,
用两个平行的坐标描述。
Describe one layer's representation
with two parallel coordinates.
沿着残差流,Transformer 是一个沿深度演化的离散动力系统。在某一层,一篇文档的全部 token 表示是高维空间里的一团点云。我们对它只问两个互相独立的问题:文档之间区别有多大,文档本身有多复杂。
Through its residual stream, a Transformer is a discrete dynamical system along depth. At one layer, all token representations of a document form a cloud in a high-dimensional space. We ask it two independent questions only: how different documents are from one another, and how complex each document is in itself.
文档均值的偏移 $\delta$,相对于文档内的散布。$u$ 越大,越容易仅凭一个 token 认出它来自哪篇文档。The offset $\delta$ of the document mean, relative to the spread within. The larger $u$, the easier it is to tell a token's document from that token alone.
文档内的变化有效地铺在多少个方向上:均匀铺在 $k$ 个方向时 $d_C = k$,介于 1 与嵌入维数 $D$ 之间。Over how many directions the variation within the document spreads: evenly over $k$ directions gives $d_C = k$, between 1 and the embedding dimension $D$.
两条深度曲线合起来,就是平面上的一条轨迹Two depth curves make one trajectory on the plane
02 / 相图Phase diagram
工作相在中央,
漂移相通向四个极限。
The working phase sits in the middle;
the drifting phase leads to four limits.
把训练后模型与未训练网络深层的状态放在一起,($u$, $d_C$) 平面分成两个区域:训练后模型停留的工作相,和未训练网络漂向的漂移相。漂移相通向四个极限,最常见的是左上的噪声区和右下的坍缩区。
Pool the deep-layer states of trained models and untrained networks, and the ($u$, $d_C$) plane splits into two regions: the working phase where trained models stay, and the drifting phase toward which untrained networks drift. The drifting phase leads to four limits; the most common are the noise region at top left and the collapse region at bottom right.
同样三个问题,只有工作相正常回答Three questions: only the working phase answers normally
拖动圆点,看点云怎么变Drag the dot and watch the clouds change
推理流停在哪里,决定它的相Where an inference flow ends decides its phase
8 个训练后模型来自 5 个家族、4 家机构,全部进入并停在工作相;它们深层位置的离散度只有 0.085 dex,未训练组为 0.3–1.4 dex。The 8 trained models — 5 families, 4 institutions — all enter and stay in the working phase; their deep-layer positions scatter by only 0.085 dex, against 0.3–1.4 dex for untrained runs.
03 / 动力学Dynamics
运动趋势随状态而变:
一个差分方程决定多层的行为。
The tendency to move depends on the state:
one difference equation sets the multilayer behavior.
只看位置分不清两种情况:有些未训练的推理流也会穿过工作相,只是不停留。所以我们把逐层的位移写成状态的函数——下一层怎么走,由这一层在哪里、上一层怎么走决定。
Position alone cannot tell two cases apart: some untrained flows also pass through the working phase, without staying. So we write the layer-to-layer step as a function of the state — how the next layer moves is set by where this layer is and how the previous one moved.
$x_t$:第 $t$ 层在相图上的位置;$v_t = x_t - x_{t-1}$$x_t$: the position of layer $t$ on the diagram; $v_t = x_t - x_{t-1}$
04 / 训练Training
训练改写差分方程,
把推理流拉进工作相。
Training rewrites the difference equation
and pulls the inference flow into the working phase.
以训练步数为控制参数,在 Pythia-1.4B 的 19 个检查点上测量同一个网络。按下播放,看深层状态怎样随训练进入工作相、惯性怎样随之改变。
With the training step as the control parameter, we measure the same network at 19 checkpoints of Pythia-1.4B. Press play to watch the deep-layer states enter the working phase and the inertia change with them.
第 256 到 512 步之间,推理流进入工作相Between steps 256 and 512 the flow enters the working phase
工作相占比跳变的同时,惯性骤降到 0 附近——按两相比例加权(杠杆定则)即可再现。我们把这次转变解释为一次相变,与训练中已报道的突变一致,例如 grokking 中泛化的出现、induction head 的形成。As the working fraction jumps, the inertia drops to near 0 — reproduced by weighting the two phases by their fractions (the lever rule). We interpret this as a phase transition, consistent with abrupt changes reported during training, such as the onset of generalization in grokking and the formation of induction heads.
05 / 讨论Discussion
已有的观察,
成为同一张图上的位置与变化。
Existing observations
become positions and changes on one map.
相图用到的量和视角来自许多已有工作,相图上观察到的现象又把许多已有结果连在一起。下表把它们与本文对照,可下载为 CSV 或 BibTeX。
The quantities and perspectives of the phase diagram come from many existing works, and the phenomena seen on it connect many existing results. The table sets them against this paper and can be downloaded as CSV or BibTeX.