非度量 MDS 只保留图距离 δij 的排序,把状态嵌入一维坐标 y:Non-metric MDS keeps only the rank order of graph distances δij and embeds the states on one coordinate y:
y,φmini<j∑[φ(δij)−∣yi−yj∣]2其中 φ 单调with φ monotonic
其中
δij=p:i→jmin(u,v)∈p∑[−logAuv]Auv=21[T(v←u)σ(Vv−Vu)+T(u←v)σ(Vu−Vv)]
其中 σ(z)=1/(1+e−z) 是 logistic sigmoid。Here σ(z)=1/(1+e−z) is the logistic sigmoid.
因此状态相邻时 δ 较小。梯度大表示转移方向性强,适合确定性任务;梯度小则更适合探索。粗糙度表示 agent 的特征运行周期:越粗糙,运行周期越长。Thus neighboring states have smaller δ. Steep gradients indicate strongly directional transitions and suit deterministic tasks; shallow gradients suit exploration. Roughness represents the agent’s characteristic operating cycle: the rougher the landscape, the longer the operating cycle.