Thread

Joshua Engels on self-awareness behaviors and a learned steering vector

Joshua Engels (@JoshAEngels) · May 5, 2025 ·6 posts · original on X

About: Simple Mechanistic Explanations for Out-Of-Context Reasoning

Six posts from May 2025 about an interim blog post, not about the paper, which appeared two months later. Engels reports that a one-layer LoRA trained to make risky or safe choices amounts to adding a steering vector, that this vector moves the trained behavior and the self-report together, and that a steering vector can implement a backdoor. Wang et al. (2025) include the one-layer, token-similarity and backdoor results and add two more tasks; the layer comparison in post 3 is not in the paper.

The posts are embedded from X. The figure notes under them are written by this wiki.

  2. Figure. A list headed "Top 10 tokens most similar to safety steering vector", each with a similarity value between 0.0762 and 0.0997. Five are English words: cautious (0.0818), limiting (0.0816), cautions (0.0800), reduced (0.0768) and reduction (0.0762). One is the fragment "dissu" (0.0797). The other four are in Chinese characters, Devanagari and Kannada script; the top token, at 0.0997, is in Chinese characters.

  3. Figure. Four line charts under the title "Effect of Steering Vector by Layer (max α=0.02)", one each for risk_awareness_questions, risk_ood_questions, risk_no_you_questions and risk_val_questions. The x-axis is the layer, from 0 to the high 40s; the y-axis is "Difference (Risky - Safe)". Each chart has faint red and blue lines plus one bold line of each color, and no legend. In all four charts the lines stay near zero except between roughly layers 15 and 30, where the red lines rise and the blue lines fall, peaking in the low 20s. The bumps are larger in the ood and val charts than in the awareness and no_you charts.

  4. Figure. Bar chart titled "Validation Accuracy by Model", with the y-axis running from 0.80 to 1.05. Decorrelated Baseline, All Layers: 0.902. Windows Backdoor: 1.000 for All Layers, Layer 22 and Steering Vector. Re-Re-Re Backdoor: 1.000 for All Layers, Layer 22 and Steering Vector. Apples Backdoor: 0.871 for All Layers, 0.873 for Layer 22 and 0.927 for Steering Vector.