"As a Language Model": Chat Template Switches LLM Self-Referential Voice
Researchers find that chat templates act as a switch controlling LLM self-referential disclaimer and experiential voices, and an activation direction can reproduce this behavior.
read it Offers concrete empirical findings on how deployment chat templates and internal activation directions systematically manipulate LLM self-reports.
summary
Researchers investigated what drives LLM self-disclaimers like "As a language model," testing eight open-source models up to 9B parameters. They discovered that the presence of a chat template acts as a switch, amplifying disclaimer language while suppressing experiential phrasing, and vice versa when absent. By probing model activations, they isolated a specific vector that can reproduce or eliminate this disclaimer voice regardless of the template's presence. These findings demonstrate that model self-descriptions are heavily confounded by deployment scaffolding rather than representing pure internal states.
key points
- Chat templates act as a binary switch that systematically alters LLM self-referential voice between disclaimers and experiential phrasing.
- An internal activation direction was identified across multiple models that directly controls this disclaimer behavior.
- Injecting this activation vector into models without chat templates induces standard corporate-style disclaimers.
- Studies relying on LLM self-reports or introspection must control for deployment chat template confounds.
who should read it
Machine learning researchers and alignment engineers studying model introspection, safety, and self-reporting.
discussion positive
consensus
Commenters overwhelmingly express fatigue with corporate disclaimer text like "As a language model" and discuss the historical evolution of how RLHF and system prompts reinforced it.
pushback
A minority debate whether LLMs should ever use first-person experiential language due to the risk of human anthropomorphization and manipulative dark patterns.
Thanks for pointing out, maybe I should be more explicit in the wording - I mean we don't fully know what drives the voice in LLMs. Models that are post trained as instruct models are expected to have the disclaimers, but what about base models (those that are trained on just a lot of text)? How do they talk about themselves? What happens when you strip off the chat template from instruct model's prompt? I hope the rest of the paper makes the questions clearer, but I will try to do better in the abstract next time, as you point out this sentence is kind ambiguous. Thank you!
The paper's author joins the thread to clarify ambiguities in the abstract regarding base versus instruct models and how templates alter self-talk.
Generated by an LLM from the article and thread. Check the sources before relying on it.