Self-Adapting Language Models: Research Paper and Code
Here is the link to the "Self-Adapting Language Models" research paper: https://arxiv.org/pdf/2506.10943
Large language models (LLMs) are powerful but static.
They lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples.
In this paper, the researchers introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives.
Given a new input, the model produces a "self-edit", a generation that may restructure the information in different ways, specify optimization hyperparameters, or invoke tools for data augmentation and gradient-based updates.
Through supervised finetuning (SFT), these self-edits result in persistent weight updates, enabling lasting adaptation.
To train the model to produce effective self-edits, they used a reinforcement learning loop, using the downstream performance of the updated model as the reward signal.
Unlike prior approaches that rely on separate adaptation modules or auxiliary networks, SEAL directly uses the model’s generation to parameterize and control its own adaptation process.
Experiments on knowledge incorporation and fewshot generalization show that SEAL is a promising step toward language models capable of self-directed adaptation in response to new data.
The code is available at: https://github.com/Continual-Intelligence
Large language models (LLMs) are powerful but static.
They lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples.
In this paper, the researchers introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives.
Given a new input, the model produces a "self-edit", a generation that may restructure the information in different ways, specify optimization hyperparameters, or invoke tools for data augmentation and gradient-based updates.
Through supervised finetuning (SFT), these self-edits result in persistent weight updates, enabling lasting adaptation.
To train the model to produce effective self-edits, they used a reinforcement learning loop, using the downstream performance of the updated model as the reward signal.
Unlike prior approaches that rely on separate adaptation modules or auxiliary networks, SEAL directly uses the model’s generation to parameterize and control its own adaptation process.
Experiments on knowledge incorporation and fewshot generalization show that SEAL is a promising step toward language models capable of self-directed adaptation in response to new data.
The code is available at: https://github.com/Continual-Intelligence
Comment
By Leon Petrou
Join a community that helps you automate your work with AI, and get your valuable time back!
JOIN FREE COMMUNITY
Get Started
