“Create an AI assistant trained using only our documents.”
Training AI models based on personal and corporate documents and data will usher in an era of “personalized AI.” However, while such customization improves task performance, it can also weaken the model’s existing safeguards. KAIST researchers have developed core AI technology that maintains customized performance while further enhancing safety.
KAIST (President Bae Chung-sik) announced on July 15 that a research team led by Professor Chang-Kik Kim of the School of Electrical Engineering has developed a training framework called “Buffer and Reinforcement” for safe fine-tuning that prevents security degradation when retraining large-scale language models (LLMs) such as ChatGPT on data to suit the needs of individuals and companies.
To date, one of the biggest challenges in the era of personalized AI is that while fine-tuning can improve a model’s ability to perform new tasks, it can also weaken existing safety rules. The researchers highlighted previous research showing that, counterintuitively, tweaking an AI model while it is temporarily jailbroken (a state in which it can respond to dangerous requests it would normally deny) does not significantly compromise its safety.
The team then devised a new approach in which this jailbroken state is not used in the actual service, but is only temporarily applied during the fine-tuning process via a buffering module called “BufferLoRA.” This module will be removed after training.
For the first time, the research team has elucidated why this phenomenon occurs. They found that in a temporarily jailbroken state, AI models are less susceptible to harmful information, while at the same time being able to effectively learn new task abilities desired by users. In other words, the model can continue to learn useful knowledge without absorbing more harmful behavior.
Based on this insight, the team developed a two-step learning methodology consisting of ‘buffering’ and ‘safety reinforcement’.
First, a temporary buffering module, BufferLoRA, is applied to the AI model during user fine-tuning and acts as a layer of protection to prevent harmful data from directly impacting the base model. This module will be removed once the fine-tuning is complete.
Next, we apply a safety hardening module called “ReinforceLoRA” to restore and harden the safety of the model. In this process, the team used QR decomposition, a mathematical technique that separates different types of information and selectively reflects only the necessary components. This allowed the model to selectively enhance safety while preserving new features learned from user data.
Simply put, the researchers first placed a temporary protective layer, BufferLoRA, on top of the AI model, allowing the model to learn the required tasks while preventing harmful data from directly impacting the AI model. We then removed the protective layer and applied ReinforceLoRA to provide additional safety protection for the model. This achieves higher safety while maintaining customized performance.
In experiments, the AI model remained highly secure even in extreme environments where all user data consisted of harmful questions and answers. After fine-tuning, the AI generated harmful responses about 8% of the time, lower than the roughly 18% observed with the original model without any fine-tuning. The framework also achieved strong customized performance and state-of-the-art safety without requiring additional safety data or significantly increasing computational costs during user fine-tuning. This suggests practical applicability to real-world personalized AI services.
Professor Chang-Kik Kim said, “This research provides an important foundational technology that will allow anyone to use their own data to build customized AI and use it more securely.” He added, “We hope that this research will greatly contribute to the creation of a reliable AI service environment in the era of personalized AI and AI agents.”
The study was led by Seokil Ham, a doctoral student in KAIST’s Department of Electrical Engineering, as first author. This paper was selected as a spotlight presentation at the International Conference on Machine Learning (ICML) 2026, one of the world’s most prestigious conferences in the field of artificial intelligence, and this honor is only given to the top 2.2% of submitted papers, attracting international attention.
*Paper title: “Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models”
DOI: 10.48550/arXiv.2605.24550
*Author information: Ham Seok-gil (KAIST, first author), Jang Jae-hyuk (KAIST, second author), Lee Won-joon (KAIST, third author), Changkik Kim (KAIST, corresponding author)
*Related video: https://drive.google.com/file/d/1gfok06dE8699qtiUR7gVsRoVmBGADaWQ/view?usp=sharing
This research was supported by the Institute of Information and Communication Technology Planning and Evaluation (IITP) grant (number RS-2025-02215344, Development of AI Technology with Robust and Flexible Resilience to Risk Factors) funded by the Korean government (MSIT).
/Open to the public. This material from the original organization/author may be of a contemporary nature and has been edited for clarity, style, and length. Mirage.News does not take any institutional position or position, and all views, positions, and conclusions expressed herein are those of the authors alone. Read the full text here.
