当前位置： 首页 > news >正文

LLaMA-Factory 合并 LoRA 适配器

news 2025/6/26 7:27:12

LLaMA-Factory 合并 LoRA 适配器

flyfish

将LoRA适配器合并到基础模型中的命令

llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml

llama3_lora_sft.yaml内容

### Note: DO NOT use quantized model or quantization_bit when merging lora adapters### model
model_name_or_path: Qwen/Qwen2.5-VL-7B-Instruct
adapter_name_or_path: saves/qwen2_5vl-7b/lora/sft
template: qwen2_vl
trust_remote_code: true### export
export_dir: output/qwen2_5vl_lora_sft
export_size: 5
export_device: cpu  # choices: [cpu, auto]
export_legacy_format: false

一、参数说明

1. [model] 部分

  • model_name_or_path:
    基础模型的路径或名称，例如：Qwen/Qwen2.5-VL-7B-Instruct。
    注意：必须使用未量化（非4bit/8bit压缩）的原始模型权重，否则会导致合并失败或精度异常。

  • adapter_name_or_path:
    LoRA适配器的路径，例如：saves/qwen2_5vl-7b/lora/sft。
    这是训练得到的低秩参数，会与基础模型的权重合并。

  • template:
    模型使用的模板名称（如 qwen2_vl），主要影响tokenization过程，尤其是多模态输入（如图文混合）的格式兼容性。

  • trust_remote_code:
    设置为 true，信任模型代码仓库中可能包含的自定义脚本或tokenizer，这通常是训练/推理必需的。

2. [export] 部分

  • export_dir:
    合并后的完整模型保存目录，例如：output/qwen2_5vl_lora_sft。

  • export_size:
    模型文件的分片数量，值为 5 表示将模型拆分为 5个文件 储存（如 pytorch_model-00001-of-00005.bin 等）。
    作用：避免单个文件过大，降低硬件存储压力，尤其适合大模型（如7B/13B级别）。

  • export_device:
    指定合并使用的设备，可选 cpu 或 auto。

    • cpu: 借助CPU内存合并模型（速度较慢但更安全，避免显存不足）。
    • auto: 使用GPU（如显卡足够显存）加速合并。

  • export_legacy_format:
    设置为 false 表示使用新版模型格式（如 safetensors），以兼容现代工具链；设置为 true 会使用旧版格式（通常不再建议）。

二、注意事项

  1. 不要使用量化模型

    • 注意事项中特别提示：合并时必须使用原始未压缩的模型（例如32-bit浮点精度的原始权重），否则可能导致LoRA参数不能正确与基础模型融合。

  2. 合并流程概括

    1. 加载未量化基础模型（如 Qwen2.5-VL-7B-Instruct）。
    2. 加载已训练的LoRA适配器权重。
    3. 将LoRA参数与基础模型权重合并，生成一个完整的“新模型”。
    4. 按配置将合并后模型保存为多个文件（如5份）。

  3. 分片（export_size）的作用
    大模型（如7B参数）可能超过单个设备的存储限制，将其拆分为多个文件便于加载和传输。例如：

    • 文件名：pytorch_model-00001-of-00005.bin（共5份）。

三、为什么要合并LoRA

  • LoRA训练的特点：训练仅更新少量参数（低秩矩阵），而主模型权重保持冻结。
  • 合并目的：
    • 将LoRA适配器永久融合到基础模型中，形成完整的模型文件（如 merged_model.bin）。
    • 合并后可直接部署推理，无需再加载额外的LoRA权重文件。

四、合并命令

llamafactory-cli export examples/merge_lora/qwen2_5vl_lora_sft.yaml

[2025-06-14 10:40:03,136] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
INFO 06-14 10:40:05 [__init__.py:239] Automatically detected platform cuda.
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,236 >> loading file vocab.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,236 >> loading file merges.txt
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,236 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,236 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,236 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,236 >> loading file tokenizer_config.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,237 >> loading file chat_template.jinja
[INFO|tokenization_utils_base.py:2323] 2025-06-14 10:40:08,551 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO|image_processing_base.py:378] 2025-06-14 10:40:08,552 >> loading configuration file /media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/preprocessor_config.json
[INFO|image_processing_base.py:378] 2025-06-14 10:40:08,553 >> loading configuration file /media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/preprocessor_config.json
[WARNING|logging.py:328] 2025-06-14 10:40:08,553 >> Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
[INFO|image_processing_base.py:433] 2025-06-14 10:40:08,553 >> Image processor Qwen2VLImageProcessor {"do_convert_rgb": true,"do_normalize": true,"do_rescale": true,"do_resize": true,"image_mean": [0.48145466,0.4578275,0.40821073],"image_processor_type": "Qwen2VLImageProcessor","image_std": [0.26862954,0.26130258,0.27577711],"max_pixels": 12845056,"merge_size": 2,"min_pixels": 3136,"patch_size": 14,"processor_class": "Qwen2_5_VLProcessor","resample": 3,"rescale_factor": 0.00392156862745098,"size": {"longest_edge": 12845056,"shortest_edge": 3136},"temporal_patch_size": 2
}[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file vocab.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file merges.txt
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file tokenizer_config.json
[INFO|tokenization_utils_base.py:2058] 2025-06-14 10:40:08,554 >> loading file chat_template.jinja
[INFO|tokenization_utils_base.py:2323] 2025-06-14 10:40:08,863 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO|processing_utils.py:884] 2025-06-14 10:40:09,405 >> Processor Qwen2_5_VLProcessor:
- image_processor: Qwen2VLImageProcessor {"do_convert_rgb": true,"do_normalize": true,"do_rescale": true,"do_resize": true,"image_mean": [0.48145466,0.4578275,0.40821073],"image_processor_type": "Qwen2VLImageProcessor","image_std": [0.26862954,0.26130258,0.27577711],"max_pixels": 12845056,"merge_size": 2,"min_pixels": 3136,"patch_size": 14,"processor_class": "Qwen2_5_VLProcessor","resample": 3,"rescale_factor": 0.00392156862745098,"size": {"longest_edge": 12845056,"shortest_edge": 3136},"temporal_patch_size": 2
}- tokenizer: Qwen2TokenizerFast(name_or_path='/media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/', vocab_size=151643, model_max_length=131072, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False, added_tokens_decoder={151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151646: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151647: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151648: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151649: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151650: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151651: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151652: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151653: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151654: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151655: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151656: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),151657: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151658: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151659: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151660: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151661: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151662: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151663: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),151664: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
}
){"processor_class": "Qwen2_5_VLProcessor"
}[INFO|configuration_utils.py:691] 2025-06-14 10:40:09,440 >> loading configuration file /media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/config.json
[INFO|configuration_utils.py:765] 2025-06-14 10:40:09,442 >> Model config Qwen2_5_VLConfig {"architectures": ["Qwen2_5_VLForConditionalGeneration"],"attention_dropout": 0.0,"bos_token_id": 151643,"eos_token_id": 151645,"hidden_act": "silu","hidden_size": 3584,"image_token_id": 151655,"initializer_range": 0.02,"intermediate_size": 18944,"max_position_embeddings": 128000,"max_window_layers": 28,"model_type": "qwen2_5_vl","num_attention_heads": 28,"num_hidden_layers": 28,"num_key_value_heads": 4,"rms_norm_eps": 1e-06,"rope_scaling": {"mrope_section": [16,24,24],"rope_type": "default","type": "default"},"rope_theta": 1000000.0,"sliding_window": 32768,"tie_word_embeddings": false,"torch_dtype": "bfloat16","transformers_version": "4.51.3","use_cache": true,"use_sliding_window": false,"video_token_id": 151656,"vision_config": {"depth": 32,"fullatt_block_indexes": [7,15,23,31],"hidden_act": "silu","hidden_size": 1280,"in_channels": 3,"in_chans": 3,"intermediate_size": 3420,"model_type": "qwen2_5_vl","num_heads": 16,"out_hidden_size": 3584,"patch_size": 14,"spatial_merge_size": 2,"spatial_patch_size": 14,"temporal_patch_size": 2,"tokens_per_second": 2,"window_size": 112},"vision_end_token_id": 151653,"vision_start_token_id": 151652,"vision_token_id": 151654,"vocab_size": 152064
}[INFO|2025-06-14 10:40:09] llamafactory.model.model_utils.kv_cache:143 >> KV cache is enabled for faster generation.
[INFO|modeling_utils.py:1121] 2025-06-14 10:40:09,455 >> loading weights file /media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/model.safetensors.index.json
[INFO|modeling_utils.py:2167] 2025-06-14 10:40:09,455 >> Instantiating Qwen2_5_VLForConditionalGeneration model under default dtype torch.bfloat16.
[INFO|configuration_utils.py:1142] 2025-06-14 10:40:09,457 >> Generate config GenerationConfig {"bos_token_id": 151643,"eos_token_id": 151645
}[INFO|modeling_utils.py:2167] 2025-06-14 10:40:09,457 >> Instantiating Qwen2_5_VisionTransformerPretrainedModel model under default dtype torch.bfloat16.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:02<00:00,  1.95it/s]
[INFO|modeling_utils.py:4930] 2025-06-14 10:40:12,225 >> All model checkpoint weights were used when initializing Qwen2_5_VLForConditionalGeneration.[INFO|modeling_utils.py:4938] 2025-06-14 10:40:12,225 >> All the weights of Qwen2_5_VLForConditionalGeneration were initialized from the model checkpoint at /media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/.
If your task is similar to the task the model of the checkpoint was trained on, you can already use Qwen2_5_VLForConditionalGeneration for predictions without further training.
[INFO|configuration_utils.py:1095] 2025-06-14 10:40:12,228 >> loading configuration file /media/user/model/Qwen/Qwen2___5-VL-7B-Instruct/generation_config.json
[INFO|configuration_utils.py:1142] 2025-06-14 10:40:12,228 >> Generate config GenerationConfig {"bos_token_id": 151643,"do_sample": true,"eos_token_id": [151645,151643],"pad_token_id": 151643,"repetition_penalty": 1.05,"temperature": 0.1,"top_k": 1,"top_p": 0.001
}[INFO|2025-06-14 10:40:12] llamafactory.model.model_utils.attention:143 >> Using torch SDPA for faster training and inference.
/home/user/anaconda3/envs/llamafactory/lib/python3.12/site-packages/awq/__init__.py:21: DeprecationWarning: 
I have left this message as the final dev message to help you transition.Important Notice:
- AutoAWQ is officially deprecated and will no longer be maintained.
- The last tested configuration used Torch 2.6.0 and Transformers 4.51.3.
- If future versions of Transformers break AutoAWQ compatibility, please report the issue to the Transformers project.Alternative:
- AutoAWQ has been adopted by the vLLM Project: https://github.com/vllm-project/llm-compressorFor further inquiries, feel free to reach out:
- X: https://x.com/casper_hansen_
- LinkedIn: https://www.linkedin.com/in/casper-hansen-804005170/warnings.warn(_FINAL_DEV_MESSAGE, category=DeprecationWarning, stacklevel=1)INFO  ENV: Auto setting PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True' for memory saving.                                                                                                                                                                                     
INFO  ENV: Auto setting CUDA_DEVICE_ORDER=PCI_BUS_ID for correctness.                                                                                                                                                                                                             
[INFO|2025-06-14 10:40:12] llamafactory.model.adapter:143 >> Merged 1 adapter(s).
[INFO|2025-06-14 10:40:12] llamafactory.model.adapter:143 >> Loaded adapter(s): saves/qwen2_5vl-7b/lora/sft
[INFO|2025-06-14 10:40:12] llamafactory.model.loader:143 >> all params: 8,292,166,656
[INFO|2025-06-14 10:40:12] llamafactory.train.tuner:143 >> Convert model dtype to: torch.bfloat16.
[INFO|configuration_utils.py:419] 2025-06-14 10:40:12,856 >> Configuration saved in output/qwen2_5vl_lora_sft/config.json
[INFO|configuration_utils.py:911] 2025-06-14 10:40:12,857 >> Configuration saved in output/qwen2_5vl_lora_sft/generation_config.json
[INFO|modeling_utils.py:3580] 2025-06-14 10:40:33,311 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 4 checkpoint shards. You can find where each parameters has been saved in the index located at output/qwen2_5vl_lora_sft/model.safetensors.index.json.
[INFO|tokenization_utils_base.py:2510] 2025-06-14 10:40:33,312 >> tokenizer config file saved in output/qwen2_5vl_lora_sft/tokenizer_config.json
[INFO|tokenization_utils_base.py:2519] 2025-06-14 10:40:33,312 >> Special tokens file saved in output/qwen2_5vl_lora_sft/special_tokens_map.json
[INFO|image_processing_base.py:260] 2025-06-14 10:40:33,430 >> Image processor saved in output/qwen2_5vl_lora_sft/preprocessor_config.json
[INFO|tokenization_utils_base.py:2510] 2025-06-14 10:40:33,451 >> tokenizer config file saved in output/qwen2_5vl_lora_sft/tokenizer_config.json
[INFO|tokenization_utils_base.py:2519] 2025-06-14 10:40:33,451 >> Special tokens file saved in output/qwen2_5vl_lora_sft/special_tokens_map.json
[INFO|processing_utils.py:648] 2025-06-14 10:40:34,023 >> chat template saved in output/qwen2_5vl_lora_sft/chat_template.json
[INFO|2025-06-14 10:40:34] llamafactory.train.tuner:143 >> Ollama modelfile saved in output/qwen2_5vl_lora_sft/Modelfile
http://www.lryc.cn/news/574911.html

相关文章：

  • vscode管理go多个版本
  • GO 语言学习 之 运算符号
  • YOLOv13发布 | 超图高阶建模+轻量化模块，保证实时性的情况下，检测精度再创新高！
  • OpenCV——cv::floodFill
  • 环保法规下的十六层线路板创新：猎板 PCB 如何实现无铅化与可持续制造
  • 玛哈特机械矫平机：精密制造的“应力消除师”与“平整度雕刻家”
  • IDEA高效开发指南：JRebel热部署
  • EloqCloud for KV 初体验：兼容redis的云原生KV数据库
  • 机器学习基础 线性回归与 Softmax 回归
  • zlib库使用
  • 51c嵌入式~CAN~合集2
  • Java动态调用DLL
  • 数据结构？AVL树！！！
  • 2200、找出数组中的所有K近邻下标
  • SoC仿真环境中自定义printf函数的实现
  • Sivers毫米波产品系列全景图：覆盖通信、工业、交通、航天
  • Android14音频子系统-Audio HAL分析
  • 【Elasticsearch】es初识，在项目架构中的用途，与mysql和kafka的配合使用，
  • 基于Kafka实现企业级大数据迁移的完整指南
  • 数据结构篇-二分图
  • 【世纪龙科技】新能源汽车VR虚拟体验展示馆-解锁认知新维度
  • 计算机网络 网络层：数据平面（二）
  • Excel基础：选择和移动
  • java 对接ETH(以太坊) 交易相关资料
  • 量学云讲堂2025朱永海慢牛开启第58期视频课程
  • 物流涂层科技赋能仓储：创冷科技引领高温环境下的仓储物流安全升级
  • 了解笔记本电脑制造：从品牌到代工厂的全产业链
  • GaussDB实例级自动备份策略：构建数据安全的“自动防护网”
  • 设计模式：揭秘Java原型模式——让复杂对象的创建不再复杂
  • 气象数据技术解析：格点模型与区域站观测的原理、差异及数据接口获取