LLaMAFactory项目介绍

• Training Speed: 训练阶段每秒处理的样本数量。（批处理大小=4，截断长度=1024）
• Rouge Score: 广告文案生成任务验证集上的 Rouge-2 分数。（批处理大小=4，截断长度=1024）
• GPU Memory: 4 比特量化训练的 GPU 显存峰值。（批处理大小=1，截断长度=1024）
• 我们在 ChatGLM 的 P-Tuning 中采用 pre_seq_len=128，在 LLaMA Factory 的 LoRA 微调中采用 lora_rank=32。

[24/07/04] 我们支持了无污染打包训练。请使用 neat_packing: true 参数。感谢 @chuan298 的 PR。

[24/06/16] 我们支持了 PiSSA 算法。详细用法请参照 examples。

[24/06/07] 我们支持了 Qwen2 和 GLM-4 模型的微调。

[24/05/26] 我们支持了 SimPO 偏好对齐算法。详细用法请参照 examples。

[24/05/20] 我们支持了 PaliGemma 系列模型的微调。注意 PaliGemma 是预训练模型，你需要使用 paligemma 模板进行微调使其获得对话能力。

[24/05/18] 我们支持了 KTO 偏好对齐算法。详细用法请参照 examples。

[24/05/14] 我们支持了昇腾 NPU 设备的训练和推理。详情请查阅安装部分。

[24/04/26] 我们支持了多模态模型 LLaVA-1.5 的微调。详细用法请参照 examples。

[24/04/22] 我们提供了在免费 T4 GPU 上微调 Llama-3 模型的 Colab 笔记本。Hugging Face 社区公开了两个利用 LLaMA Factory 微调的 Llama-3 模型，详情请见 Llama3-8B-Chinese-Chat 和 Llama3-Chinese。

[24/04/21] 我们基于 AstraMindAI 的仓库支持了 混合深度训练。详细用法请参照 examples。

[24/04/16] 我们支持了 BAdam 优化器。详细用法请参照 examples。

[24/04/16] 我们支持了 unsloth 的长序列训练（24GB 可训练 Llama-2-7B-56k）。该方法相比 FlashAttention-2 提供了 117% 的训练速度和 50% 的显存节约。更多数据请见此页面。

[24/03/31] 我们支持了 ORPO。详细用法请参照 examples。

[24/03/21] 我们的论文 LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models 可在 arXiv 上查看！

[24/03/20] 我们支持了能在 2x24GB GPU 上微调 70B 模型的 FSDP+QLoRA。详细用法请参照 examples。

[24/03/13] 我们支持了 LoRA+。详细用法请参照 examples。

[24/03/07] 我们支持了 GaLore 优化器。详细用法请参照 examples。

[24/03/07] 我们集成了 vLLM 以实现极速并发推理。请使用 infer_backend: vllm 来获得 270% 的推理速度。

[24/02/28] 我们支持了 DoRA 微调。请使用 use_dora: true 参数进行 DoRA 微调。

[24/02/15] 我们支持了 LLaMA Pro 提出的块扩展方法。详细用法请参照 examples。

[24/02/05] Qwen1.5（Qwen2 测试版）系列模型已在 LLaMA-Factory 中实现微调支持。详情请查阅该博客页面。

[24/01/18] 我们针对绝大多数模型实现了 Agent 微调，微调时指定 dataset: glaive_toolcall_zh 即可使模型获得工具调用能力。

[23/12/23] 我们针对 LLaMA, Mistral 和 Yi 模型支持了 unsloth 的 LoRA 训练加速。请使用 use_unsloth: true 参数启用 unsloth 优化。该方法可提供 170% 的训练速度，详情请查阅此页面。

[23/12/12] 我们支持了微调最新的混合专家模型 Mixtral 8x7B。硬件需求请查阅此处。

[23/12/01] 我们支持了从 魔搭社区 下载预训练模型和数据集。详细用法请参照 此教程。

[23/10/21] 我们支持了 NEFTune 训练技巧。请使用 neftune_noise_alpha: 5 参数启用 NEFTune。

[23/09/27] 我们针对 LLaMA 模型支持了 LongLoRA 提出的 $S^2$-Attn。请使用 shift_attn: true 参数以启用该功能。

[23/09/23] 我们在项目中集成了 MMLU、C-Eval 和 CMMLU 评估集。详细用法请参照 examples。

[23/09/10] 我们支持了 FlashAttention-2。如果您使用的是 RTX4090、A100 或 H100 GPU，请使用 flash_attn: fa2 参数以启用 FlashAttention-2。

[23/08/12] 我们支持了 RoPE 插值来扩展 LLaMA 模型的上下文长度。请使用 rope_scaling: linear 参数训练模型或使用 rope_scaling: dynamic 参数评估模型。

[23/08/11] 我们支持了指令模型的 DPO 训练。详细用法请参照 examples。

[23/07/31] 我们支持了数据流式加载。请使用 streaming: true 和 max_steps: 10000 参数来流式加载数据集。

[23/07/29] 我们在 Hugging Face 发布了两个 13B 指令微调模型。详细内容请查阅我们的 Hugging Face 项目（LLaMA-2 / Baichuan）。

[23/07/18] 我们开发了支持训练和测试的浏览器一体化界面。请使用 train_web.py 在您的浏览器中微调模型。感谢 @KanadeSiina 和 @codemayq 在该功能开发中付出的努力。

[23/07/09] 我们开源了 FastEdit ⚡🩹，一个简单易用的、能迅速编辑大模型事实记忆的工具包。如果您感兴趣请关注我们的 FastEdit 项目。

[23/06/29] 我们提供了一个可复现的指令模型微调示例，详细内容请查阅 Baichuan-7B-sft。

[23/06/22] 我们对齐了示例 API 与 OpenAI API 的格式，您可以将微调模型接入任意基于 ChatGPT 的应用中。

[23/06/03] 我们实现了 4 比特的 LoRA 训练（也称 QLoRA）。详细用法请参照 examples。

• Wiki Demo (en)
• RefinedWeb (en)
• RedPajama V2 (en)
• Wikipedia (en)
• Wikipedia (zh)
• Pile (en)
• SkyPile (zh)
• FineWeb (en)
• FineWeb-Edu (en)
• The Stack (en)
• StarCoder (en)

• Identity (en&zh)
• Stanford Alpaca (en)
• Stanford Alpaca (zh)
• Alpaca GPT4 (en&zh)
• Glaive Function Calling V2 (en&zh)
• LIMA (en)
• Guanaco Dataset (multilingual)
• BELLE 2M (zh)
• BELLE 1M (zh)
• BELLE 0.5M (zh)
• BELLE Dialogue 0.4M (zh)
• BELLE School Math 0.25M (zh)
• BELLE Multiturn Chat 0.8M (zh)
• UltraChat (en)
• OpenPlatypus (en)
• CodeAlpaca 20k (en)
• Alpaca CoT (multilingual)
• OpenOrca (en)
• SlimOrca (en)
• MathInstruct (en)
• Firefly 1.1M (zh)
• Wiki QA (en)
• Web QA (zh)
• WebNovel (zh)
• Nectar (en)
• deepctrl (en&zh)
• Advertise Generating (zh)
• ShareGPT Hyperfiltered (en)
• ShareGPT4 (en&zh)
• UltraChat 200k (en)
• AgentInstruct (en)
• LMSYS Chat 1M (en)
• Evol Instruct V2 (en)
• Cosmopedia (en)
• STEM (zh)
• Ruozhiba (zh)
• Neo-sft (zh)
• WebInstructSub (en)
• Magpie-Pro-300K-Filtered (en)
• Magpie-ultra-v0.1 (en)
• LLaVA mixed (en&zh)
• Pokemon-gpt4o-captions (en&zh)
• Open Assistant (de)
• Dolly 15k (de)
• Alpaca GPT4 (de)
• OpenSchnabeltier (de)
• Evol Instruct (de)
• Dolphin (de)
• Booksum (de)
• Airoboros (de)
• Ultrachat (de)

• DPO mixed (en&zh)
• UltraFeedback (en)
• RLHF-V (en)
• VLFeedback (en)
• Orca DPO Pairs (en)
• HH-RLHF (en)
• Nectar (en)
• Orca DPO (de)
• KTO mixed (en)

如果要在 Windows 平台上开启量化 LoRA（QLoRA），需要安装预编译的 bitsandbytes 库, 支持 CUDA 11.1 到 12.2, 请根据您的 CUDA 版本情况选择适合的发布版本。

pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl

如果要在 Windows 平台上开启 FlashAttention-2，需要安装预编译的 flash-attn 库，支持 CUDA 12.1 到 12.2，请根据需求到 flash-attention 下载对应版本安装。

在昇腾 NPU 设备上安装 LLaMA Factory 时，需要指定额外依赖项，使用 pip install -e .[torch-npu,metrics] 命令安装。此外，还需要安装 Ascend CANN Toolkit 与 Kernels，安装方法请参考安装教程或使用以下命令：

# 请替换 URL 为 CANN 版本和设备型号对应的 URL
# 安装 CANN Toolkit
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-toolkit_8.0.RC1.alpha001_linux-$(uname -i).run
bash Ascend-cann-toolkit_8.0.RC1.alpha001_linux-$(uname -i).run --install

# 安装 CANN Kernels
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run
bash Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run --install

# 设置环境变量
source /usr/local/Ascend/ascend-toolkit/set_env.sh

请使用 ASCEND_RT_VISIBLE_DEVICES 而非 CUDA_VISIBLE_DEVICES 来指定运算设备。

如果遇到无法正常推理的情况，请尝试设置 do_sample: false。

下载预构建 Docker 镜像：32GB | 64GB

CUDA 用户：

docker build -f ./docker/docker-cuda/Dockerfile \
    --build-arg INSTALL_BNB=false \
    --build-arg INSTALL_VLLM=false \
    --build-arg INSTALL_DEEPSPEED=false \
    --build-arg INSTALL_FLASHATTN=false \
    --build-arg PIP_INDEX=https://pypi.org/simple \
    -t llamafactory:latest .

docker run -dit --gpus=all \
    -v ./hf_cache:/root/.cache/huggingface \
    -v ./ms_cache:/root/.cache/modelscope \
    -v ./om_cache:/root/.cache/openmind \
    -v ./data:/app/data \
    -v ./output:/app/output \
    -p 7860:7860 \
    -p 8000:8000 \
    --shm-size 16G \
    --name llamafactory \
    llamafactory:latest

docker exec -it llamafactory bash

昇腾 NPU 用户：

# 根据您的环境选择镜像
docker build -f ./docker/docker-npu/Dockerfile \
    --build-arg INSTALL_DEEPSPEED=false \
    --build-arg PIP_INDEX=https://pypi.org/simple \
    -t llamafactory:latest .

# 根据您的资源更改 `device`
docker run -dit \
    -v ./hf_cache:/root/.cache/huggingface \
    -v ./ms_cache:/root/.cache/modelscope \
    -v ./om_cache:/root/.cache/openmind \
    -v ./data:/app/data \
    -v ./output:/app/output \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -p 7860:7860 \
    -p 8000:8000 \
    --device /dev/davinci0 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    --shm-size 16G \
    --name llamafactory \
    llamafactory:latest

docker exec -it llamafactory bash

AMD ROCm 用户：

docker build -f ./docker/docker-rocm/Dockerfile \
    --build-arg INSTALL_BNB=false \
    --build-arg INSTALL_VLLM=false \
    --build-arg INSTALL_DEEPSPEED=false \
    --build-arg INSTALL_FLASHATTN=false \
    --build-arg PIP_INDEX=https://pypi.org/simple \
    -t llamafactory:latest .

docker run -dit \
    -v ./hf_cache:/root/.cache/huggingface \
    -v ./ms_cache:/root/.cache/modelscope \
    -v ./om_cache:/root/.cache/openmind \
    -v ./data:/app/data \
    -v ./output:/app/output \
    -v ./saves:/app/saves \
    -p 7860:7860 \
    -p 8000:8000 \
    --device /dev/kfd \
    --device /dev/dri \
    --shm-size 16G \
    --name llamafactory \
    llamafactory:latest

docker exec -it llamafactory bash

• hf_cache：使用宿主机的 Hugging Face 缓存文件夹，允许更改为新的目录。
• ms_cache：类似 Hugging Face 缓存文件夹，为 ModelScope 用户提供。
• om_cache：类似 Hugging Face 缓存文件夹，为 Modelers 用户提供。
• data：宿主机中存放数据集的文件夹路径。
• output：将导出目录设置为该路径后，即可在宿主机中访问导出后的模型。

1. Wang et al. ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation. 2023. [arxiv]
2. Yu et al. Open, Closed, or Small Language Models for Text Classification? 2023. [arxiv]
3. Wang et al. UbiPhysio: Support Daily Functioning, Fitness, and Rehabilitation with Action Understanding and Feedback in Natural Language. 2023. [arxiv]
4. Luceri et al. Leveraging Large Language Models to Detect Influence Campaigns in Social Media. 2023. [arxiv]
5. Zhang et al. Alleviating Hallucinations of Large Language Models through Induced Hallucinations. 2023. [arxiv]
6. Wang et al. Know Your Needs Better: Towards Structured Understanding of Marketer Demands with Analogical Reasoning Augmented LLMs. KDD 2024. [arxiv]
7. Wang et al. CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning. ACL 2024. [arxiv]
8. Choi et al. FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs. 2024. [arxiv]
9. Zhang et al. AutoMathText: Autonomous Data Selection with Language Models for Mathematical Texts. 2024. [arxiv]
10. Lyu et al. KnowTuning: Knowledge-aware Fine-tuning for Large Language Models. 2024. [arxiv]
11. Yang et al. LaCo: Large Language Model Pruning via Layer Collaps. 2024. [arxiv]
12. Bhardwaj et al. Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic. 2024. [arxiv]
13. Yang et al. Enhancing Empathetic Response Generation by Augmenting LLMs with Small-scale Empathetic Models. 2024. [arxiv]
14. Yi et al. Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding. ACL 2024 Findings. [arxiv]
15. Cao et al. Head-wise Shareable Attention for Large Language Models. 2024. [arxiv]
16. Zhang et al. Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages. 2024. [arxiv]
17. Kim et al. Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models. 2024. [arxiv]
18. Yu et al. KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models. ACL 2024. [arxiv]
19. Huang et al. Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning. 2024. [arxiv]
20. Duan et al. Negating Negatives: Alignment without Human Positive Samples via Distributional Dispreference Optimization. 2024. [arxiv]
21. Xie and Schwertfeger. Empowering Robotics with Large Language Models: osmAG Map Comprehension with LLMs. 2024. [arxiv]
22. Wu et al. Large Language Models are Parallel Multilingual Learners. 2024. [arxiv]
23. Zhang et al. EDT: Improving Large Language Models Generation by Entropy-based Dynamic Temperature Sampling. 2024. [arxiv]
24. Weller et al. FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions. 2024. [arxiv]
25. Hongbin Na. CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering. COLING 2024. [arxiv]
26. Zan et al. CodeS: Natural Language to Code Repository via Multi-Layer Sketch. 2024. [arxiv]
27. Liu et al. Extensive Self-Contrast Enables Feedback-Free Language Model Alignment. 2024. [arxiv]
28. Luo et al. BAdam: A Memory Efficient Full Parameter Training Method for Large Language Models. 2024. [arxiv]
29. Du et al. Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model. 2024. [arxiv]
30. Ma et al. Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation. ICML 2024. [arxiv]
31. Liu et al. Dynamic Generation of Personalities with Large Language Models. 2024. [arxiv]
32. Shang et al. How Far Have We Gone in Stripped Binary Code Understanding Using Large Language Models. 2024. [arxiv]
33. Huang et al. LLMTune: Accelerate Database Knob Tuning with Large Language Models. 2024. [arxiv]
34. Deng et al. Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction. 2024. [arxiv]
35. Acikgoz et al. Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare. 2024. [arxiv]
36. Zhang et al. Small Language Models Need Strong Verifiers to Self-Correct Reasoning. ACL 2024 Findings. [arxiv]
37. Zhou et al. FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering. NAACL 2024. [arxiv]
38. Xu et al. Large Language Models for Cyber Security: A Systematic Literature Review. 2024. [arxiv]
39. Dammu et al. They are uncultured: Unveiling Covert Harms and Social Threats in LLM Generated Conversations. 2024. [arxiv]
40. Yi et al. A safety realignment framework via subspace-oriented model fusion for large language models. 2024. [arxiv]
41. Lou et al. SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling. 2024. [arxiv]
42. Zhang et al. Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners. 2024. [arxiv]
43. Zhang et al. TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models. 2024. [arxiv]
44. Zihong Chen. Sentence Segmentation and Sentence Punctuation Based on XunziALLM. 2024. [paper]
45. Gao et al. The Best of Both Worlds: Toward an Honest and Helpful Large Language Model. 2024. [arxiv]
46. Wang and Song. MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset. 2024. [arxiv]
47. Hu et al. Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Models. 2024. [arxiv]
48. Ge et al. Time Sensitive Knowledge Editing through Efficient Finetuning. ACL 2024. [arxiv]
49. Tan et al. Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions. 2024. [arxiv]
50. Song et al. Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters. 2024. [arxiv]
51. Gu et al. RWKV-CLIP: A Robust Vision-Language Representation Learner. 2024. [arxiv]
52. Chen et al. Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees. 2024. [arxiv]
53. Zhu et al. Are Large Language Models Good Statisticians?. 2024. [arxiv]
54. Li et al. Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning. 2024. [arxiv]
55. Ding et al. IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce. 2024. [arxiv]
56. He et al. COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities. 2024. [arxiv]
57. Lin et al. FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving. 2024. [arxiv]
58. Treutlein et al. Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data. 2024. [arxiv]
59. Feng et al. SS-Bench: A Benchmark for Social Story Generation and Evaluation. 2024. [arxiv]
60. Feng et al. Self-Constructed Context Decompilation with Fined-grained Alignment Enhancement. 2024. [arxiv]
61. Liu et al. Large Language Models for Cuffless Blood Pressure Measurement From Wearable Biosignals. 2024. [arxiv]
62. Iyer et al. Exploring Very Low-Resource Translation with LLMs: The University of Edinburghs Submission to AmericasNLP 2024 Translation Task. AmericasNLP 2024. [paper]
63. Li et al. Calibrating LLMs with Preference Optimization on Thought Trees for Generating Rationale in Science Question Scoring. 2024. [arxiv]
64. Yang et al. Financial Knowledge Large Language Model. 2024. [arxiv]
65. Lin et al. DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging. 2024. [arxiv]
66. Bako et al. Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization. 2024. [arxiv]
67. Huang et al. RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization. 2024. [arxiv]
68. Jiang et al. LLM-Collaboration on Automatic Science Journalism for the General Audience. 2024. [arxiv]
69. Inouye et al. Applied Auto-tuning on LoRA Hyperparameters. 2024. [paper]
70. Qi et al. Research on Tibetan Tourism Viewpoints information generation system based on LLM. 2024. [arxiv]
71. Xu et al. Course-Correction: Safety Alignment Using Synthetic Preferences. 2024. [arxiv]
72. Sun et al. LAMBDA: A Large Model Based Data Agent. 2024. [arxiv]
73. Zhu et al. CollectiveSFT: Scaling Large Language Models for Chinese Medical Benchmark with Collective Instructions in Healthcare. 2024. [arxiv]
74. Yu et al. Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment. 2024. [arxiv]
75. Xie et al. The Power of Personalized Datasets: Advancing Chinese Composition Writing for Elementary School through Targeted Model Fine-Tuning. IALP 2024. [paper]
76. Liu et al. Instruct-Code-Llama: Improving Capabilities of Language Model in Competition Level Code Generation by Online Judge Feedback. ICIC 2024. [paper]
77. Wang et al. Cybernetic Sentinels: Unveiling the Impact of Safety Data Selection on Model Security in Supervised Fine-Tuning. ICIC 2024. [paper]
78. Xia et al. Understanding the Performance and Estimating the Cost of LLM Fine-Tuning. 2024. [arxiv]
79. Zeng et al. Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructions. 2024. [arxiv]
80. Xia et al. Using Pre-trained Language Model for Accurate ESG Prediction. FinNLP 2024. [paper]
81. Liang et al. I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm. 2024. [arxiv]
82. Bai et al. Aligning Large Language Model with Direct Multi-Preference Optimization for Recommendation. CIKM 2024. [paper]
83. StarWhisper: 天文大模型 StarWhisper，基于 ChatGLM2-6B 和 Qwen-14B 在天文数据上微调而得。
84. DISC-LawLLM: 中文法律领域大模型 DISC-LawLLM，基于 Baichuan-13B 微调而得，具有法律推理和知识检索能力。
85. Sunsimiao: 孙思邈中文医疗大模型 Sumsimiao，基于 Baichuan-7B 和 ChatGLM-6B 在中文医疗数据上微调而得。
86. CareGPT: 医疗大模型项目 CareGPT，基于 LLaMA2-7B 和 Baichuan-13B 在中文医疗数据上微调而得。
87. MachineMindset：MBTI性格大模型项目，根据数据集与训练方式让任意 LLM 拥有 16 个不同的性格类型。
88. Luminia-13B-v3：一个用于生成 Stable Diffusion 提示词的大型语言模型。[demo]
89. Chinese-LLaVA-Med：中文多模态医学大模型，基于 LLaVA-1.5-7B 在中文多模态医疗数据上微调而得。
90. AutoRE：基于大语言模型的文档级关系抽取系统。
91. NVIDIA RTX AI Toolkit：在 Windows 主机上利用英伟达 RTX 设备进行大型语言模型微调的开发包。
92. LazyLLM：一个低代码构建多 Agent 大模型应用的开发工具，支持基于 LLaMA Factory 的模型微调.
93. RAG-Retrieval：一个全链路 RAG 检索模型微调、推理和蒸馏代码库。[blog]