Make money doing the work you believe in
11 Open-Source Frameworks for Fine-Tuning, Serving, and Deploying LLMs
1. Hugging Face Transformers provides the Trainer API, which offers a comprehensive set of training features for fine-tuning any of the models on the Hub.
2. deepspeed.ai: It empowers ChatGPT-like model training with a single click, offering 15x speedup over SOTA RLHF systems with unprecedented cost reduction at all scales.
3. LLaMA Factory is an easy-to-use and efficient platform for training and fine-tuning large language models. With LLaMA Factory, you can fine-tune hundreds of pre-trained models locally without writing any code.
4. Unsloth AI makes it super easy for you to train models like Llama 3 locally or on platforms such as Google Colab and Kaggle.
5. Colossal-AI provides a collection of parallel components for you. We aim to support you in writing your distributed deep learning models just like how you write your model on your laptop.
6. Axolotl is a tool designed to streamline post-training for various AI models
7. LiteLLM (YC W23) is an open-source LLM Gateway with 24K+ stars on GitHub and trusted by companies like Rocket Money, Samsara, Lemonade, and Adobe.
8. vLLM is a fast and easy-to-use library for LLM inference and serving. Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry.
9. OpenLLM 🇫🇷 🇪🇺 allows developers to run any open-source LLMs (Llama 3.3, Qwen2.5, Phi3, and more) or custom models as OpenAI-compatible APIs with a single command.
10. FastChat is an open platform for training, serving, and evaluating large language model-based chatbots.
11. SkyPilot is an open-source framework for running AI and batch workloads on any infrastructure.
