Run LLMs on your own machine or server, and serve them fast on less GPU.
Ranked by total GitHub stars, refreshed every day.
| # | Repo |
|---|---|
| 1 | Inactive (no push for over a year, or archived)nomic-ai/gpt4all GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use. 77.3KC++ |
| 2 | Active (pushed in the last 90 days)1Panel-dev/1Panel 🔥 1Panel is a modern, open-source Linux server management panel and a lightweight AI management platform. 1Panel 是专门为 AI 服务器打造的管理面板。 37.5KGo |
| 3 | Active (pushed in the last 90 days)mozilla-ai/llamafile Distribute and run LLMs with a single file. 26.7KC++ |
| 4 | Slowing down (last push 3–12 months ago)UFund-Me/Qbot [🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ :news: qbot-mini: https://github.com/Charmve/iQuant 18.8KJupyter Notebook |
| 5 | Active (pushed in the last 90 days)FlashML-org/FreeToken FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently. 17.2KPython |
| 6 | Active (pushed in the last 90 days)gvergnaud/ts-pattern 🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference. 15.2KTypeScript |
| 7 | Active (pushed in the last 90 days)NVIDIA/TensorRT NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. 13.5KC++ |
| 8 | Active (pushed in the last 90 days)FareedKhan-dev/kimi-k3-in-c A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU. 12.2KC |
| 9 | Active (pushed in the last 90 days)openvinotoolkit/openvino OpenVINO™ is an open source toolkit for optimizing and deploying AI inference 11.2KC++ |
| 10 | Active (pushed in the last 90 days)triton-inference-server/server The Triton Inference Server provides an optimized cloud and edge inferencing solution. 11.1KPython |
| 11 | Active (pushed in the last 90 days)aws/amazon-sagemaker-examples Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker. 11KJupyter Notebook |
| 12 | Inactive (no push for over a year, or archived)huggingface/text-generation-inference(archived) Large Language Model Text Generation Inference 10.9KPython |
| 13 | Inactive (no push for over a year, or archived)mistralai/mistral-inference(archived) Official inference library for Mistral models 10.8KJupyter Notebook |
| 14 | |
| 15 | Active (pushed in the last 90 days)RunanywhereAI/runanywhere-sdks Production ready toolkit to run AI locally 10.3KC++ |
| 16 | Active (pushed in the last 90 days)LearningCircuit/local-deep-research ~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted. 9.6KPython |
| 17 | Active (pushed in the last 90 days)miurla/morphic An AI-powered search engine with a generative UI 9.3KTypeScript |
| 18 | Active (pushed in the last 90 days)ai-dynamo/dynamo A Datacenter Scale Distributed Inference Serving Framework 8.5KRust |
| 19 | Active (pushed in the last 90 days)n4ze3m/page-assist Use your locally running AI models to assist you in your web browsing 8.3KTypeScript |
| 20 | Inactive (no push for over a year, or archived)Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB 💎1MB lightweight face detection model (1MB轻量级人脸检测模型) 7.6KPython |