Trending AI reposUpdated daily from GitHub
AI, coding & video AI repos ranked by star growth.
AI, coding & video AI repos ranked by star growth.
| # | Repo | Stars | 7d |
|---|---|---|---|
| 101 | SH Sharpiless/Yolov5-deepsort-inference Yolov5 deepsort inference，使用YOLOv5+Deepsort实现车辆行人追踪和计数，代码封装成一个Detector类，更容易嵌入到自己的项目中 Inference & local LLM | 1.5K | — |
| 102 | JO jofizcd/Soul-of-Waifu 🌌 Give a soul to your favorite characters. Soul of Waifu is a desktop roleplay & AI companion app featuring Live2D/VRM avatars, voice chat & local LLM. Evolve together across immersive chat, RPG adventures, and your desktop. Inference & local LLM | 1.5K | — |
| 103 | KI kitops-ml/kitops An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact. Inference & local LLM | 1.4K | — |
| 104 | MY mythrantic/ollama-docker Welcome to the Ollama Docker Compose Setup! This project simplifies the deployment of Ollama using Docker Compose, making it easy to run Ollama with all its dependencies in a containerized environment Inference & local LLM | 1.4K | — |
| 105 | EB ebhy/budgetml Deploy a ML inference service on a budget in less than 10 lines of code. Inference & local LLM | 1.3K | — |
| 106 | HU huawei-noah/Efficient-Computing Efficient computing methods developed by Huawei Noah's Ark Lab Inference & local LLM | 1.3K | — |
| 107 | TU tucan9389/awesome-ml-demos-with-ios The challenge projects for Inferencing machine learning models on iOS Inference & local LLM | 1.3K | — |
| 108 | KE kellyvv/PhoneClaw PhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference. Inference & local LLM | 1.3K | — |
| 109 | MO ModelCloud/GPTQModel LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang. Inference & local LLM | 1.3K | — |
| 110 | EA | 1.3K | — |
| 111 | HE HelgeSverre/ollama-gui A Web Interface for chatting with your local LLMs via the ollama API Inference & local LLM | 1.3K | — |
| 112 | TH thu-ml/SpargeAttn [ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference. Inference & local LLM | 1.3K | — |
| 113 | QU qualcomm/ai-hub-models Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices. Inference & local LLM | 1.3K | — |
| 114 | KA | 1.3K | — |
| 115 | FE FenTechSolutions/CausalDiscoveryToolbox Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included. Inference & local LLM | 1.2K | — |
| 116 | RU run-house/kubetorch Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra. Inference & local LLM | 1.2K | — |
| 117 | JM jmaczan/tiny-vllm Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM Inference & local LLM | 1.2K | — |
| 118 | ST StayLameBro/backburner Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable Inference & local LLM | 1.2K | — |
| 119 | PA patchy631/time-to-first-token A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking. Inference & local LLM | 1.2K | — |
| 120 | SY synchopate/llama-modchip Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs). Inference & local LLM | 1.2K | — |
| 121 | US usamahz/cpu-performance-engineering A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section. Inference & local LLM | 1.2K | — |
| 122 | HA HarryR/z80ai Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer. Inference & local LLM | 1.1K | — |
| 123 | AL alishobeiri/thread-notebook AI-powered Jupyter Notebook. Use AI to generate and edit code cells, automatically fix errors, and chat with your data Inference & local LLM | 1.1K | — |
| 124 | XI | 1.1K | — |
| 125 | HU | 1.1K | — |
| 126 | WI wici-ai/inferssd Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp. Inference & local LLM | 1.1K | — |
| 127 | AW awslabs/multi-model-server(archived) Multi Model Server is a tool for serving neural net models for inference Inference & local LLM | 1K | — |
| 128 | KE kennss/SiliconScope Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking Inference & local LLM | 1K | — |
| 129 | JE jegly/Box The most advanced, fully offline client-side AI suite on Android today. Inference & local LLM | 1K | — |
| 130 | FE feder-cr/jev jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU. Inference & local LLM | 968 | — |
| 131 | DR | 963 | — |
| 132 | KO kossakovsky/selfhost-ai One-command installer for a self-hosted AI stack on your own server: n8n, Ollama, Open WebUI, OpenClaw, Dify, Flowise, Supabase, ComfyUI, Qdrant & 30+ tools. Docker Compose, automatic HTTPS via Caddy, built-in monitoring. Free Zapier/Make alternative. Inference & local LLM | 962 | — |
| 133 | OP OpenIntroStat/ims 📚 Introduction to Modern Statistics - A college-level open-source textbook with a modern approach highlighting multivariable relationships and simulation-based inference. Inference & local LLM | 957 | — |
| 134 | OP openvinotoolkit/model_server A scalable inference server for models optimized with OpenVINO™ Inference & local LLM | 957 | — |
| 135 | HU huawei-noah/bolt Bolt is a deep learning library with high performance and heterogeneous flexibility. Inference & local LLM | 954 | — |
| 136 | AL alvarobartt/hf-mem A CLI to estimate inference memory requirements for Hugging Face models, written in Python. Inference & local LLM | 950 | — |
| 137 | IM ImageOptim/libimagequant Palette quantization library that powers pngquant and other PNG optimizers Inference & local LLM | 940 | — |
| 138 | ME metadist/synaplan Own the flow: one open-source AI platform for your knowledge, models and channels – web, WhatsApp, email, Outlook, Nextcloud & ownCloud. Local and cloud models side by side, with the cost on every answer. Inference & local LLM | 921 | — |
| 139 | SG sgaofen/humanizer-local-model Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts Inference & local LLM | 914 | — |
| 140 | KU kubernetes-sigs/lws LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving Inference & local LLM | 908 | — |
| 141 | PA Pangu-Immortal/MagicWX Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments. Inference & local LLM | 863 | — |
| 142 | GA gavamedia/deltafin Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents. Inference & local LLM | 859 | — |
| 143 | IN IncarnaMind/IncarnaMind From messy files to a checked report. A free, open-source desktop app for macOS, Windows and Linux: write and analyse with AI from your own files, check each sentence and figure against them, and export to Word. Meeting notes too. Works with ChatGPT, Claude, Gemini or a local model through Ollama. Inference & local LLM | 857 | — |
| 144 | ZH Zhen-Dong/Awesome-Quantization-Papers List of papers related to neural network quantization in recent AI conferences and journals. Inference & local LLM | 855 | — |
| 145 | GU guan-yuan/Awesome-AutoML-and-Lightweight-Models A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering. Inference & local LLM | 854 | — |
| 146 | TR triton-inference-server/pytriton PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments. Inference & local LLM | 850 | — |
| 147 | KY kytmanov/obsidian-llm-wiki-local Karpathy’s LLM Wiki, 100% local with Ollama. Drop Markdown notes → AI extracts concepts → your Obsidian wiki auto-links and grows. Zero sharing. Your notes stay yours. Inference & local LLM | 849 | — |
| 148 | RE RedisAI/redis-inference-optimization A Redis module for serving tensors and executing deep learning graphs Inference & local LLM | 838 | — |
| 149 | HE Helldez/BigMoeOnEdge Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server. Inference & local LLM | 832 | — |
| 150 | AA aayushch/laya Laya is an open-source, local-first AI notification command center that aggregates Slack, Gmail, GitHub, Jira, Notion, Outlook, Calendar (and more) notifications using local LLMs via Ollama and LM Studio. Supports cloud models via BYOK. Inference & local LLM | 819 | — |