Repo AI đang lênCập nhật hằng ngày từ GitHub

Yolov5 deepsort inference，使用YOLOv5+Deepsort实现车辆行人追踪和计数，代码封装成一个Detector类，更容易嵌入到自己的项目中

🌌 Give a soul to your favorite characters. Soul of Waifu is a desktop roleplay & AI companion app featuring Live2D/VRM avatars, voice chat & local LLM. Evolve together across immersive chat, RPG adventures, and your desktop.

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

Welcome to the Ollama Docker Compose Setup! This project simplifies the deployment of Ollama using Docker Compose, making it easy to run Ollama with all its dependencies in a containerized environment

Deploy a ML inference service on a budget in less than 10 lines of code.

Efficient computing methods developed by Huawei Noah's Ark Lab

The challenge projects for Inferencing machine learning models on iOS

PhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference.

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

Official Inference Code of EAGLE-4

A Web Interface for chatting with your local LLMs via the ollama API

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

Pytorch-Named-Entity-Recognition-with-BERT

Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.

Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).

A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.

Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer.

AI-powered Jupyter Notebook. Use AI to generate and edit code cells, automatically fix errors, and chat with your data

Dataflow compiler for QNN inference on FPGAs

A pytorch quantization backend for optimum

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Multi Model Server is a tool for serving neural net models for inference

Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking

The most advanced, fully offline client-side AI suite on Android today.

jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.

Official implementation of Half-Quadratic Quantization (HQQ)

One-command installer for a self-hosted AI stack on your own server: n8n, Ollama, Open WebUI, OpenClaw, Dify, Flowise, Supabase, ComfyUI, Qdrant & 30+ tools. Docker Compose, automatic HTTPS via Caddy, built-in monitoring. Free Zapier/Make alternative.

📚 Introduction to Modern Statistics - A college-level open-source textbook with a modern approach highlighting multivariable relationships and simulation-based inference.

A scalable inference server for models optimized with OpenVINO™

Bolt is a deep learning library with high performance and heterogeneous flexibility.

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

Palette quantization library that powers pngquant and other PNG optimizers

Own the flow: one open-source AI platform for your knowledge, models and channels – web, WhatsApp, email, Outlook, Nextcloud & ownCloud. Local and cloud models side by side, with the cost on every answer.

Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts

LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving

Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.

Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.

From messy files to a checked report. A free, open-source desktop app for macOS, Windows and Linux: write and analyse with AI from your own files, check each sentence and figure against them, and export to Word. Meeting notes too. Works with ChatGPT, Claude, Gemini or a local model through Ollama.

List of papers related to neural network quantization in recent AI conferences and journals.

A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.

PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.

Karpathy’s LLM Wiki, 100% local with Ollama. Drop Markdown notes → AI extracts concepts → your Obsidian wiki auto-links and grows. Zero sharing. Your notes stay yours.

A Redis module for serving tensors and executing deep learning graphs

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

Laya is an open-source, local-first AI notification command center that aggregates Slack, Gmail, GitHub, Jira, Notion, Outlook, Calendar (and more) notifications using local LLMs via Ollama and LM Studio. Supports cloud models via BYOK.