ovg-project/kvcached · AI Repo StatsOVovg-project/kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
Suy luận & chạy LLM cục bộPython Apache-2.0 239tạo 2025-05-27 #elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm