zhihu/ZhiLight · AI Repo StatsZHzhihu/ZhiLight
A highly optimized LLM inference acceleration engine for Llama and its variants.
LLM & foundation modelsC++ Apache-2.0 105created 2024-12-06 #cuda#deepseek-r1#gpt#inference-engine#llama#llm#llm-inference#llm-serving#model-serving#pytorch