FMInference/H2O · AI Repo StatsFMFMInference/H2O
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
LLM & foundation modelsPython 92created 2023-06-12 #gpt-3#heavy-hitters#high-throughput#kv-cache#large-language-models#sparsity