vllm-project/llm-compressor

State-of-the-art LLM compression, built for production inference with vLLM