Verified record · from the answer index

awq-quantization

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preserva

skill:orchestra-research/ai-research-skills/awq-quantization · published by orchestra-research · source repository