LLM推理系统问答
Explain Multi-Query Attention vs. Grouped Query Attention. Why are they important for inference?
原题Walk me through what happens to the KV cache during a 1000-token generation on a 7B model.
原题What is the attention complexity bottleneck and how do modern systems address it?
原题How does RoPE enable context length extrapolation? What goes wrong if you naively extend context beyond training length?
原题Why is the decode phase memory-bandwidth-bound while prefill is compute-bound? What are the implications for serving system design?
原题You need to deploy a 70B LLM on 2 A100 80GB GPUs. What quantization strategy would you choose and why?
原题Explain the difference between weight-only quantization and weight-and-activation quantization. When does each apply?
原题What is the Straight-Through Estimator and why is it needed for QAT?
原题How does SmoothQuant address the activation outlier problem? What are its limitations?
原题Compare GPTQ and AWQ. Which would you use in production and under what conditions?
原题
暂无题目