LLM推理系统问答
Explain tensor parallelism for a transformer’s MLP block. What communication collectives are required and when?
原题When would you choose tensor parallelism over pipeline parallelism for inference?
原题Explain the expert routing mechanism in an MoE model. What challenges does it create for inference serving?
原题How does pipeline bubble fraction affect pipeline parallelism efficiency? How is it mitigated?
原题What is expert offloading for large MoE models, and what are its performance implications?
原题How does Ring Attention achieve distributed long-context inference? What is its communication cost?
原题Compare Mamba (SSM) and Transformer attention for long-context inference. When would you use each?
原题How does LLMLingua compress prompts? What are the risks of prompt compression?
原题Explain the difference between YaRN and NTK-aware scaling for RoPE context extension.
原题What is RAG, and how does it relate to long-context inference? When would you choose one over the other?
原题
暂无题目