LLM推理系统问答
What is the difference between prefill and decode in LLM inference, and why does it matter for system design?
原题Explain the iron triangle of inference. How would you make a decision when optimizing for a latency-sensitive chatbot application?
原题Why is memory bandwidth, rather than raw compute (FLOP/s), typically the bottleneck in LLM decoding?
原题How do you measure and report inference performance? What metrics would you instrument in a production serving system?
原题A colleague argues that since FLOP/s keeps growing faster than memory bandwidth, inference will soon be compute-bound. How do you respond?
原题Explain the difference between HBM bandwidth and NVLink bandwidth. When does each become the bottleneck?
原题Why does Apple Silicon perform competitively for LLM inference despite lower peak FLOP/s than discrete GPUs?
原题What is the significance of Tensor Core formats (FP16 vs INT8 vs FP8) for inference? When would you choose each?
原题How would you profile an LLM inference workload to identify whether it is compute-bound or memory-bandwidth-bound?
原题Describe the trade-offs between deploying inference on a Groq LPU versus an NVIDIA H100.
原题Explain Multi-Query Attention vs. Grouped Query Attention. Why are they important for inference?
原题Walk me through what happens to the KV cache during a 1000-token generation on a 7B model.
原题What is the attention complexity bottleneck and how do modern systems address it?
原题How does RoPE enable context length extrapolation? What goes wrong if you naively extend context beyond training length?
原题Why is the decode phase memory-bandwidth-bound while prefill is compute-bound? What are the implications for serving system design?
原题You need to deploy a 70B LLM on 2 A100 80GB GPUs. What quantization strategy would you choose and why?
原题Explain the difference between weight-only quantization and weight-and-activation quantization. When does each apply?
原题What is the Straight-Through Estimator and why is it needed for QAT?
原题How does SmoothQuant address the activation outlier problem? What are its limitations?
原题Compare GPTQ and AWQ. Which would you use in production and under what conditions?
原题Prove that speculative decoding produces outputs from the same distribution as the target model.
原题What is the acceptance rate α in speculative decoding, and what factors influence it?
原题How does Medusa differ from a separate draft model approach? What are the trade-offs?
原题In what scenario would you not recommend speculative decoding?
原题Describe the EAGLE architecture and explain why it achieves higher acceptance rates than basic Medusa.
原题Explain PagedAttention. How does it differ from pre-allocated contiguous KV cache management?
原题How does prefix caching work, and what kind of application benefits most from it?
原题What is the attention sink phenomenon? Why does naïve sliding window attention fail for StreamingLLM?
原题How would you design a KV cache eviction policy for a system serving requests with highly variable lengths?
原题Explain KV cache quantization. Can you quantize the KV cache more aggressively than model weights? Why or why not?
原题Explain why FlashAttention is faster than standard attention. What is the key bottleneck it addresses?
原题What is kernel fusion and why does it improve inference performance?
原题How does the online softmax trick enable tiled attention computation?
原题What improvements did FlashAttention-3 introduce for the H100, and how does it exploit H100-specific features?
原题When would you use Triton instead of writing raw CUDA? What are the trade-offs?
原题Explain continuous batching. Why is it superior to static batching for LLM serving?
原题What is “prefill piracy” and how does chunked prefill address it?
原题Describe the architecture of vLLM. What are its key innovations and where are its limitations?
原题How does disaggregated prefill/decode serving work? What are the engineering challenges?
原题What metrics would you use to evaluate and compare two LLM serving systems?
原题Explain tensor parallelism for a transformer’s MLP block. What communication collectives are required and when?
原题When would you choose tensor parallelism over pipeline parallelism for inference?
原题Explain the expert routing mechanism in an MoE model. What challenges does it create for inference serving?
原题How does pipeline bubble fraction affect pipeline parallelism efficiency? How is it mitigated?
原题What is expert offloading for large MoE models, and what are its performance implications?
原题How does Ring Attention achieve distributed long-context inference? What is its communication cost?
原题Compare Mamba (SSM) and Transformer attention for long-context inference. When would you use each?
原题How does LLMLingua compress prompts? What are the risks of prompt compression?
原题Explain the difference between YaRN and NTK-aware scaling for RoPE context extension.
原题What is RAG, and how does it relate to long-context inference? When would you choose one over the other?
原题You are asked to deploy a competitive language model on a smartphone with 6 GB RAM. What is your approach?
原题Explain Apple MLX and why it is well-suited for LLM inference on Apple Silicon.
原题What is the k-quant system in llama.cpp? How does it differ from standard uniform quantization?
原题Compare llama.cpp CPU inference vs GPU inference. When would you prefer CPU inference?
原题What is Hardware-Aware NAS? How does OFA (Once-For-All) enable efficient edge deployment?
原题What is inference-time compute scaling? How does it differ from training-time compute scaling?
原题Explain Process Reward Models (PRMs). How do they differ from Outcome Reward Models (ORMs)?
原题What is Best-of-N sampling? When is it efficient and when is it not?
原题Describe the GRPO training objective used in DeepSeek-R1. How does it differ from PPO?
原题For a math tutoring application, would you use a larger base model or a smaller model with extended inference compute? Justify your answer.
原题How would you debug a sudden TTFT p99 spike in a production LLM serving system?
原题What is goodput and why is it more meaningful than raw throughput or individual metric SLO compliance?
原题How would you design an A/B testing framework for comparing two LLM versions in production?
原题Explain how you would capacity plan for an LLM serving system expecting 10× user growth in 6 months.
原题How would you measure and improve Model FLOP Utilization (MFU)?
原题How do diffusion language models generate text differently from autoregressive models? What are the inference implications?
原题What is Mixture of Depths? How does it enable adaptive inference compute?
原题Why might photonic computing change LLM inference fundamentally? What are the current limitations?
原题How does multi-token prediction improve both training and inference efficiency?
原题Describe an “inference-first” model design philosophy. What architectural choices reflect this approach?
原题
暂无题目