LLM推理系统问答
What is the difference between prefill and decode in LLM inference, and why does it matter for system design?
原题Explain the iron triangle of inference. How would you make a decision when optimizing for a latency-sensitive chatbot application?
原题Why is memory bandwidth, rather than raw compute (FLOP/s), typically the bottleneck in LLM decoding?
原题How do you measure and report inference performance? What metrics would you instrument in a production serving system?
原题A colleague argues that since FLOP/s keeps growing faster than memory bandwidth, inference will soon be compute-bound. How do you respond?
原题Explain the difference between HBM bandwidth and NVLink bandwidth. When does each become the bottleneck?
原题Why does Apple Silicon perform competitively for LLM inference despite lower peak FLOP/s than discrete GPUs?
原题What is the significance of Tensor Core formats (FP16 vs INT8 vs FP8) for inference? When would you choose each?
原题How would you profile an LLM inference workload to identify whether it is compute-bound or memory-bandwidth-bound?
原题Describe the trade-offs between deploying inference on a Groq LPU versus an NVIDIA H100.
原题
暂无题目