LLM推理系统问答
Prove that speculative decoding produces outputs from the same distribution as the target model.
原题What is the acceptance rate α in speculative decoding, and what factors influence it?
原题How does Medusa differ from a separate draft model approach? What are the trade-offs?
原题In what scenario would you not recommend speculative decoding?
原题Describe the EAGLE architecture and explain why it achieves higher acceptance rates than basic Medusa.
原题Explain PagedAttention. How does it differ from pre-allocated contiguous KV cache management?
原题How does prefix caching work, and what kind of application benefits most from it?
原题What is the attention sink phenomenon? Why does naïve sliding window attention fail for StreamingLLM?
原题How would you design a KV cache eviction policy for a system serving requests with highly variable lengths?
原题Explain KV cache quantization. Can you quantize the KV cache more aggressively than model weights? Why or why not?
原题
暂无题目