AI 基础设施
How do you improve inference speed in production LLM deployments?
原题LLM optimization techniques
原题How do you select GPUs for LLM inference?
原题How does a GPU work for Deep Learning?
原题How do CUDA Kernels work?
原题How does a Google TPU work?
原题How does an LPU work?
原题Estimate the GPU memory needed to serve a 70B model (weights, KV cache, activations). Does it fit on a single 80 GB GPU?
原题Do the roofline math: how many tokens/sec can one H100 produce for a 70B model at batch size 1?
原题What is model parallelism vs data parallelism in distributed training?
原题What is tensor parallelism, and how does it help serve large models?
原题What is pipeline parallelism?
原题How does continuous batching improve LLM inference throughput?
原题What is speculative decoding, and how does it speed up inference?
原题How does Medusa (multi-head speculative decoding) work?
原题How does EAGLE (feature-level speculative decoding) work?
原题What is N-gram Speculation in LLMs, and how does it speed up generation?
原题What is KV cache, and how do you manage memory for it?
原题What is Paged Attention?
原题How does GGUF work?
原题How do you optimize inference for edge and mobile deployment?
原题What is model quantization (INT8, INT4, FP16, BF16), and how does it affect quality?
原题How do you implement auto-scaling for AI workloads?
原题What is the role of load balancing in AI serving infrastructure?
原题How do you manage GPU memory for serving multiple models?
原题What is model sharding, and when would you use it?
原题How do you implement request queuing and priority scheduling for AI services?
原题What are the cost trade-offs between self-hosted and API-based AI inference?
原题How do you handle cold start latency for serverless AI deployments?
原题How do you implement model caching to reduce redundant computations?
原题What is the difference between synchronous and asynchronous inference, and when do you use each?
原题What is FSDP (Fully Sharded Data Parallel), and how does it differ from DeepSpeed ZeRO?
原题What are TTFT, TPOT, and throughput, and how do they trade against each other?
原题How do you monitor and profile LLM inference in production (TTFT, inter-token latency, GPU utilization)?
原题What is model routing at the infrastructure level, and how do you route requests based on complexity and cost?
原题
暂无题目