大模型基础
What are foundation models, and how have they changed AI engineering?
原题What is a Large Language Model (LLM), and how does it work?
原题Inside ChatGPT: What Happens After You Hit Enter?
原题What is the Transformer architecture and how does it work?
原题What are the key components of the Transformer architecture?
原题Walk me through what happens, step by step, in one forward pass of a decoder-only Transformer.
原题What is tokenization in LLMs?
原题Explain BPE (Byte Pair Encoding).
原题Explain WordPiece and SentencePiece.
原题What is positional encoding, and why is it needed in Transformers?
原题What are embeddings?
原题Explain the Query(Q), Key(K), and Value(V) in attention.
原题What is self-attention, and how does it work in Transformers?
原题What is Cross Attention in Transformers?
原题Why do we scale the dot product attention by √dₖ in the Transformer architecture?
原题What is causal masking?
原题What are multi-head attention mechanisms? Why use multiple attention heads?
原题What are Feed-Forward Networks in LLMs?
原题What is Generative AI?
原题What is the context window in LLMs, and why does it matter?
原题Why is the context window limited in LLMs?
原题What is temperature in the context of LLMs, and how does it affect output?
原题Why is the first token slower than the rest in an LLM?
原题Explain Top-p (nucleus) sampling and Top-k sampling. How do they differ?
原题Compare greedy decoding, beam search, top-k, top-p, and temperature sampling. When does each fail?
原题What are logits, and how are they used in text generation?
原题What are stop tokens in LLMs, and how does an LLM know when to stop generating?
原题What is the difference between a stop token and a stop sequence?
原题What are skip connections (residual connections) in Transformers?
原题What is the difference between open-source and closed-source LLMs? When would you choose one over the other?
原题What is the difference between encoder-only, decoder-only, and encoder-decoder Transformer architectures?
原题What is KV cache, and how does it speed up inference?
原题Estimate the KV cache memory needed to serve a large model. How does it constrain batch size and context length?
原题KV Cache Compression
原题What is model distillation, and how is it used with LLMs?
原题What is Mixture of Experts (MoE), and how does it work in models like Mixtral?
原题What is the difference between dense and sparse models?
原题How does DeepSeek-V4 work?
原题What is Flash Attention?
原题What is Cross-Entropy Loss?
原题What is Grouped-Query Attention (GQA), and how does it differ from Multi-Head Attention (MHA)?
原题How does Sliding Window Attention work?
原题How do Attention Sinks work?
原题How does Rotary Position Embedding (RoPE) work, and why is it preferred over learned positional embeddings?
原题Explain Layer Normalization
原题Explain RMSNorm (Root Mean Square Layer Normalization)
原题Why do modern Transformers use Pre-LayerNorm (Pre-Norm) instead of Post-LayerNorm?
原题What are scaling laws (Chinchilla), and how do they guide model size vs training data decisions?
原题Your LLM keeps ignoring your instructions. How do you make it follow structured output formats?
原题Your LLM-powered tool hits the context window limit on long documents. How do you handle it?
原题Your LLM does not admit when it does not know the answer. How do you make it say "I don't know"?
原题Your LLM generates responses that are too verbose. How do you control response length?
原题Your LLM memorized proprietary training data and leaks it in responses. How do you prevent this?
原题Your LLM coding assistant generates outdated code using deprecated libraries. How do you fix it?
原题Your tokenizer splits important domain terms into meaningless subword pieces. How do you fix it?
原题Your Transformer's KV cache grows too large during long sequence generation. How do you manage memory?
原题Your Transformer runs out of memory on long documents due to quadratic self-attention. How do you scale it?
原题Your distilled student model fails on the complex reasoning that the teacher model handled. How do you close the gap?
原题After RLHF alignment, your LLM became safer but lost capability on hard tasks. How do you manage the alignment tax?
原题Your RLHF-trained LLM is gaming the reward model instead of being genuinely helpful. How do you fix reward hacking?
原题Your chatbot loses context after 10 turns in a conversation. How do you maintain a long conversation context?
原题Your chatbot fails when users switch topics mid-conversation. How do you handle topic switches?
原题Your QA system always generates an answer even when no answer exists in the context. How do you detect unanswerable questions?
原题Your summarization system hallucinated facts not in the original article. How do you fix it?
原题Your text generation repeats phrases in long outputs. How do you fix repetition?
原题Transformers work on text, so can they also understand images?
原题Small Language Models (SLMs)
原题Large Reasoning Models (LRMs)
原题Jev and System One Models
原题What are Autoregressive Models?
原题Explain the difference between autoregressive and masked language modeling.
原题Proximal Policy Optimization (PPO)
原题Direct Preference Optimization (DPO)
原题Group Relative Policy Optimization (GRPO)
原题Recursive Language Models (RLMs)
原题Continual Learning in LLMs
原题What is Recursive Self-Improvement (RSI)?
原题How do Diffusion Language Models (DLMs) work?
原题How Does LLM Watermarking Work?
原题How do RNNs and Transformers differ?
原题
暂无题目