模型与应用评测
AI Agent Evaluation
原题LLM Evaluation
原题AI Agent Observability
原题What is evaluation-driven development for AI applications?
原题Why is AI only as good as our definition of done?
原题How do you evaluate LLM outputs? What metrics do you use?
原题Explain BLEU, ROUGE, and BERTScore. When would you use each?
原题What is G-Eval, and how does it use LLMs for evaluation?
原题What is LLM-as-a-judge evaluation, and what are its limitations?
原题How do you conduct human evaluation for AI systems?
原题What is red teaming, and how do you red team an LLM application?
原题How do you detect and measure hallucinations in LLM outputs?
原题What is adversarial testing for AI systems?
原题How do you build a regression test suite for AI applications?
原题What are benchmark suites (MMLU, HumanEval, GSM8K), and how do you interpret them?
原题What is benchmark contamination, and how do you guard against it?
原题Your new model version scores higher on every benchmark, but users say it got worse. Why does this happen, and how do you find the problem?
原题How do you evaluate a RAG system end-to-end?
原题How do you evaluate the quality of AI agents?
原题How would you evaluate an autonomous coding agent? Why can SWE-bench pass rates be misleading?
原题What is the difference between offline and online evaluation for AI systems?
原题How do you measure factual consistency in LLM outputs?
原题How do you evaluate multi-turn conversation quality?
原题What is the role of golden datasets in AI evaluation?
原题How do you build an eval set when there is no labelled ground truth and domain experts are expensive?
原题How do you implement continuous evaluation for production AI systems?
原题How do you evaluate bias in AI model outputs?
原题How do you compare two models or prompts in a statistically rigorous way?
原题How do you evaluate the robustness of an LLM application across input variations?
原题What are the key differences between evaluating traditional ML vs LLM applications?
原题How do you set up an evaluation framework from scratch for a new LLM application?
原题Your model passes one fairness metric but fails another. How do you handle conflicting audit results?
原题Your model was fair at deployment, but became biased 6 months later. How do you monitor continuously?
原题An external auditor cannot reproduce your model's results. How do you ensure audit reproducibility?
原题How do you structure red teaming for an LLM chatbot before launch?
原题How do you red team a multimodal model where text-only safety tests miss cross-modal attacks?
原题
暂无题目