Retrieval-Augmented Generation (RAG) is a must-know area for LLM-application and algorithm roles in 2026. This article follows one real build pipeline: why RAG → chunking → embeddings → retrieval → reranking → multi-hop → hallucination & evaluation. For each question you get “the answer skeleton interviewers want to hear + engineering pitfalls.”
What problem does RAG solve? How does it relate to long context and fine-tuning?Must-answer
RAG supplies generation with “currently trustworthy knowledge” from external retrieval, solving three problems: knowledge freshness, private knowledge, and hallucination. Versus the other two paths: long context stuffs documents into the window — intuitive, but cost grows linearly with tokens and models get “lost in the middle” of long text; fine-tuning writes knowledge into weights — expensive, slow to update, hard to trace. In practice they complement each other: RAG owns freshness and provenance, long context is for “one deep read,” and fine-tuning is for style, format, and domain tone.
When is RAG actually the wrong approach?Advanced
① Strong-reasoning problems (math, multi-step logic) — retrieval can’t fetch a “reasoning process,” so focus on reasoning models / CoT instead; ② Low latency with high QPS — one retrieval pass plus long context raises latency noticeably; ③ The answer simply isn’t in the external store (purely internalized knowledge) — retrieval only adds noise. Ask “where does the answer live” before deciding to retrieve, rather than slapping RAG on everything.
How should you chunk? Fixed-length, recursive, or semantic?Must-answer
There’s no universal answer — chunk so that “one chunk can independently answer a class of questions.” For heavily structured documents (Markdown headings/tables/sections), cut along the structure; for ordinary prose use recursive character splitting with an overlap of 10–20% to preserve context boundaries; for code, split by function/class. Small chunks retrieve precisely but can lack context; large chunks carry full context but add noise and hit model-window and embedding-length limits. Tune chunking together with the downstream question types — don’t guess.
Why is hybrid search (vectors + BM25 full-text) so common?Advanced
Dense vectors excel at semantic similarity but go wrong on proper nouns / IDs / exact terms (contract numbers, model numbers, names); BM25 excels at exact-term matching but doesn’t understand semantics. They complement each other — common practice is RRF (Reciprocal Rank Fusion) or weighted score fusion, then a reranker does fine-grained ordering. In production a typical flow is “vector recall Top200 + BM25 Top200 → fuse → Rerank Top20.”
Why split retrieval into two stages (recall + rerank)?Must-answer
The recall stage must be broad (don’t miss candidates across a huge corpus), so it uses efficient vector/inverted indexes to take Top-K; the rerank stage must be precise (computing fine-grained relevance over tens of candidates), so it can use a stronger Cross-Encoder — which fully interactively encodes the query with each doc. It’s accurate but expensive per call, so it only runs over Top-K. Broad first, then precise — balancing quality and cost.
How do you retrieve for multi-hop / complex questions?Advanced
Simple RAG does one retrieval and one generation. For complex questions you either decompose into sub-questions (query rewriting / routing to sub-retrievals), iterate retrieval (supplement based on what’s been generated), or go GraphRAG-style — build an entity-relation graph first, then pull the relevant subgraph. The interview key is explaining “why a single retrieval isn’t enough” and the rationale for your decomposition strategy (question type, knowledge granularity).
If relevant content is retrieved, why do models still answer wrong or hallucinate?Advanced
Three root causes: ① The context never reached the model — recall missed the key passage (a recall problem); ② It was provided but the model ignored it — the context was too long and diluted by irrelevant passages, or the prompt didn’t demand “answer only from the material”; ③ The material itself is self-contradictory. Fix it methodically: first check offline whether “the answer is in the Top-K passages” to attribute blame to retrieval or generation, then improve recall, or add grounding/citation constraints and post-retrieval denoising.
How do you make answers traceable and verifiable?Advanced
Force citations in generation: in the prompt require the model to cite “chunk ids” and answer only from the given material; surface citation anchors in the frontend. During evaluation, measure whether the citations actually support the answer (attribution / groundedness), and treat “sentences with no source” as a hallucination signal.
How do you evaluate a RAG system in production?Bonus
Evaluate in layers: the retrieval layer uses recall, hit rate, MRR/NDCG (is the answer in Top-K); the generation layer uses answer correctness (including LLM-as-Judge scoring and human sampling). In engineering, maintain at least one evaluation set of query → expected document → expected answer, and run an offline regression every time you change chunking / embeddings / reranking. Don’t rely on “it looks better by eye.”
How do you monitor degradation in production?Bonus
Instrument three things: retrieval hit rate (which document a user opened / follow-up asked), the share of answers without citations, and bad-case feedback loops (negative signals). After a knowledge-base update, rerun recall regression on the new corpus so a batch of new chunks doesn’t silently degrade quality.
It’s layered from “must-answer” to “advanced” to “bonus”: foundations, chunking & hybrid search, two-stage recall + rerank, multi-hop retrieval, hallucination & grounding, and the differentiator — evaluation and monitoring. It largely covers applied / AI-engineer roles; for senior roles, try to pair each question with a real production story of your own.
First self-check for gaps using this article; then pick 1–2 areas (e.g. hybrid search or evaluation) and actually build a small system. At the interview, talk in terms of “the pitfalls I hit + before/after data” — far stronger than reciting definitions. The course includes an end-to-end project and 200+ curated questions.
Follow 「兔老板工作室」 on Xiaohongshu and DM “资料” to get a free PDF of curated big-tech interview questions; there are also an LLM evaluation platform and Agent-evaluation learning materials where you can try the eval pipeline online.
The content is location-agnostic — start by self-checking the layers above. If you’ll interview at Chinese companies back home, practice telling “what RAG systems you built and what broke” fluently in both Chinese and English, and time it to the domestic hiring season. If interviewing locally, engineering-granular explanations of chunking/reranking/evaluation carry more weight. For hands-on project coaching and realistic mocks, you can book remote lessons over WeChat.
📚 Free long-form series:LLM algorithm-role high-frequency checklist · Agent system design thinking · LLM fine-tuning & alignment points · LLM inference optimization points · Big-tech talent programs compared · Résumé & project pitfall guide · AI Jobs for Chinese Students in the US · AI Infra free practice question bank