Design a Retrieval-Augmented Generation (RAG) system. The concrete variant varies by interviewer but the core architecture is consistent. Observed variants include: (1) Generic RAG pipeline design; (2) A law firm memo Q&A agent — users ask questions and an AI agent answers based on memos scraped from big law firm websites, requiring RAG + LLM + web crawler with chunking strategy; (3) A 'Review Table' product — users submit natural-language prompts, a vault stores a large corpus of documents, and a service must retrieve and return relevant information using LLM + RAG. HR typically provides a more detailed prompt in advance, so candidates know which variant they will face. The interview is conducted in-person/onsite as one of 3–4 rounds.
Naive approach with serious trade-off — being authored.
Solid baseline with reasonable trade-offs — being authored.
Production-grade approach with explicit trade-off rationale — being authored.
Naive approach with serious trade-off — being authored.
Solid baseline with reasonable trade-offs — being authored.
Production-grade approach with explicit trade-off rationale — being authored.
Naive approach with serious trade-off — being authored.
Solid baseline with reasonable trade-offs — being authored.
Production-grade approach with explicit trade-off rationale — being authored.
Naive approach with serious trade-off — being authored.
Solid baseline with reasonable trade-offs — being authored.
Production-grade approach with explicit trade-off rationale — being authored.
Naive approach with serious trade-off — being authored.
Solid baseline with reasonable trade-offs — being authored.
Production-grade approach with explicit trade-off rationale — being authored.
Cover happy path. Clarify scope. Identify the obvious bottleneck. Pick a reasonable storage and reasonable scaling approach.
All of the above plus: explicit failure handling, durability vs latency trade-offs, choose the right batching/caching strategy, articulate why.
All of the above plus: organizational concerns (rollout, migration, on-call), quantitative analysis, multi-region considerations, what could go wrong with the proposed solution at 10x scale.
Common mistakes: Over-indexing on LLM/RAG internals (prompt engineering, model choice) while ignoring web crawler scalability, which is a primary evaluation dimension; Spending disproportionate time on RAG details without covering the full system end-to-end; Not following the interviewer's direction when they pivot to a specific sub-topic (e.g., crawler scale); Treating the design as purely an ML problem rather than a distributed systems problem; Not probing the interviewer for the specific product variant, leading to preparation mismatch
Interviewer hints: Interviewer steered candidate toward web crawler scalability when candidate stayed on LLM internals; HR provides a detailed prompt before the interview so candidates know the specific variant
What passers do: Reading and using the detailed prompt HR provides in advance to tailor preparation; Covering all system layers: crawler → chunking → embedding → vector store → retrieval → generation → evaluation; Proactively addressing web crawler scalability without waiting to be prompted; Following the interviewer's lead when they emphasize a particular component; Using Gemini or other LLMs to simulate mock RAG system design interviews beforehand
Why people fail: Candidate prepared extensively on LLM-specific knowledge but neglected web crawling infrastructure — failed SD round while other rounds were 'strong'; Spending almost all time on RAG details, never reaching scalability/infrastructure discussion; Not knowing LLM system design fundamentals at all (one candidate 'blanked completely' — 一脸懵逼秒挂); Assuming the question would be a generic backend design (e.g., Google Drive) and not preparing for AI/LLM system design
Edge cases probed: Web crawler scalability at hundreds of domains — rate limiting, deduplication, scheduling; Very long documents exceeding LLM context window; Overlapping or conflicting information across retrieved chunks; Low-recall scenarios where relevant doc is not retrieved; Evaluation metrics for RAG (both retrieval and generation sides)
Alternative approaches: Keyword / sparse retrieval only (BM25) (Simpler, no embedding infrastructure, but misses semantic matches; weaker for paraphrase-heavy queries.); Full fine-tuned LLM without retrieval (Avoids retrieval latency but requires expensive re-training when knowledge updates; hallucination risk higher for domain-specific facts.); Hybrid retrieval (dense + sparse) (Better recall than either alone; adds system complexity and requires score normalization (RRF).); Graph-based RAG / Knowledge Graph augmentation (Captures relational context well; much higher engineering cost to build and maintain the graph.)
What you just read — canonical solution, follow-up arc, what passing candidates actually did — exists for all 9 Harvey AI questions, refreshed monthly from new candidate reports.