# GenAI Interview Question Check # Section 1 — Production RAG ## 1\. Explain the architecture of a production RAG system. **Interview answer:** > A production RAG system typically has five major stages: ingestion, indexing, retrieval, generation, and evaluation/observability. > > During ingestion, documents are loaded from sources such as S3, SharePoint, databases, or APIs. We clean and normalize the documents, split them into meaningful chunks, attach metadata, generate embeddings, and store them in a vector database. > > At query time, the user's query is optionally rewritten or expanded. We then perform retrieval, usually using hybrid search combining semantic vector search and keyword search. The retrieved documents are passed through a reranker, and the top relevant chunks are sent to the LLM along with the prompt. > > Finally, we validate the response, apply guardrails, return the answer, and capture traces and metrics for observability. A good architecture to draw: ```text User Query | v Query Processing | +----------+----------+ | | Vector Search Keyword Search | | +----------+----------+ | Reranker | Top-K Context | v LLM | Guardrails/Validation | v Answer | Observability ``` * * * # 2\. How do you decide chunk size and overlap? **Answer:** > I don't choose chunk size purely based on a fixed number like 500 tokens. I consider the document structure, semantic boundaries, embedding model, retrieval performance, and context-window limitations. > > For example, for technical documentation, I prefer structure-aware chunking based on headings, paragraphs, sections, and code blocks. > > I would start with something like 300–800 tokens and a small overlap, then evaluate retrieval quality using a golden dataset. If important information frequently spans chunk boundaries, I increase overlap or use parent-child retrieval. ### Important point Don't say: > "I always use 500 tokens." Instead say: > "Chunking is an empirical optimization problem." * * * # 3\. Dense vs sparse vs hybrid retrieval ### Dense retrieval Uses embeddings. ```text Query → Embedding → Vector similarity → Documents ``` Good for semantic similarity. ### Sparse retrieval Usually keyword-based approaches such as BM25. ```text Query → Keyword matching → Documents ``` Good for exact terms, IDs, names, codes, etc. ### Hybrid Combines both. **Interview answer:** > In enterprise RAG, I generally prefer hybrid retrieval because dense retrieval handles semantic similarity while sparse retrieval handles exact lexical matches. For example, if the user searches for an invoice number or product code, keyword search can be much more reliable than embeddings. * * * # 4\. What is reranking? **Answer:** > Retrieval gives us candidate documents, but the initial ranking isn't always optimal. A reranker takes the query and retrieved candidates and calculates a more accurate relevance score, usually using a cross-encoder or another reranking model. > > So I generally think of retrieval as high-recall candidate generation and reranking as high-precision selection. ```text 100,000 documents ↓ Vector/BM25 retrieval ↓ Top 50 ↓ Reranker ↓ Top 5 ↓ LLM ``` * * * # 5\. How do you debug retrieval failures? This is **very likely to be asked**. **Answer:** > I first separate retrieval failure from generation failure. > > If the correct document isn't present in the retrieved context, it's a retrieval problem. If the correct document is present but the LLM still gives the wrong answer, it's a generation or prompting problem. > > For retrieval failures, I inspect the query, chunking, embeddings, metadata filters, top-K, similarity threshold, vector database configuration, and reranking. > > I also use a labeled evaluation dataset and measure metrics such as Recall@K, MRR, and NDCG. Debugging flow: ```text Wrong Answer | v Was correct context retrieved? | / \ No Yes | | Retrieval Generation problem problem ``` * * * # 6\. How do you evaluate RAG? Break evaluation into **retrieval and generation**. ### Retrieval * Recall@K * Precision@K * MRR * NDCG ### Generation * Faithfulness * Answer relevance * Context relevance * Groundedness * Citation correctness **Answer:** > I don't evaluate RAG as a single black box. I evaluate retrieval and generation separately. > > For retrieval, I measure whether the relevant chunks are actually retrieved. For generation, I evaluate whether the answer is grounded in those chunks, whether it answers the question, and whether it contains unsupported claims. > > I maintain a golden dataset containing queries, expected answers, and relevant documents, and run this dataset whenever I change the embedding model, chunking strategy, retriever, prompt, or LLM. * * * # 7\. Explain Recall@K Suppose: ```text Relevant documents = 5 Retrieved top 10 = 10 Relevant documents retrieved = 4 ``` Then: ```text Recall@10 = 4 / 5 = 80% ``` **Interview answer:** > Recall@K tells me how many of the relevant documents were successfully retrieved within the top K results. * * * # 8\. Explain MRR MRR focuses on the position of the **first relevant result**. If the first relevant document appears at: ```text Position 1 → 1 Position 2 → 1/2 Position 5 → 1/5 ``` **Answer:** > MRR is useful when I care about how early the first relevant result appears. It is particularly useful for evaluating ranking quality. * * * # 9\. How do you detect hallucinations? **Answer:** > I use a combination of automated evaluation and production monitoring. The response is compared against the retrieved context to determine whether claims are supported. I can use an LLM-as-a-judge evaluator, rule-based checks, citation validation, and human evaluation for high-risk use cases. > > I also design the prompt to explicitly instruct the model to answer only from the provided context and say that it doesn't know when sufficient evidence isn't available. But add: > LLM-as-a-judge shouldn't be the only evaluation mechanism because the judge itself can make mistakes. That's a strong production answer. * * * # 10\. What is query rewriting? Suppose the user asks: > "What about its pricing?" The query is ambiguous. A query rewriting component can convert it to: > "What is the pricing of Product X?" **Answer:** > Query rewriting transforms an ambiguous or conversational query into a search-optimized query. It is particularly useful in conversational RAG where the current question depends on previous turns. * * * # 11\. What is HyDE? **Answer:** > HyDE stands for Hypothetical Document Embeddings. Instead of directly embedding the user's question, we first ask an LLM to generate a hypothetical answer or document, embed that generated text, and use it for retrieval. > > The intuition is that a hypothetical answer may be semantically closer to the actual documents than the short user query. ```text User Query ↓ LLM ↓ Hypothetical Answer ↓ Embedding ↓ Vector Search ``` * * * # 12\. How do you reduce RAG latency? I would answer systematically: > I first instrument the pipeline and identify where the latency is coming from rather than optimizing blindly. > > I measure: > > * embedding latency > > * vector DB latency > > * reranker latency > > * LLM time-to-first-token > > * total generation time > > * external tool latency > > > Then I optimize accordingly. Possible optimizations: * caching * smaller embedding models * parallel retrieval * reduce retrieved chunks * reranking only when necessary * prompt compression * streaming * async API calls * connection pooling * model selection * response caching * batching * * * # 13\. How do you handle document updates? **Answer:** > I maintain document-level metadata such as document ID, version, source, timestamp, and chunk IDs. > > When a document changes, I identify all chunks associated with that document, remove or invalidate the old embeddings, generate embeddings for the updated chunks, and upsert them. > > I also maintain versioning so that stale documents don't remain retrievable. * * * # 14\. How do you handle a vector DB failure? **Answer:** > I treat the vector database as a dependency and design for graceful degradation. > > I would use connection timeouts, retries with exponential backoff, circuit breakers, health checks, and appropriate fallbacks. > > Depending on the application, we could fall back to keyword search, cached results, or return a controlled response rather than allowing the entire application to fail. * * * # 15\. How would you design RAG for millions of documents? **Answer:** > I would separate ingestion from query serving. > > Ingestion would be asynchronous and event-driven. Documents would be processed, chunked, embedded, and indexed through workers. > > For serving, I'd use a scalable API layer, horizontally scalable retrieval services, vector database partitioning/indexing, metadata filtering, caching, and potentially hierarchical retrieval. > > I'd also monitor ingestion lag, query latency, retrieval quality, vector DB utilization, token consumption, and error rates. * * * # Section 2 — Multi-Agent AI ## 1\. What is an AI agent? **Answer:** > An LLM application generally follows a predefined workflow, whereas an agent has the ability to reason about a task, decide what actions or tools are required, execute those actions, observe the results, and continue until the task is completed. > > In simple terms, an agent is an LLM combined with tools, state, decision-making, and an execution loop. ```text Goal ↓ LLM decides ↓ Tool ↓ Observation ↓ LLM decides again ↓ Final answer ``` * * * # 2\. Why use multiple agents? **Answer:** > I use multi-agent architecture when the problem naturally decomposes into specialized responsibilities. > > For example, a business analyst system might have separate SQL, data analysis, competitor research, and visualization agents. > > Specialization improves modularity, allows independent testing, and makes complex workflows easier to manage. However, I wouldn't automatically use multi-agent architecture because it increases latency, complexity, and cost. That last sentence is **very important**. * * * # 3\. What is an orchestrator agent? **Answer:** > The orchestrator is responsible for understanding the user's objective, decomposing the task, selecting the appropriate agents or tools, coordinating their execution, and passing their results to a synthesis or validation stage. Example: ```text User: "Why is ARR decreasing?" ↓ Orchestrator / | \ ↓ ↓ ↓ SQL Data Competitor Agent Agent Agent \ | / \ | / ↓ Synthesis ↓ Validation ↓ Answer ``` * * * # 4\. How does the orchestrator decide which agent to call? **Answer:** > I wouldn't rely only on free-form LLM reasoning. I would define explicit agent capabilities and routing criteria. > > The orchestrator first classifies the task and generates a structured plan. Based on that plan, it selects only the required agents. For example: ```text Question → Intent ↓ Required capabilities ↓ Agent selection ↓ Execution plan ``` You can also use: * structured output * tool schemas * routing rules * confidence thresholds * policy checks * * * # 5\. How do you prevent infinite agent loops? **Answer:** > I use multiple safeguards: > > 1. Maximum number of iterations. > > 2. Maximum execution time. > > 3. Maximum token budget. > > 4. State-based termination conditions. > > 5. Duplicate tool-call detection. > > 6. Retry limits. > > 7. Explicit success/failure states. > Example: ```python if state["iterations"] >= MAX_ITERATIONS: return "TERMINATE" ``` In production, I prefer several layers of protection rather than relying on a single condition. * * * # 6\. How do you handle conflicting agent outputs? **Answer:** > I don't blindly concatenate agent responses. I pass structured outputs to a synthesis or validation agent. > > Each agent should ideally provide its conclusion along with evidence, confidence, and source information. > > If two agents disagree, the synthesis layer should identify the conflict and either resolve it using evidence or explicitly report uncertainty. * * * # Section 3 — LangGraph ## 1\. Why LangGraph? **Answer:** > LangGraph is useful when I need explicit control over stateful, multi-step, cyclic workflows. > > Instead of treating an agent as a simple loop, I can model the workflow as a graph where nodes perform operations, edges define transitions, and conditional edges control routing. > > This makes complex agent workflows easier to debug, persist, retry, and observe. * * * ## 2\. What is a LangGraph node? > A node represents a unit of computation. It could be an LLM call, retrieval operation, SQL execution, validation step, or an entire sub-agent. * * * ## 3\. What is state? > State is the shared information passed through the workflow. It can contain the user query, retrieved documents, intermediate agent outputs, tool results, errors, iteration counts, and final response. Example: ```python state = { "query": "...", "plan": [], "documents": [], "agent_results": [], "errors": [], "final_answer": None } ``` * * * ## 4\. Why not simply use LangChain? **Answer:** > LangChain is useful for building LLM components and integrations, but when I have a complex stateful workflow involving branching, loops, retries, human approval, and multiple agents, LangGraph gives me more explicit workflow control. * * * # Section 4 — MCP ## 1\. What is MCP? **Answer:** > MCP, or Model Context Protocol, is a standardized protocol for connecting AI applications to external tools, resources, and data sources. > > Instead of implementing custom integrations for every agent and every tool, MCP provides a common interface through which an AI application can discover and invoke capabilities. * * * # 2\. MCP vs function calling This is a likely question. **Answer:** > Function calling is generally a model-level mechanism where the model produces structured arguments for a predefined function. > > MCP is an interoperability protocol that standardizes how AI applications discover and interact with external tools and resources. > > So function calling can be part of an MCP-based architecture, but MCP addresses the broader integration and interoperability problem. * * * # 3\. MCP vs A2A Memorize this distinction: > **MCP is primarily about agent-to-tool/resource interaction, whereas A2A is about agent-to-agent communication and collaboration.** Example: ```text Agent | | MCP ↓ Database / API / Search Agent | | A2A ↓ Another Agent ``` * * * # Section 5 — LLM Evaluation ## How would you evaluate an LLM application? **Answer:** > I evaluate it at multiple levels: component-level, workflow-level, and end-to-end. > > For RAG, I evaluate retrieval quality and generation quality separately. For agents, I additionally evaluate tool selection, task completion, planning quality, and execution reliability. > > I maintain a fixed evaluation dataset and run regression tests whenever I change the model, prompt, retriever, embeddings, or agent workflow. * * * # Section 6 — Guardrails ## How do you prevent prompt injection? **Answer:** > I treat retrieved documents and external tool outputs as untrusted data rather than instructions. > > I separate system instructions from retrieved content, constrain tool permissions, validate tool arguments, use input and output guardrails, and prevent the model from directly executing high-risk operations. > > For sensitive operations, I use deterministic authorization checks and potentially human approval rather than relying solely on the LLM. Very strong phrase: > **"Never trust the LLM to enforce security policy."** The LLM can help decide, but the actual authorization should be enforced by deterministic application code. * * * # Section 7 — Production Debugging ## RAG accuracy falls from 90% → 60%. What do you do? This is one of the **most important answers to prepare**. I'd answer: > First, I would avoid assuming that the model became worse. I would compare the production system against the last known-good version and isolate each layer. > > I would check: > > 1. Has the input/query distribution changed? > > 2. Has the document corpus changed? > > 3. Did chunking change? > > 4. Did the embedding model/version change? > > 5. Did vector indexing change? > > 6. Did retrieval parameters change? > > 7. Did reranking change? > > 8. Did the prompt change? > > 9. Did the LLM model/version change? > > 10. Did latency/timeouts cause incomplete execution? > > > I would use traces and an evaluation dataset to determine whether the failure is retrieval, generation, infrastructure, or data related. Then: ```text 90% → 60% | +-- Data? +-- Retrieval? +-- Embeddings? +-- Reranker? +-- Prompt? +-- Model? +-- Infrastructure? +-- Evaluation? ``` This is the kind of answer that demonstrates **production thinking**. * * * # Section 8 — Latency Optimization ## The application takes 8 seconds. How do you reduce it? **Answer:** > First I would establish a latency budget and trace every stage rather than optimizing the entire application blindly. For example: ```text Total = 8 sec Retrieval 0.5 sec Reranker 1.0 sec LLM 5.0 sec Tool calls 1.0 sec Network 0.5 sec ``` Now the LLM is clearly the biggest contributor. Potential solutions: * smaller/faster model * streaming * reduce context * parallel tool calls * caching * prompt optimization * speculative techniques where appropriate * async execution * eliminate unnecessary agent iterations * * * # Section 9 — Cloud / DevOps ## How would you deploy a RAG system? A strong answer: ```text API Gateway / LB | v FastAPI Pods / \ / \ RAG Service Agent Service | | +------+-------+ | +------------+------------+ | | | S3 Vector DB Redis | Documents Monitoring: CloudWatch / OpenTelemetry ``` Then explain: > I would containerize the application with Docker, deploy it on ECS or Kubernetes depending on the operational requirements, use S3 for document storage, a managed vector database for retrieval, Redis for caching where useful, and centralized logging and tracing for observability. * * * # Section 10 — Behavioral Questions ## Tell me about yourself For this interview, your answer should follow: ```text Current role/background ↓ GenAI experience ↓ RAG ↓ Agentic AI ↓ Production/backend ↓ Why this role ``` Don't spend 2 minutes discussing everything you've ever done. Aim for **60–90 seconds**. * * * ## "Tell me about your most complex GenAI project." Use this structure: **Problem → Architecture → Your contribution → Challenges → Solution → Metrics → Production impact** For example: > "The objective was to build an enterprise RAG/agentic system..." > > "I designed the orchestration layer..." > > "The main challenge was retrieval quality..." > > "I introduced hybrid retrieval and reranking..." > > "We evaluated using Recall@K and answer-level metrics..." > > "We also added tracing and monitoring..." **Do not invent metrics.** Use the actual numbers from your projects/resume. * * *