Gaurav Sen — searchable YouTube transcripts & AI Q&A

AI Interview #9 - Why is the Agent so slow?

The speaker argues that slow agentic applications often stem from synchronous tool calls, which force each tool to wait for the previous one's response. He recommends making independent tool calls asynchronous, filtering MCP server tools to only those needed, and caching static data like employee leave balances to reduce latency and context bloat. He also suggests composing tools with overlapping tasks into a single tool, while cautioning that batching across multiple agents introduces unnecessary complexity for a single-agent performance review system.

AI Interview #8 - Why is the chatbot so slow?

The chatbot's slowness in a RAG-based system is most likely due to the vector database search, especially when the database contains millions of vectors. The speaker recommends tuning the ANN algorithm, running searches in parallel, using caching with Redis, and horizontally scaling or sharding the database. For document uploads, the semantic chunker is often the bottleneck because it repeatedly computes similarity scores.

AI Interview #7 - Why is this chatbot failing?

The speaker argues that a RAG pipeline's accuracy issues often stem from retrieving too many chunks, which pollutes the context and causes the LLM to hallucinate. He recommends reducing the top-k from 20-25 to 5-10 and adding a reranker (cross-encoder) to filter chunks by token-level similarity, which can improve accuracy by 5-10% at the cost of some latency.

Is system design still relevant in 2026?

Software engineers should focus on AI literacy and foundational concepts rather than fearing AI, argues Arvind. He emphasizes that engineers' unique value lies in evaluating AI-generated code and making design decisions, which requires taste and context that AI lacks. Building in public and not waiting for permission are key to staying relevant.

AI Interview #6 - 1M context window in LLM?

Tanisha explains that a RAG architecture solves the problem of large documents not fitting into an LLM's context window by chunking documents, converting chunks into embeddings, and storing them in a vector database for similarity search. The user query is embedded with the same model, relevant chunks are retrieved, and the LLM generates a response based on that context, reducing hallucination. For PDFs with tables or images, she suggests using layout detection to apply different parsers for text and table elements.