LLM Agent Development Interview Guide: Tool Calling, Planning, and Memory Systems Full Assessment

Background

Last year I started exploring the AI Agent direction, tinkering with everything from LangChain to AutoGen and various Agent frameworks. Later, I interviewed at several companies building Agent products and found that Agent development interviews are very different from traditional LLM interviews—they focus more on your understanding of system design rather than individual model details. Interviewers ask questions like "how to design an Agent that can call tools," "how to make an Agent do long-term planning," and "how do multiple Agents collaborate." I've organized the 5 major assessment areas for Agent interviews, each with common interview questions and answer approaches, hoping to help anyone currently preparing.

Interview Process Review

The interview process for AI Agent positions typically follows: resume screening → first technical round (Agent fundamentals + tool calling) → second technical round (planning + memory system design) → third technical round (multi-Agent system design or deep project dive) → HR round. A very obvious characteristic of Agent interviews is that system design questions carry a large weight. The first round might ask you to design a customer service Agent architecture; the second round, an Agent that can autonomously complete programming tasks; the third round, a multi-Agent collaboration system. Interviewers care less about which model you use and more about how you orchestrate models, tools, memory, and planning components. Additionally, Agent interviews highly value engineering practice—if you can describe pitfalls you've encountered in actual development (like unstable tool calling formats, Agents getting stuck in infinite loops, context window overflow, etc.), it's a big plus.

Question Collection

1. Agent Architecture Design

Common Question 1: What's the difference between ReAct and Plan-and-Execute? What scenarios is each suited for?

Answer approach: ReAct is a reasoning-action alternating pattern—thinking then acting at each step, observing results, then continuing to think. Advantages: flexible and adaptive, can adjust strategy based on intermediate results; disadvantages: high token consumption and easy to lose direction in long tasks. Plan-and-Execute is a plan-then-execute pattern—first create a complete plan, then execute step by step. Advantages: good global view and high execution efficiency; disadvantages: plans may not adapt to environmental changes, requiring replanning. Selection advice: use ReAct for simple exploratory tasks (like information retrieval, simple Q&A), use Plan-and-Execute for complex multi-step tasks (like project development, data analysis). In practice, they're often combined: coarse-grained planning first, then ReAct within each step.

Common Question 2: How to design a reliable Agent architecture?

Answer approach: Key components of a reliable Agent architecture: intent recognition (determining if user requests are within Agent capabilities, avoiding out-of-scope operations); tool selection (choosing appropriate tools for tasks, avoiding irrelevant tool calls); error recovery (automatically retrying or switching approaches on tool call failure, rather than just reporting errors); state management (maintaining Agent's current state, avoiding duplicate operations); safety guardrails (limiting Agent's operation scope, e.g., prohibiting file deletion or dangerous command execution). Architecturally, I recommend layered design: perception layer (understanding input) → decision layer (selecting actions) → execution layer (calling tools) → feedback layer (processing results), each layer independently testable.

2. Tool Calling (Function Calling)

Common Question 3: What is the principle of Function Calling? How to design good tool descriptions?

Answer approach: The essence of Function Calling is having the LLM output structured tool call requests (function name + parameters) rather than free text. Mainstream implementation: OpenAI's function calling injects tool definitions (name, description, parameter JSON Schema) into the system prompt, and the model, after fine-tuning, can output tool calls in a specific format. Keys to designing good tool descriptions: semantic names (e.g., search_web not tool1); specific descriptions (explaining functionality, applicable scenarios, notes); clear parameters (type, range, defaults, required/optional); rich examples (providing typical call examples). Common pitfalls: too many tools (model selection accuracy drops with 20+ tools—suggest dynamic tool loading by task); ambiguous descriptions (model easily confuses tools with similar functions); parameter formats (complex nested parameters error-prone—suggest flattening).

Common Question 4: How to handle tool call failures?

Answer approach: Tool call failure handling strategies: retry (for transient errors like network jitter, exponential backoff retry 2-3 times); parameter correction (if parameter format error, have LLM correct parameters based on error message and retry); tool replacement (if a tool is unavailable, try other tools for the same functionality); graceful degradation (if no alternative tool, have LLM answer with its own knowledge and inform user information may be incomplete); user confirmation (for irreversible operations, ask user whether to retry or switch approaches after failure). The key is to add error handling logic to the Agent loop rather than simply try-catching and returning errors. A good Agent should be like an experienced employee—when encountering problems, it finds solutions rather than giving up immediately.

3. Planning and Reasoning

Common Question 5: What's the difference between CoT and ToT? How to make Agents plan better?

Answer approach: CoT (Chain of Thought) is linear reasoning, deriving answers step by step. ToT (Tree of Thought) is tree-shaped reasoning, exploring multiple branches at each decision point and selecting the optimal path after evaluation. CoT suits high-certainty tasks (math, logic), ToT suits exploration-needed tasks (creative generation, strategic planning). Methods for better Agent planning: task decomposition (breaking large tasks into small steps with clear inputs/outputs); self-reflection (Reflexion—having Agent evaluate its output quality, replanning if unsatisfied); external planner (using a dedicated planning model to generate plans, execution model follows plans); dynamic replanning (automatically adjusting plans when finding them infeasible during execution). The most effective method in practice is the task decomposition + self-reflection combination.

Common Question 6: What to do when an Agent gets stuck in an infinite loop?

Answer approach: Agent infinite loops are one of the most common problems in actual development. Causes: tool call results don't meet conditions (Agent repeatedly calls the same tool but results are always wrong); unreasonable planning (Agent bounces between two steps); context loss (long conversations cause Agent to forget previous decisions, repeating actions). Solutions: set maximum iteration count (e.g., max 10 tool call rounds, force termination if exceeded); detect repeated operations (if same tool called 3 consecutive times with similar parameters, trigger exception handling); state summarization (periodically summarize conversation history to avoid context loss); introduce metacognition (have Agent evaluate "am I making progress" at each step, switch strategies if no progress for multiple steps). The most practical solution is maximum iterations + repeat detection—simple and effective.

4. Memory Systems

Common Question 7: How to design an Agent's memory system? What's the difference between short-term and long-term memory?

Answer approach: Short-term memory is the current conversation context, stored in the LLM's context window, including conversation history, tool call results, and intermediate reasoning. Long-term memory is cross-session persistent information, including user preferences, historical interaction summaries, and knowledge bases. Short-term memory challenge: limited context window (128K tokens isn't enough for long conversations)—solution is sliding window + summary compression. Long-term memory implementations: vector databases (like Chroma, Pinecone, storing and retrieving semantically similar memories); structured storage (like Redis/SQL, storing structured info like user profiles and preferences); knowledge graphs (storing entity relationships, supporting complex reasoning). The key to memory retrieval is relevance + recency: retrieving memories relevant to the current task while prioritizing the most recent. MemGPT is a good reference architecture that uses the operating system's virtual memory concept to manage Agent memory.

Common Question 8: How to solve context loss in long conversations?

Answer approach: Long conversation context loss is a core challenge in Agent development. Solutions: sliding window (keep only the most recent N conversation rounds—simple but loses early info); summary compression (use LLM to summarize historical conversations, retaining key info while compressing token count); hierarchical memory (recent conversations fully retained, mid-term conversations summarized, distant conversations only key conclusions); memory retrieval (store conversations in vector database, retrieve relevant history based on current question); working memory (maintain a "current task state" variable containing goals, progress, to-do items, updated each round). The most effective in practice is the summary compression + working memory combination: periodically summarize conversations while maintaining a structured working memory, ensuring the Agent always knows "what am I doing, where am I, what's next."

5. Multi-Agent Collaboration

Common Question 9: How to design multi-Agent systems? What collaboration patterns exist?

Answer approach: Main multi-Agent collaboration patterns: master-slave (one master Agent handles task allocation and coordination, multiple slave Agents execute specific subtasks, like AutoGen's GroupChat); peer-to-peer (multiple Agents collaborate as equals, communicating via message passing, like CrewAI); pipeline (tasks pass sequentially between Agents, each responsible for one stage, like writing Agent → review Agent → publishing Agent); debate (multiple Agents propose different solutions to the same problem, selecting the best through debate, like Multi-Agent Debate). Design considerations: clear role definitions (each Agent has defined responsibilities and capability boundaries); standardized communication protocols (unified message format between Agents, avoiding understanding deviations); conflict resolution mechanisms (how to decide when Agents disagree); termination conditions (when to consider the task complete, avoiding infinite loops).

Common Question 10: What are the challenges of multi-Agent systems? How to solve them?

Answer approach: Core challenges of multi-Agent systems: communication overhead (message passing between Agents consumes massive tokens, N Agents may produce O(N²) communication); consistency (multiple Agents may give contradictory information about the same question, needing consensus mechanisms); debugging difficulty (emergent behaviors from multi-Agent interactions are hard to predict and debug); cost control (each Agent calls LLM, N Agents cost N+ times more than a single Agent). Solutions: reduce communication (use shared memory instead of direct communication, Agents read/write shared state rather than messaging each other); hierarchical coordination (introduce coordinator Agent to manage others, reducing direct inter-Agent communication); simulation runs (test multi-Agent interactions in simulated environments before real deployment); cost optimization (simple tasks use small model Agents, complex tasks use large model Agents, allocated on demand). In reality, multi-Agent systems in most scenarios aren't much better than single Agent + good tools—don't use multi-Agent just for the sake of it.

Key Takeaways

The most important advice for Agent interviews is to definitely have actual development experience. Interviewers love asking "what problems did you encounter when developing Agents, and how did you solve them." If you've only read papers and called APIs, it's hard to answer these well. I recommend building at least one Agent system from scratch, like an assistant that can search the web + summarize documents + write emails, and documenting all the pitfalls you encounter during development.

My second piece of advice is to understand Agent limitations. Interviewers dislike "Agent omnipotence" claims—if you say "Agents can solve everything," you'll likely fail. Be able to articulate what Agents are good at, what they're not, when to use Agents, and when not to. For example, simple Q&A doesn't need Agents—direct LLM is fine; complex tasks requiring multiple tool calls are where Agents shine.

My third piece of advice is to focus on Agent safety and controllability. This is an increasingly important direction for interviewers. You should be able to explain how to prevent Agents from executing dangerous operations, how to limit their operation scope, and how to intervene promptly when Agents make errors. These engineering problems are harder than algorithm problems and are key differentiators for interviewers evaluating candidates.

FAQ

Q: Which frameworks should I master for Agent development?

A: LangChain (most mainstream, most complete ecosystem), LlamaIndex (strong for RAG scenarios), AutoGen (multi-Agent), CrewAI (multi-Agent, simpler). I recommend mastering at least one and understanding others. Interviews won't test framework APIs but will test your understanding of framework design philosophies.

Q: What if I don't have Agent development experience?

A: You can use LangChain to build a simple Agent demo, like a search + summarize assistant. The key is being able to describe problems encountered during development and their solutions, like how to handle unstable tool calls or compress overly long contexts.

Q: Will Agent interviews test algorithm problems?

A: Generally not traditional algorithm problems. But they may test system design, like "design an Agent that can autonomously complete programming tasks." I recommend preparing 1-2 Agent system design cases where you can draw architecture diagrams and explain key components and design decisions.

Q: What's the difference between Agents and traditional RAG?

A: RAG is retrieval + generation, single-turn and passive. Agents are perceive-decide-execute loops, multi-turn and proactive. Agents can call RAG as one of their tools, but can also call other tools, do planning, and maintain memory. Simply put, RAG is one capability of an Agent.

Q: Are multi-Agents really useful?

A: It depends on the scenario. For complex tasks requiring multiple perspectives and specialized skills (like code review + testing + deployment), multi-Agents are indeed better than single Agents. But for most daily tasks, single Agent + good tools is enough. Don't blindly advocate multi-Agents in interviews—be able to analyze pros and cons.