Understanding AI Agents - How they work
Understanding AI Agents - How they work
10 min
Alban DumouillaIn Chapter 1, you learned that AI agents are different from raw LLMs. But what exactly makes them different? And more importantly, when should you use an agent versus just asking GPT, Gemini or Claude a quick question?
In this chapter, we'll open the hood and look at how agents work. By the end, you'll understand the core building blocks of any AI agent and be able to spot good opportunities to use them in your work.
1. The three building blocks of AI agents
Think of an LLM as a new hire who knows everything on the internet but nothing about your company, your processes, or what you actually need them to do. If you just say "help me," they'll try their best, but the results will be unpredictable.
AI agents solve this by adding three critical components on top of the raw LLM:
1.1. Instructions (the rulebook)
Instructions are the agent's Constitution: a set of rules that guide every response it generates. They define:
- Role and expertise: "You are a customer support specialist for a SaaS company"
- What to do (and what not to do): "Always search the knowledge base before answering. Never make promises about shipping dates."
- How to format responses: "Use bullet points. Keep answers under 200 words. Always include a link to relevant documentation."
- When to say 'I don't know': "If you can't find the answer in the knowledge base, say 'I don't have that information' and suggest contacting the team directly."
Without clear instructions, you get inconsistent results. With good instructions, the agent behaves predictably—like a well-trained team member.
Example: The@helpagent in Dust has instructions that tell it to search Dust's documentation, provide step-by-step guidance, and include links to relevant help articles. That's why it gives consistently helpful answers about Dust, rather than generic AI advice.
1.2. Knowledge (the reference library)
Remember: LLMs don't store facts. They predict probable word sequences based on patterns in their training data. This is why they hallucinate when asked about your company's Q4 revenue or your latest product specs: they literally don't have that information.
Knowledge sources solve this by giving agents access to your company's actual data:
- Internal documents (Google Docs, Notion, Confluence)
- Code repositories (GitHub)
- Communication history (Slack, email)
- Databases and CRMs
- Any other structured or unstructured data
When you ask an agent with knowledge sources a question, here's what happens:
- You ask: "What's our remote work policy?"
- Agent searches its connected knowledge sources for relevant documents
- Agent retrieves the most relevant passages (this is called RAG - Retrieval Augmented Generation)
- Agent generates an answer based on what it found, not just on probabilistic patterns
This dramatically reduces hallucinations because the agent is grounded in real information.
1.3. Tools (the capabilities)
Tools are actions an agent can take beyond just having a conversation. Think of them as superpowers you grant to the agent.
There are two main categories:
Knowledge tools (read-only):
- Web search (for external information)
- Data visualization (creating charts and graphs)
Action tools (can change things):
- Sending emails or Slack messages
- Creating or updating tickets in support systems
- Writing to databases or CRMs
- Generating files or reports
The@deep-diveagent on Dust, for example, has access to web browsing tools AND your internal knowledge. That's why it can conduct research tasks combining your internal data with external sources (something a basic LLM can't do).
1.4. How These Three Work Together
Let's see these components in action with a real example:
Scenario: An employee asks: "Can I expense my home office chair?"
LLM without agent structure (like asking @gpt5):
- Has no access to your expense policy
- Generates a probable answer based on general patterns
- Might say "Typically yes, but check with your manager" (could be wrong for your company)
- No citations or references
AI agent with instructions + knowledge + tools:
- Instructions guide the approach: "Search expense policy documents before answering"
- Knowledge tool activates: Searches connected documents for "home office" and "furniture"
- Retrieves relevant policy: "Home office equipment up to $500 is reimbursable with manager approval"
- Generates grounded answer: "Yes! According to our expense policy, home office furniture up to $500 is reimbursable. You'll need to get manager approval first. Here's the link to the expense submission form."
- Instructions ensure format: Provides citation and next steps
See the difference? The agent isn't guessing: it's retrieving, then generating based on facts.
2. Types of Agents You'll Encounter
Not all agents are created equal. They exist on a spectrum from simple to sophisticated:
2.1. Personal Assistants
Characteristics:
- General purpose and adaptable
- Learn from your interactions over time
- Connected to broad knowledge sources
- Good for everyday tasks
Example:@dustis your personal assistant in the Dust platform. It adapts to how you work and has access to your company's connected data sources.
Best for: Daily questions, quick lookups, draft generation, summarization.
2.2. Specialized Agents
Characteristics:
- Created for your organization's specific processes
- Tailored instructions, knowledge sources, and tools
- Shared across teams
- Maintained and improved over time
Examples:
- Sales RFP generator
- Engineering documentation helper
Best for: High-frequency team workflows, standardized processes
Note: You'll learn how to build custom agents in the "Build your first agent" course. For now, just understand that they're highly specialized tools designed for your team's unique needs.
2.3. Autonomous Agents (Advanced)
Characteristics:
- Can break down complex goals into sub-tasks
- Execute multi-step workflows independently
- Use multiple tools in sequence
- Require more sophisticated monitoring
Example: An agent that: researches a topic online → summarizes findings → creates a presentation → schedules a review meeting → sends invitations
Best for: Complex, time-consuming workflows that follow predictable patterns
3. How Agents "Think" (The Agent Loop)
Let's demystify what happens when you send a message to an agent. Understanding this loop helps you write better prompts and recognize when something's going wrong.
3.1. The Basic Agent Loop
Here's what happens behind the scenes:
1. Understand the request
- Agent receives your message
- Instructions tell it how to interpret the request
- It identifies: What's being asked? What's the goal?
2. Search for relevant knowledge (if knowledge sources are connected)
- Agent queries connected data sources
- Retrieves the most relevant documents or passages
- This is the "grounding" step that prevents hallucinations
3. Decide which tools to use (if any)
- Based on instructions, the request, and the available tools, agent determines: Do I need to search the web? Query a database? Run a calculation?
- Tools are used to gather additional information or take actions
4. Generate a response
- With all the context gathered, agent uses the LLM to generate an answer
5. Verify against instructions
- Agent checks: Does this response follow my rules?
6. Deliver the response
- You see the final answer
- Often includes citations showing where information came from
3.2. Why this matters for you
Understanding the agent loop helps you:
Write better prompts:
- Be specific about what you need (helps step 1: understand)
- Mention relevant time frames or sources (helps step 2: search)
- Request specific formats (helps step 4: generate)
Recognize failures:
- No results? → Search might not have found relevant knowledge (check your question's keywords)
- Wrong format? → Instructions might need to be clearer
- Hallucination? → Knowledge sources might not contain the answer
Trust but verify:
- Check citations to see where information came from
- If there are no citations, be more skeptical—agent might be inferring rather than retrieving
4. Common misconceptions about agents
Before we wrap up, let's clear up some myths:
❌ "Agents learn from every conversation automatically"
Reality: Agents follow their instructions. They don't automatically update their knowledge or improve their instructions based on conversations. If you want an agent to "learn," someone needs to update its instructions or knowledge sources.
Why it matters: Don't expect an agent to remember your preferences unless that's explicitly built in. Provide context in each conversation.
❌ "Agents can figure out what I need"
Reality: Agents are literal. They need clear, specific requests. Vague prompts like "help me with the report" will get vague results.
Why it matters: Treat agents like efficient but literal colleagues—be explicit about what you need.
❌ "More tools and more knowledge = smarter agent"
Reality: Too many options can confuse agents. An agent with 20 tools might struggle to choose the right one. An agent connected to your entire Google Drive might retrieve irrelevant documents.
Why it matters: Focused agents with curated tools and knowledge often outperform "everything bagel" agents.
❌ "Agents are always right if they cite sources"
Reality: Agents can retrieve the wrong documents or misinterpret what they find. Citations mean "this is where I got the info," not "this is definitely correct."
Why it matters: Always verify important information, even if the agent provides sources.
Test Your Knowledge
Answer 5 questions — score 3 or more to pass
Ready to test your understanding of this chapter?