Retrieval Augmented Generation (RAG), undefined

Agentic Retrieval: Giving the Model Tools

In the tool-calling approach, nothing is retrieved before the model runs. Instead the model is handed a small set of functions over your document store and decides for itself what to look at — searching, reading, noticing a reference, and following it, exactly the way a research assistant would work through a filing cabinet.

The minimal tool set covers four verbs, and almost every agentic RAG system is some variation of them:

list_documents(prefix, filters, limit) — browse the corpus structure. Returns titles, IDs, dates, and sizes so the model can orient itself before reading anything.
search_documents(query, filters, limit) — find candidates by keyword, full-text (BM25), or metadata match. Returns snippets with document IDs, not whole files.
read_document(doc_id, offset, limit) — read actual content, paginated. This is the tool people get wrong: without offset/limit, a single 300-page PDF blows the context window and ends the conversation.
get_metadata(doc_id) — authorship, dates, source, version, related documents. Cheap, and often enough to rule a document in or out without reading it.

Structure is what makes this work. The model has no embeddings to lean on, so the organisation of the corpus is the retrieval quality. A manifest or table-of-contents file it reads first, descriptive file names, a short README per folder, consistent front matter (title, date, summary, tags) at the top of each document, stable IDs, and section anchors so reads can be targeted rather than sequential — these do the work that an embedding index would otherwise do.

The loop: the model emits a tool call, your code executes it against blob storage and the catalog, the result comes back as a tool result, and the model either calls another tool or answers. Two to five iterations is typical; a good system prompt telling the model what the corpus contains and how it is organised cuts that number substantially.

Advantages:

No ingestion pipeline: no chunking, no embedding costs, no index to rebuild. Point the tools at the store and it works.
Always fresh: the model reads the file as it exists right now. There is no staleness window between a document changing and answers changing.
Exact lookups are exact: "the Q3 invoice from Acme Corp" or "error code E-4471" resolves by name or by literal match — precisely the queries where vector similarity is weakest.
Multi-hop reasoning: the model can read a document, notice it cites another, and go fetch that one. A single vector search cannot follow a reference.
Permissions at read time: authorization is checked when the tool executes, using the caller's identity — not baked into an index that was built with someone else's access.
Whole-document context: the model reads a section in place, with its surrounding headings and caveats intact, instead of an isolated chunk stripped of its context.
Auditable: the transcript shows exactly which files were opened and in what order. Debugging a bad answer means reading the tool calls.

Disadvantages:

Latency and token cost: each tool call is a full round trip to the model. Three iterations of search-then-read can mean several seconds and tens of thousands of input tokens.
Depends on model reasoning: a weaker model searches badly, gives up early, or reads the wrong file. The retrieval quality is only as good as the planner.
Needs a good search tool or a bounded corpus: browsing works for hundreds or thousands of well-organised documents. Across ten million files with no search index, the model is lost.
Non-deterministic: the same question can take different paths on different runs, which complicates evaluation and caching.

Best Use Cases: codebases and repositories, structured document libraries with clear naming, legal and financial records where the exact document matters more than a semantically similar one, corpora that change constantly, and any system where per-user permissions must be enforced at query time.

Implementation Example:

# Tool definitions handed to the model alongside the user's question.
TOOLS = [
  {
    "name": "list_documents",
    "description": "Browse the document store. Use this first to see what exists.",
    "input_schema": {
      "type": "object",
      "properties": {
        "prefix": {"type": "string", "description": "Path prefix, e.g. 'contracts/2025/'"},
        "limit":  {"type": "integer", "default": 50},
      },
    },
  },
  {
    "name": "search_documents",
    "description": "Full-text search. Returns matching snippets with doc_ids, not full files.",
    "input_schema": {
      "type": "object",
      "properties": {
        "query":  {"type": "string"},
        "source": {"type": "string", "description": "Optional metadata filter"},
      },
      "required": ["query"],
    },
  },
  {
    "name": "read_document",
    "description": "Read a document's text. ALWAYS paginated - never returns a whole book.",
    "input_schema": {
      "type": "object",
      "properties": {
        "doc_id": {"type": "string"},
        "offset": {"type": "integer", "default": 0},
        "limit":  {"type": "integer", "default": 8000, "description": "Characters"},
      },
      "required": ["doc_id"],
    },
  },
]

MAX_CHARS = 8000  # hard ceiling: one tool result must never fill the context window

def read_document(doc_id, offset=0, limit=MAX_CHARS, user=None):
    row = catalog.get(doc_id)
    authorize(user, row["acl"])              # permissions enforced at read time
    text = s3_get_text(row["text_key"])      # pre-extracted markdown, not the raw PDF
    window = text[offset : offset + min(limit, MAX_CHARS)]
    return {
        "doc_id": doc_id,
        "title": row["title"],
        "offset": offset,
        "returned_chars": len(window),
        "total_chars": len(text),
        "has_more": offset + len(window) < len(text),   # tells the model to page on
        "content": window,
    }

# The agent loop: the model decides what to fetch, your code executes it.
def answer(question, user):
    messages = [{"role": "user", "content": question}]
    for _ in range(6):                        # bound the loop
        resp = llm.create(messages=messages, tools=TOOLS, system=CORPUS_GUIDE)
        if resp.stop_reason != "tool_use":
            return resp                       # model has enough to answer
        results = [dispatch(call, user) for call in resp.tool_calls]
        messages += [resp.as_message(), {"role": "user", "content": results}]
    return summarize_partial(messages)

MCP (Model Context Protocol) is the emerging standard for exposing exactly this kind of tool surface. Rather than hand-wiring tools into one application, you run a server that publishes list/search/read over your document store, and any MCP-compatible client can use it. See the dedicated section at the end of this article.