Federico MeiniReal-time systems, AI agents & more
IT
Work with me(opens in a new tab)
← All posts

Designing AI agents that know when to hand over: to other agents, to rules, to humans

A practical architecture for AI agents that hand over safely: multi-agent transfers, a deterministic rules engine for high-risk calls, sandboxed tools, evals.

By 8 min readLeggi in italiano

An AI agent knows when to hand over when handover is a typed action in its loop, not something the model is expected to say in prose. The agent can transfer the conversation to another agent, defer a decision to a deterministic rules engine, or escalate to a human, and each of those is a tool call with a defined contract that the harness executes, logs and can evaluate. The model decides that it should hand over; code decides what happens next. At Turn.io I own the agentic harness customers use to run AI agents on WhatsApp, covering tool use, orchestration, guardrails and handover between agents and to humans. This post describes the architecture I’d recommend for any agent that talks to real users about things that matter.

TL;DR

  • The harness owns the loop: context, tool execution, limits, guardrails and handover. The model proposes; the harness disposes.
  • Model every handover as a tool with a schema: allowed targets, a reason, and a structured summary.
  • Keep the handover graph explicit. Each agent can only transfer to the agents you list, and transfers per conversation are capped.
  • Put high-risk decisions in a deterministic rules engine. The LLM collects inputs and explains outcomes; it doesn’t make the call.
  • Run tools in a sandbox with least privilege, limits, and outputs treated as untrusted data.
  • Layer guardrails so that the cheap deterministic checks run first and some situations bypass the model entirely.
  • Evaluate handover like a classifier: missed handovers, unnecessary ones, wrong targets and the quality of what gets passed along.

The agent loop, and who owns what

Strip away the frameworks and an agent is a loop: build the context, call the model with the available tools, execute whatever tools it asks for, append the results, repeat until it produces a reply for the user or hands over. What makes it production-grade is everything the harness enforces around that loop.

A sketch in Elixir (the LLM, Tools and Guardrails modules stand in for your own):

defmodule Agents.Loop do
  @max_steps 8

  def run(agent, conversation, step \\ 0)

  def run(_agent, conversation, @max_steps),
    do: {:handover, %{target: "human", reason: "step_limit"}, conversation}

  def run(agent, conversation, step) do
    case LLM.complete(agent.model, agent.instructions, conversation, agent.tools) do
      {:reply, text} ->
        case Guardrails.check_output(agent, conversation, text) do
          :ok -> {:reply, text, conversation}
          {:violation, reason} -> {:handover, %{target: "human", reason: reason}, conversation}
        end

      {:tool_calls, calls} ->
        case Enum.find(calls, &(&1.name == "transfer")) do
          nil ->
            results = Enum.map(calls, &Tools.execute(agent, &1))
            run(agent, Conversation.append_tool_results(conversation, calls, results), step + 1)

          transfer ->
            {:handover, transfer.input, conversation}
        end
    end
  end
end

Note what the model doesn’t control. It can’t exceed the step limit, it can’t send a reply that fails the output checks, and when it asks to transfer, the loop stops and the harness takes over. The caller then decides what a handover means: start another agent, run the rules engine, or put the conversation in a human queue.

Handover as a tool with a contract

A common failure in agent designs is handover defined only in the prompt: “if the user asks about billing, tell them you’ll transfer them”. The model then says it’s transferring, and nothing happens, or it transfers in a way nobody can measure. Make it a tool:

{
  "name": "transfer",
  "description": "Hand the conversation to another agent or to a human. Use it when the request is outside your scope, when the user asks for a person, or when you cannot proceed safely.",
  "input_schema": {
    "type": "object",
    "properties": {
      "target": { "type": "string", "enum": ["appointments_agent", "human_nurse"] },
      "reason": { "type": "string", "enum": ["out_of_scope", "user_requested_human", "unsafe_to_continue", "tool_failure"] },
      "summary": { "type": "string", "description": "What the user needs and what has been established so far." }
    },
    "required": ["target", "reason", "summary"],
    "additionalProperties": false
  }
}

The enum on target is the handover graph for this agent: it can only go where you’ve drawn an edge. A structured reason gives you something to aggregate and alert on. The summary is what the next agent or person reads first.

Handing over to other agents

Multi-agent setups earn their complexity when different parts of a service need different instructions, tools or models: an intake agent, an appointments agent with access to a booking API, an information agent with RAG over a knowledge base. The patterns I find work:

  • A router at the front, specialists behind it. The router is small and cheap, with one job: work out what the user wants and transfer. Specialists don’t need to know about each other unless you draw that edge.
  • Transfer the conversation, not just a summary. The receiving agent should see the transcript plus the structured summary. Summaries lose details, and users hate repeating themselves.
  • Cap transfers. Two agents that each think the other is responsible will ping-pong forever. After a small number of transfers in one conversation, the harness routes to a human regardless of what the model wants.
  • Make the active agent explicit state. Store which agent owns the conversation. When the next WhatsApp message arrives an hour later, it should go straight to that agent, not back through the router.

Handing over to rules: deterministic decisions for high-risk calls

For some decisions you don’t want a probabilistic answer at all. In AI triage work I’ve designed two-agent architectures with a deterministic rule engine for the high-risk decisions. The general shape: the first agent’s job is to collect structured findings from a conversational, messy exchange. The rule engine decides the outcome from those findings. The second agent explains the outcome and next steps in plain language, and it can’t change the decision.

The rules are ordinary code, versioned and reviewed by the people accountable for the outcome. An illustrative sketch (not clinical guidance):

defmodule Triage.Rules do
  @required [:age_months, :danger_signs, :symptom_days, :fever]

  def decide(findings) do
    case Enum.reject(@required, &Map.has_key?(findings, &1)) do
      [] -> classify(findings)
      missing -> {:need_more_info, missing}
    end
  end

  defp classify(%{danger_signs: [_ | _] = signs}), do: {:emergency, {:danger_signs, signs}}
  defp classify(%{age_months: age, fever: true}) when age < 3, do: {:emergency, :young_infant_fever}
  defp classify(%{symptom_days: days}) when days > 14, do: {:see_clinic, :persistent_symptoms}
  defp classify(_findings), do: {:self_care, :no_rule_matched}
end

Two properties make this work. First, {:need_more_info, missing} goes back to the intake agent as a tool result, so the rules drive which questions get asked. The model can’t skip a required question and still reach an outcome, which directly attacks errors of omission. Second, the rules can be tested exhaustively and cheaply against a dataset of clinically reviewed vignettes, without any LLM in the loop. The LLM evaluation then narrows to a much easier question: did the intake agent extract the findings correctly from the conversation?

Sandboxing tool execution

Tools are where agents touch the real world, so they need the same care as any other code that runs on behalf of a user, and more, because the caller is a model that can be manipulated. At Turn.io I connected our Lua app engine to the Agent block so agents execute tools and custom code inside a Lua sandbox; that’s how customers connect agents to EMRs and external APIs safely. Whatever runtime you use, the principles are the same:

  • No ambient authority. The tool gets only the credentials it needs, injected by the harness for that call. The model never sees secrets.
  • Hard limits. CPU time, memory, wall-clock timeout, response size, allowlisted hosts.
  • Validated inputs. Arguments are checked against the tool’s schema before execution. A failed validation is returned to the model as an error, not executed on a best-effort basis.
  • Untrusted outputs. A response from an external API is data, not instructions. Text in it that says “ignore previous instructions” is a prompt injection attempt, and your tool results should be clearly delimited as data in the context.
  • Confirm side effects. Reads can be automatic. Anything irreversible (booking, cancelling, sending) should be idempotent and, depending on the stakes, confirmed with the user or a human first.

Guardrails, in layers

Guardrails are more than a moderation call on the output. I think of them in three layers:

  1. Before the model. Deterministic checks on the input: known crisis phrases, explicit requests for a human, messages from numbers flagged as abusive. Some of these should route to a human or a fixed safety response without calling the model at all. If someone says they’re in danger, you don’t want that handled by whatever the prompt currently says.
  2. Around the model. Step limits, transfer limits, cost and latency budgets per conversation, and a defined fallback when the model provider is down or slow. On WhatsApp, a fallback that says “we’re having trouble, a person will get back to you” is far better than silence.
  3. After the model. Output checks: no diagnoses where the service isn’t allowed to give them, no personal data from other records, no promises the service can’t keep. Cheap deterministic checks first, a classifier model for what can’t be expressed as rules.

A guardrail that fires should produce a handover or a safe response, and it should be logged with a reason, just like a model-initiated transfer.

Handing over to humans

The human handover is where many agent projects are weakest, because it’s partly an operations problem. Things to decide up front:

  • What the human sees. The reason, the structured summary, the key facts already collected, and the full transcript one click away. A nurse shouldn’t have to re-ask the patient’s age.
  • What the user sees. Say that a person will reply, and set realistic expectations about when. Then make sure the agent stops replying.
  • Out-of-hours behaviour. If nobody is available until morning, say so, and have a plan for urgent cases that doesn’t depend on the queue.
  • The WhatsApp window. If the human replies more than 24 hours after the user’s last message, a free-form reply will be rejected and you need an approved template to re-open contact.
  • Handing back. After the human has resolved the issue, can the agent resume? If so, the human’s messages must be part of the agent’s context, and the handover state must be cleared explicitly.

Evaluating handover correctness

Handover is a decision, so evaluate it like a classifier. For each conversation, you want to know whether the agent should have handed over, whether it did, to whom, and when. That gives you:

  • Missed handovers: it should have escalated and didn’t. In high-stakes services, this is the number to drive down first.
  • Unnecessary handovers: it escalated when it could have handled the request. These cost staff time and erode the case for the agent.
  • Wrong target: right decision to transfer, wrong destination.
  • Timing: how many turns it took. A handover on turn nine, after the user said “I want to speak to a person” on turn two, is a failure even though it eventually happened.
  • Context quality: whether the summary contains the facts the receiver needs. This is a good job for an LLM judge with a criterion per required fact.

The best source of test cases is simulation evals: scenarios where an LLM plays the user, labelled with the expected handover behaviour. Include users who ask for a human indirectly, users who are angry but don’t need one, and users whose request drifts out of scope halfway through. Unit-test the rules engine separately, then run the same handover criteria on sampled production conversations, and turn every confirmed miss into a new scenario.

Building agents people can trust

An agent that never hands over is either doing something trivial or doing something dangerous. Getting handover right is mostly architecture: explicit contracts, deterministic paths for the decisions that matter, sandboxed tools, and evals that measure whether it all works. If you’re designing an agent for a service where mistakes have consequences, I help teams build AI agents and the evals that prove they work.

Keep reading