The flow had a node for everything. That was the problem.
A receptionist agent was split into 12 nodes, one for each department and request. It passed every test, then went live. Replies took over 3 seconds and the agent started making things up. The fix was fewer nodes, not more.
- Nodes in the flow, before and after the fix
- 12 → 4
- Latency on live calls before the fix
- 3 s+
Stack: Retell AI · Langfuse · Tuner
The problem
A client wanted their AI agent to take over from their receptionist. That’s very doable. The agent worked perfectly in the demo and passed every test case.
The business had several departments, each with its own products and questions. So the agent’s flow in Retell was built like a phone tree:
- a first node that works out what the caller wants
- then a separate node for each department and request, with its own follow-up questions
It ended up with 12 nodes.
Then they went live and started sending real calls to the agent. Replies slowed to more than 3 seconds, and the agent started making things up.
Why more nodes made it worse
Splitting a flow into many nodes feels tidy. Each node does one job, so each should be easier for the model. Up to a point, that’s true.
But every node carried the global prompt and the persona on top of its own instructions. With a dozen nodes, the agent had a lot to carry, and every move between nodes was one more decision it could get wrong.
The tests didn’t show it. Test calls tend to follow one clean path through the flow. Real callers ask two things at once, change their minds and jump between topics.
The fix
We traced the live calls in Langfuse and Tuner. They pointed at the flow itself, not at any one prompt. So the fix was simple:
- Fewer nodes. We merged the department nodes into broader ones and cut the flow from 12 nodes to 4.
- More general prompts. Each remaining node handles a group of related requests, not one narrow case.
Replies sped back up, and the agent stopped making things up.
What agencies can take from this
- Start with as few nodes as you can. Add one only when a step really needs its own rules or tools. Every node adds weight, and every hand-off is a chance to get lost.
- A clean test isn’t a live call. Test with messy calls too: callers who ask two things at once, change their minds or start with small talk.
- Watch latency on live calls. It can look fine in testing and still climb once real callers arrive.