← All posts
#llm-evals
Testing a medical triage agent with LangWatch Scenario and pytest
How I test a medical triage AI agent with LangWatch Scenario: a simulated user, judge criteria, scripted and free-running conversations, in pytest and CI.
Designing AI agents that know when to hand over: to other agents, to rules, to humans
A practical architecture for AI agents that hand over safely: multi-agent transfers, a deterministic rules engine for high-risk calls, sandboxed tools, evals.
Simulation evals: letting an LLM play the user to test your chatbot before real users do
How to build simulation evals for LLM chatbots: persona-driven simulated users, LLM-as-judge rubrics, error rates for high-stakes flows and a feedback loop.