Timeouts and cancellation in long-running agents

An agent that can run for a long time needs limits on how long. Without them, one slow tool call can hold a run, and its resources, indefinitely.

Two levels of limits

• Per step. How long a single model or tool call may take.
• Per run. How long the whole task may take, no matter how many steps it has.

The step limit catches a hung call. The run limit catches an agent that keeps working but never finishes.

Setting a step timeout

In Python 3.11 and later, asyncio.timeout makes this simple.

import asyncio

async def run_tool(call):
    try:
        async with asyncio.timeout(30):
            return await call()
    except TimeoutError:
        return {"error": "Tool timed out after 30 seconds"}

Returning an error the model can read is often better than raising, because it can choose to retry or try something else.

Cancellation

Sometimes a user closes the page, or a newer request replaces an older one. The run should stop instead of continuing for nobody. In async code this arrives as a cancellation, and you should let it propagate rather than swallow it.

Clean up on the way out

Cancelled or timed-out work may leave things half done. Release locks, close connections and mark the run as cancelled in storage so it is not mistaken for one still in progress.

Cap the steps too

Also set a maximum number of steps per run. A loop where the agent repeats the same action is a common failure, and a step cap turns it into a clear error.