FastAPI for agent backends

An agent run can take much longer than a normal web request. A good API design does not make the client wait on one open connection.

Start the run, then check on it

The common pattern has two endpoints. One starts a run and returns an id immediately. The other returns the status and result for that id.

from fastapi import FastAPI, BackgroundTasks
from pydantic import BaseModel

app = FastAPI()

class StartRequest(BaseModel):
    question: str

@app.post("/runs", status_code=202)
async def start_run(req: StartRequest, tasks: BackgroundTasks):
    run_id = await store.create_run(req.question)
    tasks.add_task(execute_run, run_id)
    return {"run_id": run_id}

@app.get("/runs/{run_id}")
async def get_run(run_id: str):
    return await store.get_run(run_id)

The 202 status means the request was accepted and work is in progress.

Do not rely on in-process background tasks for important work

BackgroundTasks runs inside the web process, so a restart loses the work. For anything that must survive, put the job on a queue and let separate workers run it. The API only creates the run record and enqueues the job.

Stream progress

Clients often want to see steps as they happen. Server-sent events are a simple way to push updates over a single HTTP connection, and FastAPI supports streaming responses for this.

Keep the models strict

Use Pydantic models for requests and responses. You get validation, clear error messages and automatic API documentation.

Plan for cancellation

Add an endpoint to cancel a run, and make sure the worker checks for it between steps.