Stop botsitting your AI agents

Sol Rashidi, an AI strategist with two decades of enterprise experience, recently fired half her AI agents. Her reason: she had become a botsitter, spending hours checking, correcting, and re-prompting agents that were supposed to save her time. She told Business Insider she went back to hiring humans for the tasks the agents kept fumbling.

If you have deployed AI agents in the last year, you already recognise the pattern. The demo was impressive. The pilot showed promise. Then someone had to sit next to the agent every day to catch its mistakes, and the productivity math stopped working. Botsitting AI agents is not a fringe complaint. It is the default outcome when teams skip the orchestration, evaluation, and human-in-the-loop design that agents actually require.

Why botsitting happens

Most agent deployments fail in the same way. A team wires an LLM to a few tools, gives it a system prompt, and pushes it into a workflow. The agent works 80% of the time. The other 20% requires a human to notice the failure, diagnose it, and clean up. That supervisory tax often exceeds the time the agent saved.

The root cause is not the model. Frontier models are capable enough for a wide range of tasks. The root cause is that teams treat agents like software features when they should be treating them like junior employees. You would not hire a junior analyst, hand them a login, and walk away. You would define scope, set review checkpoints, and measure output quality. Agents need the same scaffolding, and most deployments have none of it.

The three missing layers

Three layers are almost always absent in botsitting scenarios. First, orchestration: the logic that decides which agent runs when, what context it gets, and how failures are handled. Second, evaluation: automated checks that catch bad outputs before a human has to. Third, human-in-the-loop design: explicit decisions about which steps require human sign-off and how that sign-off is surfaced.

Skip any of these and you have built a system that requires constant supervision. That is not automation. That is a very expensive intern who never learns.

A framework for deciding when to deploy an agent

Before writing a single prompt, run the task through four filters. If the task fails any of them, keep a human in the seat or redesign the task.

1. Reversibility. Can a mistake be undone cheaply? Sending an email to a customer is hard to reverse. Drafting an email for review is easy. Agents belong on reversible or reviewable steps.
2. Verifiability. Can output correctness be checked automatically or in seconds by a human? Code that runs a test suite is verifiable. A strategic recommendation is not.
3. Frequency. Does the task occur often enough to justify the build cost? A weekly report is worth automating. A quarterly board deck is usually not.
4. Failure cost. What happens on the worst 5% of runs? If the answer includes regulatory exposure, customer churn, or silent data corruption, the agent needs heavy guardrails or should not exist.

Tasks that pass all four are strong candidates. Tasks that pass two or three need a human-in-the-loop design. Tasks that pass fewer should stay with a person.

Architecting agents that do not create work

Once a task passes the filter, the architecture matters more than the model choice. The goal is to make the agent easy to trust and easy to correct.

Narrow scope beats general capability

A single-purpose agent that handles one workflow well is worth more than a general assistant that handles ten workflows unreliably. Narrow scope shrinks the failure surface, makes evaluation tractable, and gives the team a clear mental model of what the agent does. When Rashidi described the agents she kept, they were the ones with tightly defined jobs.

Evaluation is the product

Before an agent goes into production, the team should have a test set of 50 to 200 real inputs with known good outputs. Every prompt change, model swap, or tool addition runs against that set. Anthropic and OpenAI both publish evaluation guidance that treats evals as first-class engineering artefacts. If your team cannot answer "how do we know this agent got better this week," you are botsitting.

Surface failures, do not hide them

Good agent systems make it obvious when the agent is uncertain. Confidence scores, structured escalation paths, and clear logs turn a black box into a reviewable teammate. The human review should take seconds, not minutes. If your reviewers have to re-run the agent's reasoning to check it, the interface is wrong.

Design the human handoff

Decide up front which steps require human approval, and make approval a one-click action inside the tool the human already uses. Slack, email, or the existing ticket system. Any handoff that requires context switching adds friction that erodes the time savings.

A practical checklist before you ship

Use this before any agent goes into a production workflow.

• The task passes the reversibility, verifiability, frequency, and failure cost filters.
• There is an eval set of real inputs with expected outputs, and it runs on every change.
• The agent has a single, narrow job description a new hire could understand in one sentence.
• Failure modes are logged, categorised, and reviewed weekly for the first month.
• Human approval steps are one click and live in an existing tool.
• There is a kill switch and a documented rollback plan.
• Someone owns the agent. Not a committee. One name.

If you cannot tick every box, you are building a botsitting problem.

Where this fits in a real engineering org

Most mid-stage companies do not have the internal bench to design this scaffolding while also shipping the roadmap. That is where external senior capacity earns its keep. Devspace's AI and Data practice runs use case mapping and prototyping specifically to separate the tasks worth automating from the ones that look automatable in a demo but fail the filter. For companies without a full-time head of AI, our Fractional CTO engagements often include agent portfolio reviews: which agents to keep, which to fire, which to redesign.

On the investor side, agent sprawl has become a recurring finding in Technical Due Diligence. Portfolio companies buy or build a dozen agents, none of which have evals, and the operational risk sits invisible until someone counts the hours humans spend supervising them.

The uncomfortable conclusion

Agents are not free. Every agent you deploy is a system that needs owners, evals, and a maintenance budget. Rashidi's decision to fire half her agents was not a defeat for AI. It was the correct portfolio pruning move that every team should be making quarterly. The question is not how many agents you have running. It is how many of them would survive an honest audit against the checklist above.

If the answer is fewer than half, stop botsitting. Fire the rest and rebuild the ones worth keeping properly.