How to scope an AI chatbot or agent project before you build it

AI chatbot development has never been easier to start. A working demo over your documents can be put together in an afternoon. That is exactly why so many projects stall between demo and production: the demo answers the easy questions, and nobody wrote down which hard questions it was supposed to handle, which systems it needed, or what “good enough to launch” meant.

Scoping is the cheapest place to fix this. Below are the six questions we work through with every client before an AI assistant or agent is built, and the checklist we use to close the phase.

Chatbot or agent: be clear which one you are building

The words are used loosely, so it helps to be precise. A chatbot answers: it reads a question, finds information and replies. An AI agent acts: it calls tools on your behalf, such as checking an order, booking a slot, raising a refund or updating a CRM record, and it may chain several steps to finish a task.

The difference matters for scope. As soon as the assistant can change something in a real system, you need to decide what it is allowed to do, how each action is logged, which actions need a person to confirm, and how mistakes are reversed. Many good projects start as a chatbot and add actions one at a time once the answers are trusted.

1. Name one job, with edges

“Answer customer questions” is a capability, not a scope. A scope sounds like this: “Resolve order-status and return-eligibility questions on WhatsApp, in English and Hindi, for customers who are logged in, and hand everything else to the support team with the conversation attached.”

That sentence tells you the channel, the languages, the user, the data needed, and where the assistant stops. Write one for your project. If you cannot, the project is not ready to build yet, and that is a useful thing to learn early.

2. Map the conversations you already have

You almost certainly have a record of the questions people ask: support tickets, chat logs, call notes, sales emails or WhatsApp threads. Pull a few hundred recent examples and sort them into intents. For each intent, mark one of three things:

• Answerable from documents. Policies, product information, FAQs, manuals. These are good candidates for retrieval (RAG).
• Needs a system lookup or action. Order status, account balance, appointment availability. These need integrations and, for actions, permissions.
• Needs judgement. Complaints, exceptions, negotiations, anything with legal or medical weight. These should usually go to a person, with the assistant gathering context first.

This sort becomes your scope. Pick the intents that are frequent, low-risk and well documented for the first release. Everything else gets a clean handoff rather than a guess.

3. Inventory the knowledge and systems involved

An assistant is only as good as what it can reach. List every source it needs and ask the same few questions of each.

4. Define “good” before you build

The single most useful scoping output is an evaluation set: a list of realistic questions or tasks with the expected outcome for each. Take them from the conversations you mapped, and include the awkward ones: ambiguous questions, questions the assistant must refuse, questions in mixed languages, and attempts to make it say something it should not.

Then agree what you will measure and the threshold for launch. Typical measures are:

• Correct answers on in-scope questions.
• Correct handoff on out-of-scope or sensitive questions.
• No invented facts, prices or policies.
• Response time on the target channel.
• Running cost per conversation at expected volume.

The thresholds should be signed off by whoever owns the outcome, such as the head of support or operations, not just by the engineering team. Rerun the same set whenever prompts, models or documents change, so you know whether a change made things better or worse.

5. Design handoff and guardrails early

Decide in scoping, not after launch, when the assistant hands over to a person, what context travels with the handoff, which topics are off-limits, and how personal data is handled. For agents, add a list of every action, who is allowed to trigger it, whether it needs confirmation, and how it is undone.

6. Choose the build approach last

Only now is it worth choosing models and architecture. For most business assistants the starting point is retrieval over your own documents plus tool calls into your systems. Fine-tuning a model is rarely the first step; it helps with tone and format more than with knowledge, and it adds a training pipeline you then have to maintain. Keep the model choice swappable, because the best price and quality option changes often.

What drives the cost of an AI chatbot or agent

There are two costs to plan for: building it, and running it. The build cost is driven mostly by scope, so the scoping work above is also your estimate.

• Number of intents in the first release, and how many need system lookups rather than documents.
• Integrations: each system the assistant reads from or writes to needs connection, error handling and tests.
• Channels: a website widget is simpler than WhatsApp or voice, which add their own approvals, formats and latency limits.
• Languages: each language needs its own evaluation questions, and quality varies between models.
• Evaluation depth: a larger, harder evaluation set costs more to build and gives you more confidence.
• Running cost: model usage per conversation, hosting, monitoring, and the time to keep documents and prompts current.

A sensible way to phase the work

1. Discovery: the six steps above, ending in a written scope, evaluation set and estimate.
2. Pilot: build the first release and test it with internal users against the evaluation set.
3. Limited launch: route a share of real traffic to the assistant, with close monitoring and easy handoff.
4. Expand: add intents, channels or actions one at a time, each with its own evaluation questions.

If you would like help running discovery, our AI and automation team does this as a fixed, short engagement. If your goal is to build in-house, our AI training and consulting work helps your team run the same process themselves.

Scoping checklist

• One-sentence job description with channel, language, user and stopping point.
• Intents sorted into documents, system lookups and judgement.
• Data and system inventory with owners and access confirmed.
• Evaluation set with agreed launch thresholds.
• Handoff rules, off-limits topics and personal-data handling written down.
• For agents: every action listed with permission, confirmation and undo.
• Estimate for build and for monthly running cost.