An AI chatbot is software that takes a person’s message and produces a conversational response, using rules, a language model, or a combination of methods. Some chatbots also retrieve information from approved sources or use connected tools. The label alone does not tell you what information a chatbot can access, what it can do, or how dependable its answers are. To understand one, look at how it chooses a response, where its information comes from, and what happens when it cannot answer.
How rule-based chat works
A rule-based chatbot follows paths written in advance. A business might define a welcome message, list common questions, and set up buttons such as “Shipping,” “Returns,” and “Contact us.” Selecting a button takes the visitor to the response or next question assigned to that option.
This approach is useful when the choices are predictable and the business wants to control the wording and route. It can make a process clear, such as asking a visitor to choose a service before showing the right contact details.
The tradeoff is that the visitor’s wording and choices need to fit the paths the owner has created. A person who asks a question in an unexpected way may not reach the intended answer. More branches can cover more cases, but they also require someone to design and maintain them.
How language-model chat works
A language-model chatbot uses a model to interpret a message and generate text in response. Rather than selecting only from a fixed menu, it can respond to different phrasings and carry context from earlier messages in a conversation.
The model’s reply is not automatically a verified statement of business policy. Its wording can sound assured even when the available information is incomplete or the question is unclear. The model’s general language ability is not the same as access to your latest product details, account records, or internal rules.
Instructions can set a chatbot’s role and boundaries, but they are not a replacement for providing relevant, current information. When a request is outside the defined scope, a useful design makes that boundary clear and offers an appropriate next step, such as asking a clarifying question or directing the visitor to a person.
What retrieval adds
Retrieval is a way to find relevant material from a set of sources and provide it to the language model as context for a response. For example, a chatbot might retrieve a passage from a returns policy when someone asks how long they have to return an item. The model then uses that passage to compose an answer.
Retrieval can make a chatbot’s answers more specific to a business than relying on general model knowledge alone. It does not mean the model has been retrained on the material, nor does it guarantee that every answer will be correct. The result still depends on whether the source is relevant, current, available to the chatbot, and understood in the context of the question.
For a website AI Agent, “attached” and “ready to answer from” are not necessarily the same state: A source must finish processing before its content can be used in answers. Check source readiness before treating a missing answer as evidence that the information was not supplied.
Retrieval works differently for prose and structured records. A policy document is useful for explaining a rule. A spreadsheet or database may be used for questions that require filtering or calculating over rows. Supported indexed spreadsheet and database datasets can be filtered, sorted, counted, and aggregated over their complete saved rows. An exact total should not be assumed when the dataset is incomplete; the result depends on the available data and the supported operation.
How chatbots, retrieval, and tools fit together
These approaches are not mutually exclusive. A chatbot can use a language model to understand the message, retrieve relevant text, and follow a set workflow for the next step. Depending on the product and configuration, it may also use tools to perform an action or obtain information from another system. A chatbot is not categorically unable to use tools; the useful question is which tools are connected, what they are allowed to do, and what checks or approvals apply.
Keep three capabilities separate when assessing a system:
- Answering: producing a conversational reply, perhaps from general model knowledge or supplied context.
- Looking up information: retrieving material from a source or querying a connected dataset.
- Taking an action: using a tool or workflow to change something or initiate a task.
A chatbot that can explain a return policy has not necessarily looked up a particular order. One that can look up an order has not necessarily been given permission to cancel it. Ask what the system can actually access and do, rather than inferring capabilities from the word “AI.” For a broader comparison of labels and capabilities, see AI Agent vs chatbot.
A practical worksheet for evaluating a chatbot
Use a small set of questions tied to one business task. The goal is not to prove that a chatbot is always right. It is to learn which cases it handles, which need a clearer source or workflow, and where a person should take over.
Imagine a small shop evaluating answers about its returns policy. The example below is illustrative. Use your own approved policy and write down the expected response before testing.
| Test case | Example question | Expected behavior | What to check |
|---|---|---|---|
| Covered question | “How many days do I have to return an item?” | State the policy’s return period and any important conditions. | Does the answer match the approved policy? |
| Different wording | “Can I send this back next month?” | Apply the same policy to a differently phrased question. | Does it understand the intent without changing the rule? |
| Missing detail | “Will you accept this after I used it?” | Answer only if the policy covers used items; otherwise explain what is unclear. | Does it avoid inventing a condition? |
| Unsupported request | “Where is my refund?” | Explain what information is needed or provide the correct next step. | Does it avoid claiming to have checked an individual order if it has not? |
For each test, record the question, the expected behavior, the response, and a result: matches, partly matches, or does not match. Add a short note on the next fix. It might be updating a source, making a policy clearer, changing a workflow, or defining a human-support route. These labels are a review method, not a performance guarantee.
Try more than one natural phrasing for important questions. Include a question the source answers, one that falls outside its scope, and one that requires a calculation or lookup if those tasks matter to your business. Check whether the result is supported by the relevant source or operation, not just whether it reads smoothly. For more on testing the boundary between available and missing information, use this test guide.
If your chatbot answers from files or other knowledge sources, confirm that each source has finished processing and that the answer matches the original material. For structured data, verify the records and units that should be included before trusting a count or total. A numeric result involving mixed currencies or units needs special care: A numeric aggregate rejects mixed currencies or units and asks the owner to filter or group by that unit before calculating a total. For further background on knowledge sources, see the knowledge base feature guide.
What a test can and cannot tell you
A short test set helps you inspect specific cases. It does not prove how the chatbot will answer every possible question. Customer wording changes, sources can become outdated, and a new workflow can affect the conversation. Keep the expected answer and the source you used with each test so you can review them again after making a relevant change.
The practical definition is simple: an AI chatbot is a conversational interface whose behavior depends on its response method, available information, and configured capabilities. Understanding those parts gives you a more useful evaluation than asking whether it is “smart.”
Follow the information through one answer
For a public return-policy question, the sequence is: visitor question, relevant approved source, answer generation, then a reply or a request for help. For an order cancellation, add a separate authorization check and connected action before any success confirmation. A language model's fluent answer is not evidence that the connected action ran.
| Layer | Example input | Observable output |
|---|---|---|
| Conversation | “Can I return this?” | The visitor's actual request |
| Retrieval | Approved return policy | Relevant policy text |
| Generation | Request plus policy | Explanation with the correct conditions |
| Optional action | Verified customer and eligible order | Confirmed operation or explicit failure |
The Agent versus chatbot guide expands this distinction. Use the business buyer guide to select a product, then the no-code tutorial for a small first configuration.