Website chat has been through several generations: live chat staffed by people, rule-based bots with button menus, and now assistants powered by large language models. The technology has improved enormously, yet plenty of AI chatbots still annoy visitors more than they help.
The difference between a chatbot that frustrates and one that books meetings is rarely the model. It is the design around it. Here is what works.
Start with jobs, not with a chatbot
"We need a chatbot" is not a goal. Before choosing any technology, list the specific jobs visitors are trying to get done on your site. For a service business that might be: understand what you offer, check if you cover their area, get a rough price, and book a call. For an ecommerce store: find the right product, check stock and delivery, and track an order.
Each job becomes a capability you design and test. Anything outside those jobs should be handled gracefully, usually by pointing to a person.
Ground every answer in your own content
A model on its own knows a lot about the world and nothing reliable about your business. The standard approach is retrieval-augmented generation (RAG): your pages, documents, policies and product data are split into passages and indexed. When a visitor asks a question, the most relevant passages are retrieved and the model answers using only that material.
Good RAG depends on good content. In nearly every project we find gaps and contradictions, such as an old price on one page or two different refund policies. Cleaning those up is often the most valuable part of the work, and it improves the website for human readers too.
Connect it to real actions
The biggest leap in usefulness comes when the assistant can do things, not just talk. Modern models support tool calling: the model decides that it needs to check availability, look up an order or create a lead, and calls a function you have defined. Useful actions include:
- Booking a meeting into a real calendar.
- Creating a qualified lead in the CRM with a conversation summary.
- Checking order status or stock levels.
- Generating a ballpark quote from your pricing rules.
- Opening a support ticket with the right category and priority.
Every action should be narrowly scoped, validated on your server, and logged. The model requests the action; your code decides whether it is allowed.
Design the human handover
Visitors do not mind talking to AI. They mind being trapped by it. Make it easy to reach a person, and make the assistant proactive about offering that when a conversation is complex, sensitive or high value. Pass the full transcript and a summary along, so nobody has to repeat themselves. Outside business hours, offer a callback or email follow-up rather than a dead end.
Set guardrails that match your risk
Decide in advance what the assistant must never do. Common rules include not giving legal, medical or financial advice, not promising discounts, not discussing competitors, and not collecting sensitive personal data in chat. These rules belong in the system instructions, in server-side checks on the output, and in your test set.
Prompt injection is a real risk once an assistant can take actions. Treat everything a visitor types as untrusted input, never give the model more permissions than the task needs, and require confirmation for anything irreversible.
Get the interface details right
Small interface choices have an outsized effect on engagement:
- Open with two or three suggested questions instead of "How can I help?"
- Stream responses so the visitor sees text appearing immediately.
- Keep answers short, with links for detail.
- Make sure the widget works well on mobile and does not cover key buttons.
- Do not pop the chat open automatically on every page.
Measure outcomes, not conversations
Conversation counts are a vanity metric. Track what matters to the business: leads created, meetings booked, tickets resolved without a person, and the questions the assistant could not answer. That last list is gold. It tells you exactly what content your website is missing.
Review a sample of conversations every week for the first couple of months. You will find wrong answers to fix, new capabilities to add and recurring objections your sales team should know about.
What does it cost to run?
Model costs have fallen sharply, and for most small and mid-sized business websites the AI usage itself is modest compared with the value of a single extra lead. The real investment is in setup: content preparation, integrations, testing and ongoing review. Budget for the review time. It is what turns a decent assistant into a great one.
The short version
- Define the jobs the assistant is for.
- Ground answers in clean, current content and show sources.
- Connect it to a small set of safe, useful actions.
- Make handover to a person easy.
- Measure business outcomes and review conversations weekly.
We design and build AI assistants for websites and internal teams. If you want one that actually earns its place on your site, let's talk.
