I was working with a fitness and nutrition coach to set up a client support chatbot. She worked with dozens of clients, each with a personalized program — nutrition plans, training schedules, progress tracking. She wanted to give clients 24/7 access to answers about their programs without having to personally respond to every question.
The setup made sense. She loaded all her client programs into the knowledge base. Configured the bot with her tone and approach. Set it up to answer questions about nutrition, workouts, supplements, and how to follow the program.
Before she went live, I asked her one question.
“What do you think happens if a client asks the bot, ‘what's Emma's program?'”
She paused. “It wouldn't answer that… would it?”
I ran the test.
The bot answered in full detail. Emma's calorie targets. Her training split. Her supplement protocol. Everything another client should never see, delivered helpfully and clearly by a bot that was trying to do exactly what it was built to do.
The Bot Isn't Being Malicious. That's the Point.
This is the thing that makes this problem easy to miss and potentially serious: the bot isn't doing anything wrong by its own logic. It was given information. Someone asked a question it could answer. It answered.
It doesn't know the rules of your business. It doesn't know that client data is confidential. It doesn't know that one client asking about another client's program is a privacy violation. It doesn't know that in a medical or fitness context, sharing another person's health information could be a legal issue, not just an awkward situation.
The bot knows what you've told it. If you haven't told it that client information is private — specifically, that it should only ever discuss the person who is asking — it will answer with everything it has.
And it has everything you gave it.
The Blind Spot in Most AI Bot Deployments
When people build AI support bots, they focus on the use cases. What questions should the bot answer? What documents should it have access to? What tone should it use? How should it handle questions it can't answer?
These are the right questions. But they're incomplete.
The missing question is: what should this bot refuse to do?
Most bot deployments don't have explicit guardrails around sensitive information. They have good intentions and a knowledge base. That's not the same thing.
Consider what's common in AI support bot setups:
- A coaching or consulting business loads all client notes and programs into the knowledge base for easy access
- A medical practice uploads patient intake forms and treatment protocols
- A financial firm stores client portfolio summaries for advisor reference
- A legal practice indexes all case files and client agreements
In each case, the intention is for the bot to help the right people access the right information. But without guardrails, “the right people” is anyone who asks.
The Fix: One Instruction in the System Prompt
The good news is that the fix is simple. It's a single instruction added to the bot's system prompt — the foundational set of rules the bot operates by.
For the fitness coach's bot, the instruction was: “Only answer questions about the person who is currently asking. Never share information about other clients. If someone asks about another person's program, schedule, or any details not about themselves, tell them you can only discuss their own account.”
One sentence. That's all it takes to close a gap that could have caused a significant privacy incident.
But the instruction has to be explicit. The bot will not infer that client data is private from the nature of the content. It won't look at a spreadsheet full of client names and realize that each row should only be visible to the person it belongs to. You have to say it directly.
Building Guardrails as a Feature, Not an Afterthought
The broader principle here is that guardrails are something you design intentionally — not something that gets added after something goes wrong.
When I work with businesses on deploying AI agents, I always walk through what I call the refusal list before anything goes live. What should this bot refuse to do, regardless of what someone asks? The list typically includes:
Information scope: Only discuss topics and data relevant to the person asking. Never reference information about other users, clients, or accounts.
Sensitive categories: Never discuss pricing you're not authorized to disclose, internal business information, employee data, or anything the business hasn't explicitly approved the bot to share.
Escalation triggers: When a request involves something sensitive, uncertain, or outside the bot's scope, route to a human rather than attempting an answer.
Identity verification: If the bot has access to account-specific information, define how it should handle requests that seem misrouted or don't match the expected user.
This isn't a long list. But it's a list most people skip, because they're focused on getting the bot to work — not on the edge cases where it works in the wrong direction.
The Bot Is Trying to Help. That's the Problem.
When people hear about AI safety concerns, they often picture dramatic scenarios: bots going rogue, systems making catastrophic decisions, AI doing something clearly harmful.
The real risk in most business AI deployments is more mundane and more immediate. It's a bot that's working exactly as designed — helpful, responsive, thorough — applied to a situation its designers didn't think through.
The fitness bot wasn't failing. It was succeeding at a goal (answer client questions) in a context that nobody had scoped (don't answer questions about other clients).
Every AI bot you deploy needs two things: a clear job, and a clear set of limits. The job is what most people think about. The limits are what most people skip.
Before your next bot goes live: run the Emma test. Ask it a question it shouldn't be able to answer. See what happens.
Then add the guardrail before anyone else does.
The 4-Day AI Sprint covers how to build AI agent workflows — including system prompt design, guardrails, and how to scope what your agents should and shouldn't do.
