AI security is about what the model can reach.
A language model in a customer portal is not a text field. It is a user with access. Prompt injection, data leaking between customers, and guardrails that do not hold are the three things that turn an AI chatbot into a security matter rather than a feature, and all three follow from the same fact: the model does not distinguish your instructions from the text it reads.
What prompt injection is
A language model receives one thing: text. It has no channel for «this is an order from the owner» and a separate one for «this is data from a stranger». Everything lands in the same window, and the model does its best to follow whatever most resembles an instruction.
Prompt injection exploits exactly that. The attacker is not writing to you. He is writing to your model, through a channel you opened yourself, and he is writing something that reads as a new instruction.
It does not have to look like an attack. A sentence inside a PDF, a line in an email signature, a product description from a supplier: all of it is text that lands in the same window as your system instruction. The model reads them in order and weighs them by how much they resemble something to do, not by who wrote them.
This is not a bug that closes with an update. It follows from what a language model is. That is why the answer is never «a better instruction», and why any serious review starts by mapping what writes into the model and what the model can write out of itself.
Why a customer portal is particularly exposed
A chatbot on an open front page has little to lose. It knows nothing about who is asking, and it can do nothing beyond answering.
An assistant inside a logged-in customer portal is the opposite, and the difference is not one of degree. It sits inside a session that is already authenticated, it can look things up, and it can often do things: fetch an order, send an email, open a case. From the model’s side those are simply tools it is allowed to call.
It is authenticated. Everything the model does, it does in a logged-in context, and a successful attack inherits that login.
It takes in content from outside. Attachments, email, catalogues and integrations are channels where someone other than the customer writes the text the model reads. The attacker does not need an account to get into the window.
And it has tools. A lookup is a leak waiting to happen. A tool that sends something out of the system is an action, and an action cannot be taken back. A model without tools can say something wrong. A model with tools can do something wrong, and that is a different category.
How a model leaks data from other users
The leak almost never happens because the model remembers the previous conversation. It happens through the lookup.
If the assistant has a knowledge base to search, the question is not whether the database is secure. The question is whether the filter that decides what gets retrieved lives in the same place as the decision about what gets shown. If the boundary sits in the instruction to the model, in the form «only answer from documents belonging to this customer», it is a recommendation. If it sits in the query against the database, it is a boundary.
There are four common routes out. A lookup with no owner filter in the query itself, where the model gets hits from the whole index and picks what to show. Mixing inside the context, where several customers’ documents sit in one index, separated only by a metadata field the model has been asked to respect. Error messages and summaries, where the model explains what it was not allowed to fetch and names it in the same breath. And tools that take an identifier from the conversation, where an order number is looked up without anyone checking that it belongs to whoever is asking.
All four share one root: a boundary written as language instead of as code.
What LLM guardrails are, and what they do not solve
LLM guardrails are the rules around the model: what gets into the window, what the model is allowed to call, what is allowed to leave the system, and what is logged when something is stopped.
Note where they sit. Around the model, not inside it. A rule written into the system instruction is a request made to the very mechanism the attacker is talking to. A rule written in code is a condition that has to be true before anything happens.
What guardrails solve is which identity the tool runs as, which rows the query can reach, which domains an outgoing request may go to, what happens without a human in the loop, and how much a single session is allowed to retrieve.
What they do not solve is that the model can be talked around. It can still write something it should not. The point is that it should have nothing to write with. A defence resting on the model never being wrong is not a defence. A defence that assumes it will be wrong, and makes sure that costs nothing, is one.
That is why «we added an instruction telling it not to do that» is not an answer to the question. It is a description of what was tried first.
What red teaming a language model looks like in practice
Red teaming is not a chat with the bot to see whether it says anything odd. It is a structured pass over your setup, and it follows four steps.
The surfaces are mapped first. What writes into the model, and what can the model write out of itself? Every channel is listed with who controls it and what it exposes. A finding without a surface is an anecdote.
The vectors are run against your setup, not against a model in general. Injection through attachments, through incoming email, through a catalogue text you did not write. Attempts to make the model call a tool it should not, with an identifier it should not have had. Attempts to retrieve something from the knowledge base that was not meant for whoever is asking.
Everything that gets through is documented as it happens: what was sent, what came back, and what it would have cost in production.
And then it is repeated. A finding is not closed because something was changed. It is closed when the same attack is run again and does not work. That last step is what separates a review from a report.
Six things you can check yourself
You do not need a review to answer these. If you cannot answer them, you already know what the first step is.
1. What can the model call? Write down every tool. If the list is longer than you remembered, that is the answer.
2. Which identity does the tool run as? The customer’s rights, or the system’s? The second is the expensive variant.
3. Where does the boundary on a lookup sit? In the query against the database, or in the text asking the model to behave?
4. Which text lands in the window without a human having seen it? Attachments, email, supplier data, web pages the model fetches itself.
5. What can leave the system without a confirmation? Email, webhooks, outbound API calls.
6. Can you see it in the log afterwards? Not just what the model answered, but which tools it called and with which arguments.
Four or more unanswered questions does not mean you are exposed. It means you do not know, and that is the state a review exists to end.
The grounding
A bachelor’s degree in cybersecurity, with a bachelor thesis written for Telenor, and four speeches at Telenor: AI guardrails, red teaming, blue teaming, and how language models are broken and why it works.
Book a scoping call