AI & Automation
AI with business data: API, RAG and privacy explained
AI with business data means that a language model answers on the basis of your own documents and data, which your software looks up for each question and sends along via an API. The model itself does not know your business: it was trained on general texts and knows nothing about your customers, prices or procedures. The technique of retrieving first and then letting the model answer is called RAG, retrieval-augmented generation.
By Dries Nijs, Organic Web Specialist
Last updated: 19 November 2026
At a glance
How does AI work with business data? The short answer
A useful answer about your business requires two kinds of knowledge. Model knowledge is what the language model learned during its training: language, general facts, reasoning. Business knowledge is what exists only in your systems. That second kind is not in the model, and does not need to be.
Instead of retraining the model, you give it the right information at the right moment. Your software looks up the passages relevant to the question, sends them along and asks the model to rely on those alone. Your documents stay where they are; only what is needed for that one question goes to the model.
As a result, three things become more important than the model itself: the quality of your documents, the access rights during retrieval and the agreements with the provider about what happens to the data you send.

Comparison
Model knowledge and business knowledge: what is the difference?
A language model sounds as if it knows everything. Yet what it says comes from two clearly different sources.
| Characteristic | Model knowledge | Business knowledge |
|---|---|---|
| Origin | The texts the model was trained on | Your own documents, databases and systems |
| Content | Language, general knowledge, reasoning | Products, prices, customers, procedures, arrangements |
| Timeliness | Goes up to the point when training stopped | As current as your own sources |
| Updating | Only with a new version of the model | You update the document and the answer changes with it |
| Pointing to the source | Not possible | The answer can refer to the source document |
| Access | The same for everyone who uses the model | Depends on who is asking |
If a model gets a question about your business without having been given any business knowledge, it fills the gap with something that sounds plausible. Such a made-up answer is called a hallucination. The solution is not a smarter model, but the right information alongside the question.
Integration
The API: how your software talks to the model
Your software talks to a language model via an API: it sends a request with an instruction and the necessary text, and receives a response. Everything the model may use for that response is contained in that single request.
Two properties determine how you handle business data. The model itself remembers nothing between two calls: whatever you sent last time is gone. And there is a limit to the amount of text per call, so you cannot send your entire archive. Besides, more text does not automatically make the answer better.
This leads to the key question of this topic: which pieces of information do you send with which question? The answer is retrieval: looking up the right passages. How such an integration fits into existing software is explained in AI integrations: connecting AI to existing systems.
Technology
How does RAG (retrieval-augmented generation) work?
RAG stands for retrieval-augmented generation: first look things up (retrieval), then let the model answer (generation) with the retrieved text as a supplement (augmented). In six steps:
01
Preparing documents
Your sources are split into smaller chunks, such as a paragraph or a chapter, so that they can later be searched in a targeted way.
02
Index
An embedding is calculated for each chunk: a series of numbers that represents the meaning of the text. These series are stored in an index, often a vector database.
03
Retrieving
A question is converted in the same way and compared with the index. The chunks whose meaning is closest to the question come out on top, even when they use different words. This is often combined with keyword search.
04
Augmenting
The retrieved chunks are sent to the model along with the question and an instruction: answer only on the basis of this text, and say so when the answer is not in it.
05
Answering
The model formulates an answer from the passages supplied. It does not look anything up itself: it reads what the retrieval step supplied.
06
Referring to the source
The answer states which document it comes from, so a member of staff can check it.
Sources
Which documents and data are suitable?
An answer from RAG is never better than the source it comes from. Whatever is wrong, outdated or contradictory in your documents ends up in the answer in the same way.
Well suited
- manuals, procedures and work instructions
- product information and technical data sheets
- frequently asked questions with a verified answer
- terms and policy documents in their current version
- documents with a clear title, date and owner
Sort out first
- old versions that remain alongside the current one
- documents that contradict each other
- scans and images without readable text
- knowledge that exists only in people’s heads or mailboxes
- files that nobody knows are still accurate
Structured data, such as orders, stock or customer records, is better looked up directly in the database than via text passages. A good solution often combines both: passages from documents for explanations, a targeted query for figures and statuses. If the model is allowed to choose and run such queries itself, you are talking about an agent: see AI agents for businesses: 7 processes you can automate.
Agreements
Access rights, privacy and security
As soon as a model works with business data, it is no longer just about the quality of the answer, but also about who gets to see what and where the data ends up.
- Permissions during retrieval The retrieval step only returns documents that the person asking is allowed to view. That check takes place in your software, before anything goes to the model, and not in the model.
- Send as little as possible Only the passages the question requires are sent. Anything not needed, such as names or contact details, is left out or anonymised.
- What leaves your systems Map out which data goes to the model provider, and whether it includes personal data or confidential information.
- Where it is processed Check in which region the data is processed and how long the provider retains it.
- Use for training Check the provider’s terms to see whether the data you send is used to train models, and whether you can opt out.
- Keys and index The API key stays on the server. The index also contains your business information and deserves the same security as the documents themselves.
If the documents contain personal data, the GDPR is the framework within which you weigh all of this up. This is not legal advice: have your specific situation reviewed by someone with knowledge of privacy law.
Limitations
Why you should not simply put your business data into a model
Pasting into a public chat service
Anyone who pastes a contract or a customer list into a public chat service sends that data to an external party without your business having made any agreements about it. So set out internally which data may go into which tools, and provide an alternative that does fall under your own agreements.
Training a model on your own data
Further training a model, also called fine-tuning, mainly adjusts its style or behaviour. It is less suitable as a store of knowledge: you cannot remove or update a specific fact in it, the model cannot point to where an answer comes from, and it does not distinguish between who is asking. For knowledge that changes or is not meant for everyone, retrieving it at the moment of the question is the more logical route.
Human oversight remains necessary
Even with the right passages, a model can misread something or combine two sources incorrectly. Have a person check, approve or correct answers wherever a mistake has consequences, and always show the source. How I approach automation and AI integrations for businesses is explained on the page AI automation.
FAQ
Frequently asked questions about AI with business data
What is RAG in simple terms?
RAG is an open-book exam for a language model. Instead of answering from memory, the model receives the relevant passages from your own documents with each question and formulates an answer from them. The abbreviation stands for retrieval-augmented generation.
Do I need to train a model on my own data?
For knowledge, usually not. Retrieving at the moment of the question keeps your information current, respects access rights and lets you show the source. Further training is better suited to a consistent style or a very specific task.
Does a language model know my business?
At most, whatever was publicly available about it when the model was trained, and even that is incomplete and out of date. It does not know your internal documents, prices, customers or arrangements, unless you send them along with the question.
Can AI with business data still make mistakes?
Yes. RAG reduces the chance that the model makes something up, because the answer is based on supplied text, but it does not rule out errors. The retrieval step can find the wrong passage, or the model can misread a passage. A source reference and a person who checks where it matters remain necessary.
Is RAG safe for confidential documents?
That depends on the set-up. The deciding factors are the access rights during retrieval, where the index and the documents are stored, and the model provider’s terms. For highly sensitive data, a model on your own infrastructure may be an option.
Do I need a vector database?
Not always. For a limited set of documents, an ordinary search function or an extension to your existing database may be enough. A separate vector database becomes worthwhile with large volumes of text or when searching by meaning gives clearly better results.
Is AI with business data feasible for an SME?
The technique is not reserved for large companies. What determines the work is the state of your documents, the number of sources and the access requirements. A well-defined start, such as one set of manuals for one team, keeps it manageable. Bespoke work is priced on request.
Further reading
The related service
AI automation as a service
Would you like a language model to work with your own documents or systems? The service page explains how I approach automation and AI integrations, with attention to data, permissions and oversight.

About the author
Dries Nijs, Organic Web Specialist
I am Dries Nijs, a developer from Maasmechelen. I have been building for the web since the 1990s: web applications in React, Next.js and Node.js, and websites in WordPress. In my own platforms, such as BM Jumpfun, Pelletkachelzorg and SEO Websites, I use AI via APIs. MadeByDries is the personal brand under which I work.
Want AI to work with your own data?
Tell me which questions the system needs to answer and where the information is stored today. Together we will then look at what is feasible and responsible.