The model may not say 'I don't know'
A person who does not know an answer usually says so, or hedges, or goes to look it up. A language model can do something else. Faced with a gap, it may produce the most statistically plausible continuation, which reads exactly like a real answer because that is what it was trained to imitate.
So the failure mode of an ungrounded model is not always silence. It can be confident fabrication. It may name a regulation that does not exist, cite a clause it invented, or assert a requirement it has no basis for, all in the same measured tone it uses when it is right. The output is fluent. The content can be fiction. And without citations, retrieval evidence, or review, there may be nothing in the answer itself that tells you which is which.
NIST has a name for it: confabulation
This is not a fringe concern. NIST, the U.S. standards body, names it directly in its AI Risk Management Framework Generative AI Profile. It defines confabulation as 'the production of confidently stated but erroneous or false content, known colloquially as hallucinations or fabrications, by which users may be misled or deceived.'
The framing matters. NIST lists confabulation among the risks organizations must manage, and its suggested actions include documenting model details, verifying provenance for training, testing, evaluation, validation, verification, fine-tuning, and retrieval-augmented generation data, and using transparency controls that support traceability. In other words, the recognized standard does not say 'buy a better model.' It says control and document what the model is working from. Grounding is not a feature. It is risk management.
An AI is only as trustworthy as what it reads
Here is the whole thesis in one line: an AI is only as trustworthy as the material it is allowed to read. Give it access to your curated, current, authoritative documents and its answers can be traced back to them. Give it access to nothing relevant and it has to rely on model memory, which may be stale, generic, or simply wrong for your context.
The model is not the variable you control most. The inputs are. Two identical models, one pointed at verified source material and one pointed at open-ended memory, can behave completely differently on your questions. The first can cite retrieved evidence. The second can only answer from what it already has available. Same engine, different trust story, and the difference is largely about what it was allowed to read.
Grounded AI needs a trust chain
Grounding is a chain, not a label. The source has to be allowed, current, permissioned, retrievable, cited, and reviewable. If any link breaks, the answer should say so instead of filling the gap with confidence.
This is where search quality and source governance meet. The system can search broadly, but it still has to rank authoritative sources, preserve access rules, quote the passage behind the claim, and refuse when the evidence is not enough. ISO/IEC 42001 points in the same direction for AI management systems by requiring a documented process for recording the provenance of data used in AI systems. Grounded does not mean closed. It means traceable.
Grounding only works if the sources are curated
Grounding is necessary, but it is not a magic word. Stanford benchmarked leading legal AI tools built around retrieval-augmented generation and found they still produced incorrect information more than 17% of the time for Lexis+ AI and Ask Practical Law AI, and more than 34% of the time for Westlaw's AI-Assisted Research. Pointing a model at documents is not enough if the wrong document gets retrieved, if unrelated passages get blended, or if the source set is a junk drawer.
So the quality of grounding depends on the quality and control of the source set. Curated, permissioned, trusted material produces answers you can defend. A pile of everything produces confident noise. The goal is not just 'grounded AI.' It is grounded in the right things, with every answer traceable to which thing, so a human can verify it fast.
How Sorena controls what the AI reads
This is why grounding at Sorena starts with the inputs, not the model. Sorena Integrations let you connect the trusted systems and documents your work actually lives in, so the AI reasons over verified material instead of relying only on open-ended memory. You decide what it is allowed to read. It cannot cite what you did not give it.
Those inputs feed Sorena SSOT, our Single Source of Truth, where the curated, permissioned source set lives. Because answers are designed to be drawn from and traced back to governed material, grounding is not a hopeful setting. It is the architecture. Control the sources, and you reduce fiction at the point where it starts.
The model can change. The discipline cannot.
Models will keep improving, and you should be able to use better ones as they arrive without rebuilding your trust story. That is fine, because trustworthiness should not come from the specific model alone. It should come from the discipline around it: curated inputs, enforced citation, human review.
Swap the engine and the discipline still holds. Keep the same engine and remove the discipline and you are back to confident fiction. This is the honest limit worth stating plainly: no model, present or future, is trustworthy on its own for GRC work. What is trustworthy is a system that constrains what the model reads and traces what it says. The model is a component. Grounding is the control surface.
Control the inputs or expect invention
An ungrounded model is not neutral. It can become a fluent guesser that does not reliably tell you when it is guessing. NIST calls that output confabulation. In GRC you would call it a finding.
The defense is not just a smarter model and not more faith. It is control: decide what the AI is allowed to read, curate those sources, and make every answer trace back to them so a human can check the work. Do that, and the AI has a basis for confidence. Skip it, and the confidence can become fiction with good posture.
Frequently asked questions
Doesn't a more advanced model just fix hallucination on its own?+
No. Capability and grounding are different things. Stanford found even purpose-built legal RAG tools still produced incorrect information more than 17% to more than 34% of the time, and NIST treats confabulation as a risk to be managed through provenance, testing, documentation, and traceability, not something a model simply outgrows. Trustworthiness comes from controlling what the model reads and tracing what it says, not from the model alone.
Is grounding the AI in our documents enough by itself?+
Grounding is necessary but not sufficient. It only works if the source set is curated, current, and permissioned, and if the system retrieves the right passage and cites it. A poorly controlled source set produces confident noise. The goal is grounding in the right material, with every answer traceable to which document, so a human can verify it quickly.
Can a grounded AI answer be treated as compliant advice?+
No. Grounding makes an answer traceable and verifiable, which supports a human decision. It is not legal advice and not a substitute for qualified counsel or a compliance professional. The model is a component; the accountability stays with the people who curate the sources and approve the outcome.
Sources
- NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative AI Profilehttps://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf?ref=sorena.io
- AI Risk Management Framework (NIST)https://www.nist.gov/itl/ai-risk-management-framework?ref=sorena.io
- ISO/IEC 42001:2023, Artificial intelligence management systemhttps://www.iso.org/standard/81230.html?ref=sorena.io
- AI on Trial: Legal Models Hallucinate in 1 out of 6 or More Benchmarking Queries (Stanford HAI)https://hai.stanford.edu/news/ai-trial-legal-models-hallucinate-1-out-6-or-more-benchmarking-queries?ref=sorena.io


