The model may not say 'I don't know'
Faced with a gap, a language model may produce a statistically plausible continuation that reads like a real answer.
An ungrounded model may name a regulation that does not exist, cite an invented clause, or assert a requirement without support, using the same tone it uses when it is right. Without citations, retrieval evidence, or review, the answer may not reveal the error.
NIST has a name for it: confabulation
NIST names this risk in its AI Risk Management Framework Generative AI Profile. It defines confabulation as 'the production of confidently stated but erroneous or false content, known colloquially as hallucinations or fabrications, by which users may be misled or deceived.'
NIST lists confabulation among the risks organizations must manage. Its suggested actions include documenting model details, verifying provenance for training, testing, evaluation, validation, verification, fine-tuning, and retrieval-augmented generation data, and using transparency controls that support traceability. Organizations need to control and document what the model works from.
Source quality constrains answer quality
Give an AI access to curated, current, authoritative documents and its answers can trace back to those sources. Without relevant material, it relies on model memory that may be stale, generic, or wrong for your context.
Inputs are a major control. Two identical models can behave differently when one uses verified source material and the other relies on memory. The first can cite retrieved evidence; the second cannot.
Grounded AI needs controlled sources and traceability
Grounding requires allowed, current, permissioned, retrievable, cited, and reviewable sources. If any of those checks fails, the answer should report the gap.
The system must rank authoritative sources, preserve access rules, quote the passage behind each claim, and refuse when evidence is insufficient. ISO/IEC 42001 requires a documented process for recording the provenance of data used in AI systems. A grounded answer should be traceable.
Grounding only works if the sources are curated
Stanford benchmarked leading legal AI tools built around retrieval-augmented generation and found incorrect information in more than 17% of responses for Lexis+ AI and Ask Practical Law AI, and more than 34% for Westlaw's AI-Assisted Research. Retrieval can still select the wrong document, blend unrelated passages, or draw from a poor source set.
Curate and permission the source set, then trace every answer to the material it used so a human can verify it.
How Sorena controls what the AI reads
Sorena starts grounding with the inputs. Sorena Integrations connects trusted systems and documents so the AI can use verified material instead of relying only on model memory. You decide what it may read.
Those inputs feed Sorena SSOT, our Single Source of Truth, where the curated, permissioned source set lives. Answers draw from and trace back to that governed material.
Keep source controls when models change
Models will keep improving, and you should be able to adopt them without rebuilding source controls. Trust depends on curated inputs, enforced citations, and human review as well as model capability.
No model is trustworthy on its own for GRC work. The surrounding system must constrain what it reads and trace what it says.
Control the sources and trace each answer
An ungrounded model may guess without saying so. NIST calls this output confabulation.
Decide what the AI may read, curate those sources, and trace every answer back to them so a human can check the work.
Frequently asked questions
Does a more advanced model eliminate hallucinations?+
No. Stanford found that purpose-built legal RAG tools still produced incorrect information in more than 17% to more than 34% of benchmark responses. NIST treats confabulation as a risk to manage through provenance, testing, documentation, and traceability. Control what the model reads and trace each answer to its sources.
Is grounding the AI in our documents enough by itself?+
Grounding is necessary but not sufficient. It only works if the source set is curated, current, and permissioned, and if the system retrieves the right passage and cites it. A poorly controlled source set produces confident noise. The goal is grounding in the right material, with every answer traceable to which document, so a human can verify it quickly.
Can a grounded AI answer be treated as compliant advice?+
No. Grounding makes an answer traceable and verifiable, which supports a human decision. It is not legal advice and not a substitute for qualified counsel or a compliance professional. The model is a component; the accountability stays with the people who curate the sources and approve the outcome.
Sources
- NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative AI Profilehttps://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf?ref=sorena.io
- AI Risk Management Framework (NIST)https://www.nist.gov/itl/ai-risk-management-framework?ref=sorena.io
- ISO/IEC 42001:2023, Artificial intelligence management systemhttps://www.iso.org/standard/81230.html?ref=sorena.io
- AI on Trial: Legal Models Hallucinate in 1 out of 6 or More Benchmarking Queries (Stanford HAI)https://hai.stanford.edu/news/ai-trial-legal-models-hallucinate-1-out-6-or-more-benchmarking-queries?ref=sorena.io


