Internal AI search: check document versions, access and sources
Decide which documents are current, who may read them and how an answer can be checked before choosing an AI search tool.
The workflow illustrates a possible design and is not a client case study.
- Check document access
- Retrieve and cite
- Review answer quality
Why can retrieved text still lead to a wrong answer?
An AI search can find a document and still give the wrong answer if an old version remains in use. A shared search index may also reveal information to someone who cannot open the original. Sort out the current versions, document owners and access rules before adding more material.
Check existing document permissions
Microsoft 365 Copilot uses the permissions on the original data. That still leaves you responsible for correcting access that is too broad. Compare custom search when sources span several platforms or you need a particular citation or workflow, including the work of keeping permissions up to date.
Attach sources and versions to answers
Start with approved documents from one department. Show the source and version with each answer, and say when there is not enough evidence to answer. Plan for deleted documents, changed permissions and expired content. Check access again when displaying a result.
Check that revoked access stays blocked
Ask the same questions with accounts that have different access. Then revoke access, delete documents and try conflicting versions or questions the documents cannot answer. Have the document owner check that each cited passage actually supports the answer.
Bring questions with verified sources
Bring the document groups, access rules and common questions, with the correct source for each answer.