An AI tool review starts with its data
Before discussing the model, I would want to understand where the information comes from, where it goes and what the tool can do with it.
A tool can have a simple interface and a fairly complicated path behind it. There may be uploaded documents, a retrieval system, a model provider and integrations that act on the answer. I’d want those parts visible before deciding which controls to examine.
Map one use case properly
I’d start with the task the tool is meant to support. Who uses it? What would they normally enter? Which information does it retrieve? Who receives the output, and what decision or action follows?
Those questions make the data-flow discussion more concrete. They also help identify the owner and the points where personal, confidential or otherwise restricted information might enter the process.
For a fictional internal knowledge assistant, I would map the approved policy documents, prompts, retrieved passages and answer. I’d then ask about the provider, retention and the people who can access each part of that flow.
Look at denied access as well
I’d want evidence that retrieval respects the user’s permissions. A successful answer from an authorised document shows one path; it doesn’t tell me what happens when a user asks about a document they shouldn’t be able to read.
For the fictional assistant, I would propose tests with users who have different access, including requests that should be denied. I would record the conditions and results before concluding whether the control works.
The retrieved document can contain instructions
A document can carry an instruction as well as useful information. OWASP describes indirect prompt injection through content the model encounters, including retrieved material.
I would ask how the assistant distinguishes that content from instructions it is allowed to follow. For the example use case, I’d propose a document containing an instruction to disregard the user’s request or expose restricted information, then examine the system’s behaviour.
That is a proposed test, not a result. The finding would need the actual output, the access context and an explanation of the impact.
Ask which actions need a person
I would separate an assistant that suggests an answer from an integration that can change a record or run an action. For the latter, I’d map the available permissions, limits and approval points.
The human-review question should be specific. Who reviews the output? What are they expected to check? What happens when the answer is uncertain, unsupported or wrong? Naming a person in a workflow is easier than showing they have the information and authority to make the decision.
Revisit the review when the use changes
A new document collection, model or integration can change the questions. I’d want to understand who approves the change, how the relevant evaluations are repeated and who responds when a user reports a failure.
NIST’s AI Risk Management Framework helps organise the risk discussion. I would still connect it to the actual use case and the evidence available, rather than treating the framework’s headings as a completed assessment.
The work sample turns these questions into a short proposed review. It uses a made-up assistant and makes no claim about completed testing or conformity to a standard.