Third-Party AI Model Vendor Risk Assessment

The EU AI Act is the most useful forcing function here, not because it is the only regulation that matters, but because it makes the liability chain explicit in a way that removes ambiguity about where the enterprise stands. Under Article 26, deployers of high-risk AI systems carry direct obligations: human oversight, input data quality, monitoring, record-keeping. The vendor built the model; the deployer owns the consequences of using it. Pointing at the API provider when something goes wrong is not a defense, and regulators have been clear enough about that to make the argument embarrassing rather than merely unsuccessful.

The enforcement timeline structures planning horizons in concrete ways. Prohibited practices and AI literacy obligations became enforceable in February 2025. General-purpose AI model obligations followed in August 2025. Full Annex III high-risk obligations, covering most enterprise use cases in healthcare, finance, employment, and critical infrastructure, land August 2, 2026. If your organization has not yet formalized AI vendor governance, that calendar math is uncomfortable.

Two substantive benchmarks sit alongside the EU AI Act. NIST AI RMF 2.0, published in February 2024, provides a structured framework for evaluating vendor governance maturity and appears increasingly in procurement requirements and audit frameworks. NIST's December 2025 Cyber AI Profile (IR 8596) goes further, identifying AI-specific threat vectors, including data poisoning, adversarial inputs, and the insufficiency of conventional continuous monitoring, that traditional security assessments miss entirely.

The practical upshot: due diligence for an AI vendor now means documented evidence that you evaluated model drift, bias testing, training data provenance, and human oversight mechanisms. A questionnaire that doesn't touch those dimensions isn't defensible, legally or operationally. The bar moved, and it moved without asking whether procurement was ready.

How to Tier AI Vendors Before the Detailed Assessment Begins

Assessment effort is finite. Without tiering, a high-risk integration receives the same scrutiny as a low-stakes chatbot widget. That's like using the same map to navigate a minefield and a parking lot: technically consistent, practically catastrophic.

Four dimensions determine where a vendor lands. Data sensitivity: does the model process PII, PHI, financial records, or proprietary intellectual property? Decision autonomy: is the model advising a human who retains final authority, or is it triggering consequential actions directly? Integration depth: an API call embedded in a compliance workflow carries categorically different risk than the same call in a non-critical feature. Replaceability: how difficult is it to swap the vendor if model behavior degrades, the vendor is acquired, or the vendor fails?

Tier 1 vendors, those scoring high on at least two dimensions, warrant continuous monitoring, detailed contractual obligations, right-to-audit clauses, and quarterly key risk indicator reviews. Tier 2 vendors receive annual formal assessment, semi-annual risk classification validation, and a documented escalation path. Tier 3 gets a lightweight checklist and periodic review, with a standing flag to re-tier if integration scope expands.

ISACA's 2025 guidance adds a refinement worth taking seriously: AI introduces risks, specifically hallucinations, model drift, and opaque supply chain changes, that require targeted questions even at the tiering stage, not only in the detailed assessment downstream. Build AI-native criteria into the tier determination itself. Otherwise, a model embedded in a moderate-risk workflow gets categorized identically to a conventional SaaS product, and everything downstream is calibrated against the wrong baseline. That miscalibration compounds quietly until something breaks.

Gartner projects that large enterprises will run an average of ten GRC software products by 2028, up from eight in 2025. Tiering determines which vendors generate formal GRC workflow overhead and which get lighter-touch review. That isn't a minor efficiency question; it is the difference between a governance program that scales and one that collapses under its own process weight.

Data Governance Questions Every AI Vendor Assessment Must Answer

Training data provenance is the first and most consistently avoided topic in vendor conversations. The core question is whether the vendor can disclose what data the model was trained on and demonstrate it was lawfully acquired. Many cannot answer this completely, and that inability is itself a finding, not a gap to acknowledge and move past.

Data isolation at inference is equally consequential. When the enterprise sends prompts and context to the model, that data needs to stay where it belongs. Does the vendor log prompt and completion data, and for how long? Does inference data feed back into model training? For Tier 1 vendors, an explicit contractual prohibition on using customer data for model training is non-negotiable. Data residency commitments, covering where inference occurs and where logs are stored, belong in the same conversation. Vague language about "commercially reasonable efforts" to protect data has a way of meaning nothing when you actually need it to mean something.

Retention and deletion obligations need specific terms: how long does the vendor retain inference data, and can the enterprise trigger deletion? "Reasonable retention periods" is not enforceable governance; it is a placeholder that benefits the vendor.

Supply chain depth is the dimension most enterprises overlook entirely. A fine-tuned model built on a base model from a third party inherits that base model's training data risks, copyright exposure, and bias characteristics. Vendors rarely volunteer this information. The ask is simple: provide the subprocessor list, including base model providers. I was once in a vendor conversation where the representative didn't know which foundation model their product was built on — not evasion, actual ignorance, which is worse. It was the kind of moment where you realize the vendor can't name the shoulders they're standing on, and that answer belongs in the risk register.

The documentation to request: the vendor's data processing agreement specific to AI inference workloads, the subprocessor list, and any model card or system card documenting training data categories and known exclusions. Vendors who refuse to disclose subprocessors, cannot confirm data isolation, or lack a DPA specific to AI inference are communicating their governance posture without intending to.

Security Controls That Need to Be Present at the Model API Boundary

Prompt injection is the primary AI-native attack vector, and it is structurally unlike anything in conventional application security. Malicious instructions embedded in user input, or in retrieved documents and tool outputs, can override system instructions and redirect model behavior. The distinction between direct injection, arriving from the user, and indirect injection, arriving through retrieved context or tool outputs, matters because the latter is considerably harder to detect and considerably more common in agentic deployments. Ask vendors what detection and mitigation is built into the inference layer itself, not what the enterprise is expected to bolt on afterward.

Output filtering for PII and secrets leakage needs explicit documentation. Models can inadvertently reproduce training data or regurgitate sensitive content from context windows; ask for the vendor's documented controls on completion filtering.

At the API authentication layer, the question is whether the vendor supports OAuth 2.1 and OIDC or relies on static API keys. Static keys represent a known credential leakage risk, one that has grown more acute as MCP-based agentic deployments normalize credential storage in local configuration files. This is not a theoretical concern.

Security certifications to verify: SOC 2 Type II, ISO 27001, and any AI-specific attestations. Request the actual audit report, not the badge on the marketing page. The same logic applies to penetration testing: confirm that scope explicitly covers model inference infrastructure, not only the surrounding application layer. Vendors sometimes produce impressive-looking security documentation that, on close reading, says nothing about the model itself. Read it closely.

Incident response terms need specificity. What is the vendor's obligation to notify the enterprise of a security incident that has exposed inference data or model weights, and what is the SLA for that notification? "We will notify affected parties in a timely manner" is not a contractual commitment; it is ambient goodwill dressed up as one.

NIST IR 8596 calls out data poisoning controls and anomaly detection as AI-specific requirements that traditional security assessments routinely omit. These should appear in the vendor's own controls documentation. Their absence doesn't mean the vendor has gaps; it means the vendor hasn't looked. That's a different problem, and in some ways a worse one.

Model Transparency and What Vendors Should Be Able to Show You

A model card is the baseline artifact of evaluation maturity. It documents intended use cases, known limitations, evaluation methodology, and out-of-scope uses. The absence of any model card is not a neutral finding; it indicates the vendor has not conducted systematic evaluation, which means there is no documented basis for understanding where the model fails. Some vendors will tell you the model card is proprietary. That is a position, and it is one worth noting in the risk register.

Bias and fairness testing documentation should include the evaluation datasets, the demographic or domain slices tested, and the results. Vendors with mature processes can share methodology even when they decline to publish full results. Vendors who respond to this question with marketing language about their "commitment to responsible AI" are answering a different question than the one you asked. Redirect them.

Hallucination rate benchmarks require context to be useful. Generic question-answering benchmarks tell you almost nothing about model behavior in a legal, medical, or financial workflow. Ask for accuracy and factual consistency benchmarks on tasks that actually resemble the enterprise's intended use case, and treat vendor-supplied benchmarks on dissimilar tasks with proportionate skepticism. The gap between "our model scores well on MMLU" and "our model is reliable for your compliance workflow" is wide enough to matter operationally.

Model versioning and update notification deserve their own contractual treatment. When the vendor updates the model, does the enterprise receive advance notice? How long are prior model versions available for regression testing? Is there a changelog describing behavioral changes, not merely infrastructure changes? These are not unreasonable requests; they are the minimum needed to maintain a stable production integration. Vendors who find them burdensome are telling you something about how they operate.

For autonomous or agentic deployments, human oversight mechanisms become structurally important. Does the vendor's system support human-in-the-loop checkpoints, or does it assume full automation? The EU AI Act's Article 26 obligations make this a compliance question, not a design preference.

The practical tell on vendor governance maturity: vendors with documented processes welcome these questions and have the materials ready. Vendors who treat transparency questions as competitive intelligence threats are revealing exactly what their governance looks like.

Ongoing Observability After the Vendor Is Onboarded

A point-in-time vendor assessment is a snapshot of a system that is changing continuously. Model updates, vendor acquisitions, and base model supply chain changes all shift behavior without any action on the enterprise's part. The risk profile of a third-party AI vendor on onboarding day is not the risk profile six months later. This is the part of AI vendor governance that feels obvious in retrospect and gets skipped in practice, usually because the team that completed the assessment considers their job done.

The AI-specific key risk indicators worth tracking: model drift rate, measured by whether outputs on the same classes of inputs are shifting over time; bias incident frequency; latency and error rate changes, which are often the first observable signal of an undisclosed model update; and cost per inference, where sudden changes can indicate model substitution or configuration drift. That last one gets attention faster than the others, for obvious reasons.

Liminal's 2026 governance guide recommends a structured cadence: technical controls review quarterly, risk classification validation semi-annually, full vendor assessment annually. That is a reasonable baseline. Tier 1 vendors warrant more frequent review depending on integration depth and regulatory exposure.

Audit logging at the inference layer is both a compliance requirement and a forensic necessity. Every prompt and completion touching sensitive workflows should be logged with identity, timestamp, and session context. Under frameworks like the EU AI Act, the ability to reconstruct what the model was asked and what it returned is not optional.

Shadow AI complicates the picture significantly. More than 80% of workers use unapproved AI tools, according to UpGuard's data. That figure means the enterprise's sanctioned vendor governance program is covering a minority of actual AI usage. Observability infrastructure needs to surface what AI is being used across the organization, not only monitor how approved tools are behaving. Governing the approved list while ignoring everything else is a governance program in name only, and everyone downstream knows it.

Established GRC platforms, including Credo AI and IBM watsonx.governance, provide substantive coverage for governance documentation, vendor tracking, and policy management. Neither currently provides inference-time output blocking or automated threshold enforcement at the API boundary; that gap is where a dedicated AI control plane adds coverage that documentation-layer tools cannot replicate.

One thing worth stating plainly: the monitoring infrastructure built for AI vendor governance — audit logging, cost telemetry, drift detection, shadow AI visibility — is the same infrastructure the enterprise needs to govern its own internal agentic deployments. The investment compounds. That's rarer than it should be in enterprise architecture.

What a Complete Vendor Assessment Package Looks Like in Practice

The complete package has three distinct layers, each serving a different function in the governance lifecycle.

Pre-contract documentation establishes the basis for the go/no-go decision. Request: the model card or system card; a data processing agreement specific to AI inference workloads; the subprocessor list including base model providers; the SOC 2 Type II report and ISO 27001 certificate; bias and accuracy evaluation methodology and results; and an incident response policy with AI-specific provisions. This is the minimum.

Contract provisions are where assessment findings become enforceable obligations. The clauses worth holding the line on: explicit prohibition on training the model on customer data, stated rather than implied; advance notice of model updates with a behavioral changelog; version retention periods sufficient for regression testing; annual right to audit AI governance practices; data residency and deletion obligations with specific terms; breach notification SLA covering inference infrastructure. Some vendors push back on right-to-audit clauses. How hard they push back, and what specifically they object to, is informative.

Post-onboarding monitoring keeps the assessment live rather than archival. This layer includes the KRI dashboard, the review cadence, and the inference-layer logging that makes everything auditable. Without it, the pre-contract and contract layers are aspirational rather than operational.

Vendor non-cooperation at any stage is itself a signal. A vendor that declines to provide a model card, refuses a right-to-audit clause, or cannot name their base model providers is demonstrating their governance posture. That signal belongs in the risk register because it predicts behavior across every future interaction, not only this one.

One internal clarification the assessment process reliably surfaces: teams that cannot answer tiering questions about data sensitivity or decision autonomy have not yet resolved those questions internally. The vendor conversation cannot be meaningful until the enterprise understands what it is actually deploying the model to do. Start there.

Where the Governance Layer Sits in the Enterprise's Own Architecture

Vendor assessment produces commitments on paper. Paper does not intercept a prompt injection attack at 2 a.m. on a Tuesday. Enforcement requires an architectural layer that the enterprise controls, sitting between its applications and every AI vendor it uses.

The function of this layer is to close the gap between what the vendor promises and what actually happens at runtime. Identity and access enforcement determines which users and agents can call which models, ideally resolved through existing identity providers like Okta or Microsoft Entra ID rather than managed as a separate credential silo. Real-time threat detection catches prompt injection, PII in outbound requests, and secrets in completions before they reach the model or the user. Immutable audit logging generates the per-call records with identity, session context, and content hashes that compliance frameworks require. Cost and usage telemetry feeds KRI dashboards and surfaces model substitution or configuration drift before those changes show up as a production incident.

This layer is not a replacement for good vendor contracts or a mature GRC program. It is the enforcement mechanism that makes those commitments operational rather than aspirational. A well-scoped AI control plane, whether built on an internal API gateway extended for AI workloads or a purpose-built solution, is what turns a governance document into a runtime policy with actual teeth.

The architecture point is also the organizational point. Vendor risk assessment for AI is not a procurement event that concludes at contract signing. It is a continuous function, and the infrastructure that supports it needs to be designed as such from the beginning. Enterprises that treat the assessment as a checkbox and the architecture as an afterthought will find themselves reassessing the same vendors under worse circumstances, with less time and more regulatory exposure than they had when they started. Not a prediction; just pattern recognition at this point.