Vendor Risk Assessment: How to Run One That Security Teams Actually Follow

A vendor risk assessment is a structured review of a third party's security, privacy, and operational risk, completed before the vendor is granted access to your data, identities, or applications. It matters because vendor access does not stay theoretical. Once an OAuth grant is issued or an integration is authorized, that vendor holds live authority inside your environment.

The practical purpose is narrower than most programs admit. A vendor risk assessment answers four questions:

• What data will this vendor see or store?
• What actions can it take inside our applications?
• What identity does it use to take those actions?
• If it were compromised tomorrow, what is the blast radius?

Everything else in a standard questionnaire, the certifications, the policy documents, the continuity plan, exists to support those four answers. None of it replaces them.

The reason this has moved up the priority list is that third-party integrations now carry authority that behaves like an employee's. One compromised integration token can be replayed across every tenant that authorized it, which is why a single vendor breach reaches hundreds of organizations at once. The anatomy of a SaaS supply-chain attack traces that chain stage by stage.

It also helps to separate the assessment from the thing procurement runs.

The timing row is the one teams get wrong. Signature is a commercial event. Authorization is the security event. A vendor can be signed for weeks before anyone clicks the consent screen, and that screen is where the actual scope gets decided.

Tier the vendor before you send a single question. Assessment depth should be a function of exposure, and exposure comes from three inputs.

Data sensitivity. What classification of data touches this vendor? Regulated personal data, source code, financial records, and customer contract data sit at the top. Marketing copy and public product docs sit at the bottom. Score on what the vendor will actually receive, not what it theoretically could.

Access type. This is the input most programs underweight. Read-only access to one object is not the same as write access across an org. Rank from lowest to highest exposure:

1. No persistent access; data shipped manually or by file exchange
2. Read-only, scoped to specific objects
3. Read-only, org-wide
4. Write or delete, scoped
5. Write or delete, org-wide
6. Administrative or agentic access that can take actions on behalf of users

Operational dependence. If this vendor is unavailable for 48 hours, what breaks? A payroll processor and a screenshot annotation tool are not in the same tier even if both hold employee names.

Why not tier by contract value? Because the correlation is weak. Browser extensions, free-tier automation tools, and AI note-takers routinely request broad scopes and cost nothing. A free tool with write access to your CRM carries more blast radius than a six-figure contract that never touches production data. If your intake form routes by spend, your highest-exposure vendors are skipping review entirely.

One decision rule covers most edge cases: if a vendor can take an action inside a production application without a human in the loop, it is Tier 1 regardless of price, headcount, or contract length.

Every step has a specific failure mode. The failure note matters as much as the step.

1. Intake and use case definition. Capture what the business wants the vendor to do in one or two sentences, plus the data objects and applications involved. Assign the tier here. What goes wrong: intake forms ask for vendor name and cost but not data objects or access type. Without those you cannot tier, so everything defaults to the heavy process and the business starts routing around you.

2. Data flow mapping. Document where data goes: which systems the vendor reads from, what it stores, where that storage lives, which subprocessors receive copies, how long retention runs. What goes wrong: teams accept "encrypted at rest and in transit" as a data flow answer. Encryption is a control, not a map. You still do not know who holds a copy.

3. Questionnaire and evidence collection. Send a tier-appropriate questionnaire. For Tier 1, request the current audit report, penetration test summary, subprocessor list, and incident notification commitments. For Tier 3, an attestation and a scope check is enough. What goes wrong: accepting a certification as the whole answer. A report tells you a scope was audited on a date. It does not tell you whether the feature you are buying was inside that scope.

4. Access and scope review. Read the actual authorization scopes the integration will request. Compare each one against the use case from step 1. Any scope that does not map to a stated function gets challenged before approval. What goes wrong: this is the highest-value failure in the whole process. Teams assess the vendor thoroughly and then never read the consent screen. Someone with admin rights clicks authorize, the integration receives every scope in the request bundle, and what was granted has little to do with what was reviewed. Put a named owner on the consent screen and require a screenshot of the granted scopes in the assessment record.

5. Identity and credential review. Determine what identity the integration authenticates as, how the credential is stored, whether it expires, who can rotate it, and whether the vendor's own staff can use it. Bearer tokens deserve specific attention, because possession alone implies authorization in most implementations. What goes wrong: the integration is authorized under a named admin's account rather than a dedicated service identity. When that admin leaves, the integration either breaks or keeps running as an orphaned non-human identity with no owner. The OAuth grants that survive offboarding shows how often that happens.

6. AI and agent review. Run the agent-specific question set in the next section. Treat it as its own step, not a subsection of the security questionnaire, because the answers change the tier. What goes wrong: the questionnaire was written before the vendor shipped AI features. The reviewed product and the deployed product are different products.

7. Logging and evidence review. Confirm what the vendor logs about its own activity inside your tenant, whether those logs are exportable to your SIEM, and how long they are retained. What goes wrong: teams check that logging exists and never check that it is exportable. In-console logs with a 30-day window are not an investigation capability.

8. Decision, conditions, and record. Issue one of three outcomes: approve, approve with conditions, or reject. Record the tier, the granted scopes, the identity used, the AI answers, the reviewer, and the date. Conditions get owners and due dates. What goes wrong: "approve with conditions" becomes "approve" because nobody tracked the conditions. A condition with no owner and no date is a comment, not a control.

Standard vendor questionnaires ask about model training and skip the question that determines exposure: what identity and permissions do the product's agents operate under. Vendors answer it vaguely. Push until you get a specific answer.

The question that matters most: under what identity do your product's agents execute actions in our tenant, the invoking user's identity, a dedicated service identity, or an elevated application identity?

The three answers have very different consequences.

• Invoking user's identity. The agent inherits the caller's permissions. Acceptable and easiest to reason about, provided the vendor can show that permission checks happen per request rather than at session start.
• Dedicated service identity. The agent has its own fixed permission set. Workable if you can scope it. Dangerous if the scope is org-wide, because every user of the feature effectively gets the service identity's reach.
• Elevated application identity. The agent runs with privileges higher than any invoking user. This is where privilege escalation happens quietly. A user with no access to a data object asks the agent, the agent uses its own elevated credentials, and the user receives data your access controls were supposed to withhold. Nothing malfunctioned. AI agent privilege escalation covers the mechanism and how to detect it.

If the vendor answers "it depends on configuration," ask what the default is and whether you can see the setting. Configuration is not reality unless you can verify it.

The rest of the set:

Are AI features on by default, and can an administrator disable them per tenant? Default-on features change your exposure without a procurement event. You need a per-tenant off switch, not a roadmap promise.

Can agents take actions, or only read? Read means data exposure. Write means record modification, deletion, message sending, and workflow triggering. Ask for the list of write operations, and whether that list expands with product updates.

Can agent permissions be scoped within our tenant? Can you restrict the agent to named objects, fields, or record types? Can you restrict which user groups can invoke it? If the answer is no, the agent's reach equals your entire dataset.

Which model providers and AI subprocessors receive our data? Named providers, regions, and the data categories each receives. Ask whether the list can change without notice.

Is our data used for training or model improvement? The default, the opt-out mechanism, and whether prompts, outputs, and metadata are treated differently. Ambiguity here is a finding.

What agent activity logging exists, and is it exportable? Per-action records: which agent, invoked by whom, what it accessed, what it changed, when. In-product dashboards with short retention do not support an investigation.

What happens to the agent when its creator's account is disabled? Orphaned agents keep running on inherited credentials with no accountable owner. This question surfaces whether the vendor has any agent lifecycle model at all. Non-human identity management for AI agents describes what a real lifecycle looks like.

Standard questionnaires tell you how a vendor secures its infrastructure. Agent questions tell you what the vendor's software can do inside yours. Only one of those describes your blast radius.

Reassess by tier and by trigger, not by calendar habit. Annual reviews of every vendor generate volume without insight. Tier 1 gets a full reassessment annually plus on any trigger. Tier 2 every 18 to 24 months plus on trigger. Tier 3 gets a scope check at renewal only.

Trigger events force immediate review regardless of tier: the vendor ships AI or agent features, requested scopes change, the vendor discloses an incident, the subprocessor or model provider list changes, ownership changes, or your use case expands to new data or applications. The AI trigger is the active one right now. A vendor assessed 14 months ago may have added agentic features that read and write across your tenant, and the assessment on file describes a product that no longer exists.

Some answers should change your decision on their own:

• A vague answer on agent execution identity. If a vendor cannot say plainly whether agents run as the invoking user or as an elevated application identity, they may not know. That is a finding.
• Refusal to share a subprocessor list. The practical translation is that you cannot map your own data flow.
• No exportable audit logging. If activity cannot leave the vendor's console, you cannot investigate on your own timeline.
• AI features on by default with no per-tenant control. Your exposure changes on the vendor's release schedule.
• Requested scopes far beyond the use case. A calendar tool asking for full mailbox write access is a scope design problem, not a bug.
• An audit report whose scope excludes the product you are buying. Read the system description, not the cover page.
• A personal admin account used for authorization. It signals no service identity model.
• Pressure to skip review before quarter end. Timeline pressure is not a risk assessment.

The mistakes that repeat across programs of every size: tiering by contract value, treating a certification as the answer, never reading the consent screen, assessing at signature instead of at authorization, using one questionnaire for everything, not tracking conditions, skipping the agent questions, and filing the assessment and stopping. Most of these are process failures, not knowledge failures, which is the good news. They are fixable without a bigger team.

A usable template is a spreadsheet with three tabs: tiering inputs, the tier-appropriate question set, and a decision record. Anything more complex slows the process, and slow processes get bypassed. The checklist below is the content of those tabs.

Scope and tiering: use case in one or two sentences; data classifications identified; access type ranked on the six-level scale; operational dependence rated; tier assigned and recorded.

Security and data: current audit report reviewed, with scope confirmed to include the purchased feature; subprocessor list obtained; retention and deletion terms documented; incident notification timeline in writing, in hours; encryption and key management described.

Access and identity: requested scopes listed and mapped to the use case; consent screen reviewed by a named owner; granted scopes captured in the record; service identity used rather than a personal admin account; credential rotation and expiry confirmed.

AI and agents: agent execution identity answered specifically; default AI feature state and per-tenant disable confirmed; read versus write capability enumerated; in-tenant permission scoping confirmed; model providers and AI subprocessors named; training use and opt-out documented; exportable agent activity logging confirmed; orphaned-agent behavior answered.

Decision: outcome recorded with reviewer and date; conditions assigned owners and due dates; next review date set by tier.

If you would rather start from a structured version of the agent inventory and authority questions, the SaaS AI agent risk assessment covers them in a repeatable format.

Then the part the template cannot do. The assessment describes state at authorization. The access stays live and drifts. Scopes get expanded during a support call, new AI features ship enabled, an owner leaves and the integration keeps running, a vendor adds a model subprocessor without a procurement event. Nothing in a completed questionnaire notices any of that, and it is the reason a good assessment program still needs the continuous layer described in third-party risk management in the AI era.

Obsidian is that layer. It correlates integrations, non-human identities, agents, and their actual actions into one view, so security teams can compare what the assessment recorded against what the integration is doing now. That is the difference between theoretical configuration and effective authority: what the vendor's software was set up to do versus what it can reach inside your applications after every entitlement resolves. In practice it closes the specific gaps an assessment leaves open: granted scope versus reviewed scope, the identity an agent actually used versus the one the vendor described, integrations and agents whose owner has been disabled, AI tools that entered the environment without an assessment at all, and the toxic combinations where several of those stack on one connection. The product background is in introducing Obsidian integration risk management.