How to evaluate an AI vendor's security: 12 questions for your IT lead

Before any AI vendor touches your clients’ data, get clear written answers on six things: how your data is kept apart from other customers, how it’s encrypted, what the AI models keep, who approves what, what’s been independently audited, and how you get out. The 12 questions below cover all six. A good vendor answers them plainly. A vague answer is itself an answer.

Why this matters more for AI

AI vendors don’t just store your data; they send it to models, combine it with your firm’s knowledge and act on it. That creates new places for data to leak: model providers that keep prompts, shared systems where one customer’s data sits next to another’s, and automated actions nobody approved. Your clients trust you with their information, and regulators and insurers increasingly expect you to show how your vendors protect it. The FTC Safeguards Rule, for example, requires covered financial institutions (including many tax and accounting firms) to oversee their service providers’ safeguards.

Isolation

1. How is our data kept apart from your other customers? Look for isolation enforced by the database itself (for example, row-level security on every table), not just by application code. Application-only separation fails when there’s a bug.

2. Can your own staff or services bypass that separation? Ask which accounts can see across customers, and how that access is limited and recorded.

Encryption

3. Is our data encrypted with keys specific to our firm? Encryption “at rest” with one shared key protects against stolen disks, not against mix-ups between customers. Firm-specific keys, held in a managed key service, limit the blast radius.

4. What happens to our keys and data when we leave? You want your records returned to you as a verified export before your keys are destroyed, so what remains can’t be read.

AI models

5. Which AI models see our data, and under what terms? Ask for the contractual terms, not a marketing page: does the provider retain prompts and outputs, and is your data ever used for training?

6. Is zero data retention actually in effect, and how is it enforced? “Available” isn’t “in effect.” Ask whether the vendor’s system blocks a model from being used unless those terms are in place. See zero data retention, explained.

7. Is every model call recorded? You should be able to learn which job called which model, when, and with what result.

Control and approvals

8. Does anything reach our clients without a person approving it? The right answer is no. Agents should prepare; people should approve.

9. Can approval records be changed after the fact? Approvals should be permanent and tied to exactly what was approved, with separation of duties between who prepares and who approves.

Independent evidence

10. What’s been independently audited? Ask for a SOC 2 report or its status. If the vendor hasn’t been audited yet, they should say so plainly and show you their written policies and the controls behind their claims. Treat unverifiable certifications as a red flag.

11. Where is our data hosted, and how is it backed up? Get the cloud provider, the region and the backup retention in writing.

Exit and retention

12. Can we set retention and place a legal hold? Your records policy and any litigation hold should override the vendor’s defaults, and deletion should be reviewable before it happens.

How to use the answers

Score each answer as enforced by the system, policy only, or not addressed. “Enforced by the system” beats “policy only” every time. Keep the written answers; they’re your evidence when a client, insurer or auditor asks how you chose the vendor.

How Precision AI OS answers

We publish our answers on our security page: data isolated in the database on every table, firm-specific encryption, no language model without zero-retention and no-training terms, a permanent record of approvals and model calls, and retention and legal hold under the retention rules in your agreement. We haven’t completed a SOC 2 audit yet, and we say so.