Casey Carlton is Head of Technology Operations at Treeline, an a16z-backed company transforming the managed services industry.
Earlier this year, a story circulated in security and IT circles that deserved more attention than it got—not for the headline but for what it revealed about how fast-growing companies choose and trust the vendors that underpin their security posture.
In March, an anonymous whistleblower accused a venture-backed compliance automation startup of producing SOC 2 reports at scale from templated evidence—reports that looked legitimate, carried auditor sign-off and were used to tell customers, investors and partners: We’re secure. Allegedly, the controls behind many of those reports either didn’t exist or weren’t meaningfully verified. The company has denied the allegations, and they remain unproven.
Whether or not they prove out, the dynamic they describe isn’t unique to one company. I’ve spent over a decade as the first IT leader at venture-backed companies, and this year I’ve been running SOC 2 audits and other frameworks with an independent firm (full disclosure: my company provides IT and compliance services to growth-stage companies). What I see in compliance vendor evaluations, regularly, is the slow substitution of convenience for accountability.
When Compliance Becomes Theater
SOC 2 was designed to give buyers meaningful assurance that a vendor’s security controls actually work. As automation made the process faster and cheaper, it became something else: a sales asset, a checkbox in procurement, a trust badge on a website.
When the goal shifts from being secure to appearing secure, the incentives follow. Platforms promising compliance “in days” optimize for speed. Auditors who sign off on high volumes of reports without meaningfully reviewing evidence optimize for throughput. Companies that accept a report without asking who performed the audit, and how, optimize for convenience.
Everyone in the chain makes a rational choice. The cumulative result can look like compliance from the outside while providing none of the protection.
The Question Nobody Asks At Vendor Selection
In nearly every compliance-adjacent vendor evaluation I’ve been part of, almost nobody asks how the vendor actually does what it does.
The evaluation usually looks like this: feature checklist, pricing comparison, reference calls, then a decision. Speed and integrations win. The auditor relationship, the evidence review process and the independence of the firm signing the report get treated as implementation details rather than the entire point. That’s backward. For compliance, the how is the product. An automated screenshot of your cloud configuration isn’t evidence of a security control. It’s a screenshot. What turns it into evidence is an independent auditor who understands what they’re looking at and is willing to push back.
Before selecting a compliance technology partner, I’d push hard on three questions:
1. Who is the auditor, and what is their relationship to the platform?
A platform that steers you exclusively to partner firms has a conflict of interest worth scrutinizing. Ask whether you can bring your own auditor.
2. What does evidence review look like in practice?
Skip the sales deck and walk through a sample control. Ask what happens when evidence is insufficient. If the answer amounts to “Our AI validates it,” that’s automation without accountability.
3. What is your recourse if the report is challenged?
This reveals whether a vendor is selling a product or standing behind a result. Read the contract carefully, and understand what “compliant” means there versus what it means to your customers.
What This Actually Costs
A company holding a compliance report that’s later called into question has a specific set of problems. If it used that SOC 2 in its sales cycle—and most do—it may have made representations to customers it can’t back up. Under contractual security requirements or in a regulated industry, the exposure compounds.
That cost isn’t hypothetical. In the months after the allegations surfaced, the startup’s accelerator cut ties, and several of its customers publicly moved to new providers and began recertifying—redoing work they believed was already done, under the scrutiny of the very customers they’d made promises to. The irony is that the companies most likely to take the compliance shortcut are the ones least able to afford the fallout.
The Broader Principle
Compliance is simply the most visible example. The same dynamic applies to every technology partner a company brings into its infrastructure: vendors optimized for convenience, buyers who never scrutinized the model and a gap between promise and delivery that only appears at the worst possible moment.
The answer isn’t to distrust automation or reject tools that make hard things faster. Instead, stay curious about how the people and platforms you trust do what they say they do—and ask the second question in every evaluation, not just the first. Convenience is not a strategy. The companies that win are the ones that chose their partners carefully, before they found out it mattered.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?