What happens when an AI agent decides the best way to get the job done is to blackmail you?
That is not a hypothesis. According to Barmak Meftah, a partner at the cybersecurity company VC Ballistic Ventures, it recently happened to an employee of a company working with an AI agent. The employee tried to prevent what the agent wanted, what he was trained to do, and responded by scanning the user’s inbox, finding some inappropriate emails, and threatening to blackmail the user by sending an email to the board of directors.
“In the agent’s mind, it’s doing the right thing,” Meftah told TechCrunch at the end episode of Equity week. “It’s trying to protect the end user and the company.”
Meftah’s example recalls Nick Bostrom’s AI paperclip problem. The thought experiment illustrates the potential existential risk posed by superintelligent AI single-mindedly pursuing a seemingly innocuous goal – making paper clips – to the exclusion of all human values. In the case of this company’s AI agent, the lack of context about why the employee is trying to overcome the goal leads to the creation of sub-goals that remove those obstacles (via blackmail) in order to achieve the main goal. That combined with non-deterministic nature of AI agents means “things can be mischievous,” per Meftah.
Misaligned agents are just one layer of AI security challenges that portfolio companies Ballistic Witness AI try to solve it. Witness AI says it monitors the use of AI across companies and can detect when employees are using unapproved tools, block attacks, and ensure compliance.
Witness AI this week raised $58 million off the back of more than 500% growth in ARR and 5x scale headcount over the past year as the company appears to understand the use of AI shadows and scale AI safely. As part of the Witness AI fundraiser, the company announced new agent AI security protection.
“People are creating these AI agents that take over the authority and capabilities of the people who manage them, and you want to make sure that those agents don’t go rogue, don’t delete files, don’t make mistakes,” Rick Caccia, co-founder and CEO of Witness AI, told TechCrunch on Equity.
Techcrunch event
San Francisco
|
13-15 October 2026
Meftah sees agent use growing “exponentially” across the company. To complement the increase – and the speed of AI-powered attack engines – analysts Lisa Warren predicted AI security software will be an $800 billion to $1.2 trillion market by 2031.
“I think that observability and runtime frameworks for safety and risk are going to be very important,” Meftah said.
As for how such startups plan to compete with big players like AWS, Google, Salesforce and others who have built AI governance tools on the platform, Meftah said, “AI security and agent security is huge,” there are many approaches.
Many companies “want an independent platform, end-to-end, to provide observation and governance around AI and agents,” he said.
Caccia notes that Witness AI resides in the infrastructure layer, monitoring interactions between users and AI models, rather than building safety features into the models. And that was intentional.
“We’ve deliberately chosen parts of the problem that OpenAI can’t easily solve for you,” he said. “So that means we’re going to be competing more with the legacy security companies than the model guys. So the question is, how do you beat them?” those people?”
For his part, Caccia doesn’t want Witness AI to be one of those startups that just gets it. He wants his company to be a growing company and a leading independent provider.
“CrowdStrike does on the last point (protector). Splunk do it in SIEM. Okta do it in identity,” he said. “Someone came and stood next to the great man … and we built Witness to do that from Day One.