How to Navigate the Stiff-Arm When Protecting Your Generative AI Initiatives

In the rush to build and deploy generative AI projects, it seems that many internal security teams are being kept at an arm’s length from the work that their data science teams are doing to push this emerging technology into the business portfolio. 

Perhaps they’re being seen as the clichéd “department of no.” Perhaps the old (and out-of-date) guidance to “move fast and break things” has become the adopted mantra of the AI promoters in their organizations.  

But as someone who tests both hosted foundational models and open-sourced home-grown generative AI models – and the various applications and data pipelines that support these – I can tell you that well-placed AI security tests need to be part of the proactive security journey in protecting your generative AI investments. There are a variety of ways to help your cybersecurity team persuade the business to understand how to build and deploy these initiatives securely. 

When done properly, such testing and oversight won’t disrupt your generative AI business case; it will protect it. By using a focused approach, you can integrate lessons learned and increase security awareness for the data science and business teams. In addition, you can help your AI governance committee identify when critical investments need to be made to manage the unique (and not so unique) risks that surround every AI project.

Common Attack Vectors for Generative AI 

The unique risks posed by adopting an LLM into your software portfolio are getting a lot of attention these days, and deservedly so. As this new tech becomes more and more mainstream, new attack vectors and vulnerabilities surface almost daily. It’s hard to keep up on what’s happening and decipher between a real risk versus just a potential or perceived one.  

At NetSPI, a lot of our focus is on helping our customers evaluate their models, data science pipelines, and applications for risks like evasion, poisoning, and extraction, and other attack scenarios. While these threats may seem esoteric to many, they are proven attack vectors in the real world. Many organizations that are experimenting with these data models are discovering that if they had integrated security testing earlier in their project’s pipeline, then they could have avoided a lot of pain from the surprises and disruptions that are part of addressing these unique risks too late. 

Focus on Protecting What You Can Control 

Just as we all discovered in the adoption of cloud-based computing, there are demarcation lines of responsibility that you should use as guidance for assessing risk in generative projects. 

If you’re using a hosted model, then a lot of those attack scenarios that I listed are out of your control. Why not focus on what you can control and use that to showcase your security team’s value to the business?  

Much of the security testing and risk management in a generative AI project can be addressed just by looking at the underlying infrastructure, data management, and development processes. Most of our customers’ security and risk management teams use this approach to both showcase their value and get a seat at the table for the ongoing generative AI work that they’ve been kept from advising on.  

Remember, as an attacker, threat actors want to achieve at least one of three goals:  

1. Hiding from you. 
2. Taking from you. 
3. Disrupting your business. 

They don’t have to be experts in generative AI to do this.  

As a defender, you need to consider these threats in the context of your projects (and data pipelines). At NetSPI, we often advise focusing on the basics when protecting your company’s generative AI projects:  

1. Protect the credentials of those accessing and deploying the project. 
2. Protect the infrastructure that they run on. 
3. Protect the data that goes into training these models, as well as the data that comes out of them. 

The 80/20 rule seems to run the world these days and it applies here as well. I’m sure you’d find that 80% of your security efforts could focus on these three areas and be very productive.  

Conduct an AI-focused Threat Model Exercise 

At NetSPI we’ve seen many LLM-based software projects – from the large foundational models to the smaller, scenario-focused LLMs, to filter prompts that manage what the LLM returns to the users. All of them benefit from aligning in the above three areas as a foundational AI cybersecurity strategy.  

If you’re still getting stiff-armed from the business because they feel that you could jeopardize the rush to deploy LLMs (FOMO is a big part of many AI projects), then consider the recent advice of our Field CISO Nabil Hannan: conduct an AI-focused threat model exercise.  

I’m amazed at how these simple, low-cost exercises are an eye-opener for the executives and operational staff who participate in them. Whether it’s a multi-day exercise or one that takes just a few hours, these often serve as the first steppingstone in a focused journey toward building a robust proactive security and risk management approach to your generative AI and data science projects.  

Continue learning about protecting AI/ML implementations by accessing our eBook, The CISO’s Guide to Securing AI/ML Models.