With every technological advancement, each innovation introduces groundbreaking opportunities and new security challenges. While Gen AI has taken the tech world by storm, security concerns have slowed adoption across organizations concerned with data privacy, securing novel architectures, and the lack of control while leveraging third-party LLMs. These new attack surface areas could potentially create large opportunities to secure them. Reflecting on past trends across our cybersecurity investments, we’ve seen massive cybersecurity companies emerge to secure this new tech and enable widespread adoption. The rise of new endpoints from BYOD led to Crowdstrike’s EDR, the rise of Cloud led to companies like Zscaler to secure users and data outside the network perimeter, and the growth of customer accounts helped IAM providers like Foregrock, all companies we’ve been fortunate to partner with in the past. 

AI infrastructure is acutely vulnerable and presents the first attack surface where users and attackers could interact with models directly. However, AI model security goes beyond the prompt injections — MLOps pipelines, inference servers, and data lakes face serious threats if not safeguarded with advanced security measures, continual vulnerability updates, and adherence to stringent standards like FIPS and PCI-DSS. 

We’re focused on this opportunity for new players to innovate in AI model security and are eagerly looking for teams to back. While existing cybersecurity companies secure some aspects of the AI model posture, such as network security protocols, endpoint protection, and encryption practices, they often need to improve in directly tackling the unique risks associated with the latest AI/ML pipelines. We believe this gap in the cybersecurity landscape has paved the way for a new market, where specialized vendors are emerging to address the end-to-end security needs of ML systems, from securing training data to controlling the behavior of the models themselves.

Key Problems and Main Use Cases:

Deploying AI models involves tackling complex security issues such as data security, biases and fairness, and explainability. The sensitivity of the data and the need for equitable outcomes necessitate stringent security and fairness protocols. For instance, ensuring data security and privacy involves implementing robust encryption, access control, and data anonymization techniques. Additionally, making AI models more explainable and interpretable is crucial for trust and regulatory compliance, as it helps in understanding their decision-making processes.

Attack Method:

​​Attackers could use various methods to exploit weaknesses in AI models. Understanding these methods is vital for creating effective defenses against these security challenges: 

• Model Inversion: Attackers could reconstruct training data, potentially exposing sensitive information.
• Model Stealing: Attackers could replicate AI models to bypass security protocols or replicate proprietary algorithms.
• Prompt Injection Attacks: Malicious prompts might manipulate models into generating harmful or unintended outputs, such as inappropriate content or sensitive data exposure.
• Model Evasion: Attackers could modify input data to induce errors, such as misclassification, potentially enabling evasion of security measures like spam or malware detection.
• Data Poisoning: This involves manipulating training data, which could lead to either specific misclassifications or a degradation in overall model performance. There are two main types of data poisoning attacks:‍‍
• Targeted Poisoning Attacks: These would be aimed at causing specific misclassifications, like making a facial recognition system misidentify a person.‍
• Exploratory Poisoning Attacks: Designed to potentially degrade a model's general performance, often by introducing random noise into the training data.

Areas of Opportunity: 

As the trust and performance of AI continue to improve, industries will begin to rely on AI models for increasingly critical workflows, and Ansa plans to back these new platforms. We expect this to create a step function increase in demand for security tooling across the following vectors, as the consequences of a breach could be catastrophic. The following points highlight key vulnerability areas for new and emerging AI model security companies:

Enhancing Training Data Security & Privacy:

• Generating artificial datasets to train machine learning models.
• Altering how training data is represented to minimize the risk of sensitive information leaks.
• Strengthening the security of data sharing and access with protective measures and encryption.

Tools for Testing Model Security:

• They are utilizing simulations of adversarial attacks and defense mechanisms such as FGSM, PGD, and Carlini-Wagner Attack to uncover vulnerabilities and rigorously test models before they go live.

Monitoring Tools for Post-Production:

• They continuously track models after deployment to spot anomalies or declines in accuracy and performance.
• These tools also enhance understanding of the decision-making process of ML models, contributing to model explainability.

Detection and Response Tools for Model Attacks:

• Implementing protective measures for ML models, coupled with intrusion detection and threat intelligence systems, enables prompt detection and counteraction of ML-based threats, including data tampering, evasion tactics, and model replication.

AI Governance, Risk, and Compliance Management Tools:

• Implementing safety and compliance tools ensures that AI models adhere to regulatory standards and organizational guidelines.

This landscape will continue to evolve, and we expect a new category to emerge to match AI adoption's security and safety needs. At Ansa, we’re committed to working with founders at the forefront of AI model security. If you are a founder with innovative ideas or solutions in AI model security, we’d love to connect! Reach out to Allan Jean-Baptiste at allan.jean-baptiste@ansa.co and Ryan Sullivan at ryan@ansa.co.