CyberArk unveils open-source tool to help businesses protect against AI model hijacking

CyberArk announces the launch of FuzzyAI, a next-generation open-source framework that has jailbroken every AI model tested using it. FuzzyAI helps businesses identify and address vulnerabilities in AI models, such as bypassing guardrails or generating harmful outputs, whether hosted in the cloud or on-premises. As a preview, visitors to the Black Hat Europe 2024 event can explore the features and applications of its new tool, to understand how businesses can adopt AI without creating cyber risks.

FuzzyAI, to meet what needs?

AI models are transforming all industrial sectors through innovative applications, for customer interactions, internal processes or automation. Using these models internally also raises new security challenges that leave most companies helpless.

FuzzyAI helps address these challenges by providing businesses with a systematic approach to testing AI models against various random inputs, highlighting potential vulnerabilities in their security systems and making AI development and deployment safer . A powerful fuzzer, a tool that reveals software defects and vulnerabilities, is at the heart of FuzzyAI, and makes it possible to highlight these already identified aspects using more than a dozen distinct attack techniques, from filter bypass ethics to the revelation of hidden system prompts.

Key features of FuzzyAI include:

• Random data testing (“fuzzing”): FuzzyAI subjects AI models to various attack techniques with the aim of exposing vulnerabilities, such as guardrail bypass, information leakage, injection prompt or generation of harmful outputs.

• An extensible framework: Companies and researchers can add their own attack methods to tailor tests to domain-specific vulnerabilities.

• A collaborative approach: a growing community ecosystem guarantees the permanent development of malicious techniques and the advancement of defense mechanisms.

“The launch of FuzzyAI further underlines CyberArk’s commitment to AI security and helps companies take an important step in addressing the security issues inherent in the evolving use of AI models. of AI, said Peretz Regev, Chief Product Officer of CyberArk. Developed by CyberArk Labs, FuzzyAI has demonstrated the ability to “jailbreak” all AI models tested. The tool strengthens the ability of businesses and researchers to identify their weaknesses and proactively fortify their AI systems against emerging threats.”

– Availability of FuzzyAI

FuzzyAI’s extensible framework has been available as open-source software on CyberArk Labs’ GitHub page since December 11, 2024. CyberArk Labs will host a Capture The Flag (CTF) workshop at the Black Hat Europe event Arsenal to present its practical applications and highlight the real impact of vulnerabilities resulting from jailbreaks and the urgent need to proactively solidify its defenses to mitigate them.