JFrog & Hugging Face join forces to secure AI models

JFrog has partnered with Hugging Face to integrate its advanced security capabilities into the Hugging Face Hub, enhancing the security of machine learning (ML) models available on the platform.

The collaboration aims to improve the security of ML models, which are becoming increasingly important in critical business applications. Asaf Karas, Chief Technology Officer of JFrog Security, said, "As ML models become integral to critical business applications, ensuring these models are secure is crucial for preventing breaches, data leaks, and decision-making errors."

Machine learning introduces new elements to the supply chain, including models and datasets. These have inherent security challenges that increase an organisation's vulnerability to potential attacks. Such gaps allow malicious actors to execute remote code to spread harmful code, potentially granting access to key internal systems, leading to data breaches or corporate espionage.

With this integration, Hugging Face will utilise JFrog Advanced Security scans, allowing each ML model on the Hugging Face Hub to be checked for threats before download. Threats range from model serialisation attacks and known CVEs to backdoors. The scan results will be displayed prominently for users.

The new feature offers a more robust scanning approach than currently available solutions. JFrog's method involves decompiling malicious code and conducting detailed data flow analyses, enabling the elimination of over 96% of false positives. This advancement identified 25 models on Hugging Face as zero-day malicious models not detected by other existing scanners.

Julien Chaumond, Chief Technology Officer at Hugging Face, commented on the move, stating, "For a long time, AI was a researcher's field, and the security practices were quite basic, but as the popularity and widespread use of AI grows, so do the number of potentially bad actors who may want to target the AI community in general and our platform more specifically."

"As the leading collaboration platform for AI models, we're delighted to deepen our partnership with JFrog to implement high-quality scanning capabilities for our AI/ML models and deliver greater peace of mind for developers looking to create the next generation of AI-powered applications."

Surveys indicate that while over 80% of enterprises are experimenting with AI applications, more than 90% feel unprepared for the security challenges posed by AI. Cybersecurity agencies from the U.S., the U.K., and Canada have urged businesses to thoroughly scan pre-trained models for harmful code.