The National Institute of Standards and Technology (NIST) has unveiled its final guidelines for safeguarding artificial intelligence systems from cyberattacks, shedding light on emerging threats targeting both predictive (PredAI) and generative (GenAI) models.
NIST’s report released on March 24 introduces updated attack classifications and mitigation strategies that address key components of AI systems, their life cycle stages, and the tactics used by attackers based on their knowledge, access, and intent.
“The statistical, data-based nature of ML systems opens up new potential vectors for attacks against these systems’ security, privacy, and safety, beyond the threats faced by traditional software systems,” NIST stated in its report. “Such attacks have been demonstrated under real-world conditions, and their sophistication and impacts have been increasing steadily.”
The final version differentiates between PredAI and GenAI compared to NIST’s initial draft in 2024. PredAI threats are categorized based on the attacker’s objectives and capabilities, while GenAI faces unique risks from prompt injection attacks and misuse techniques like jailbreaking.
The report explained that hackers targeting GenAI often exploit a lack of separation between data and communication channels, sneaking in malicious instructions to manipulate the application’s behavior.
These prompt injection attacks occur in two ways: direct attacks, where hackers input malicious commands to override trusted instructions, and indirect attacks, where external data sources influencing the model are manipulated.
PredAI models face other attack types, including evasion attacks, data poisoning, and privacy breaches. The timing and method of these attacks often depend on the AI model’s lifecycle stage, according to NIST.
To counter these threats, the report outlines actions that can be taken to defend AI systems including adversarial training and safety-focused tuning, data sanitization, input validation, output monitoring, model inspection, and red teaming.
While steps can be taken to mitigate attacks, the report also noted that as hackers use increasingly sophisticated methods and new challenges arise, that “managing risks in AI systems is an area for ongoing work.”