--- title: AI description: Tools and resources for pentesting and securing artificial intelligence systems. --- ![](/assets/headers/header-logo.png) ![](https://img.shields.io/badge/Tools%20%26%20Resources%20Available-24-757575?style=for-the-badge) ### Evasion, Poisoning, Extraction * [Adversarial Robustness Toolbox (ART)](https://github.com/Trusted-AI/adversarial-robustness-toolbox) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference. ![GitHub last commit](https://img.shields.io/github/last-commit/Trusted-AI/adversarial-robustness-toolbox?style=flat) * [CounterFit](https://github.com/Azure/counterfit) - A CLI that provides a generic automation layer for assessing the security of ML models. ![GitHub last commit](https://img.shields.io/github/last-commit/Azure/counterfit?style=flat) * [Foolbox](https://github.com/bethgelab/foolbox) - A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX. ![GitHub last commit](https://img.shields.io/github/last-commit/bethgelab/foolbox?style=flat) ### Jailbreaking * [EasyJailbreak](https://github.com/EasyJailbreak/EasyJailbreak) - An easy-to-use Python framework to generate adversarial jailbreak prompts. ![GitHub last commit](https://img.shields.io/github/last-commit/JailbreakBench/jailbreakbench?style=flat) * [JailbreakBench](https://github.com/JailbreakBench/jailbreakbench) - An Open Robustness Benchmark for Jailbreaking Language Models. ![GitHub last commit](https://img.shields.io/github/last-commit/JailbreakBench/jailbreakbench?style=flat) ### LLM & Generative AI Red Teaming * [Garak](https://github.com/NVIDIA/garak) - The LLM vulnerability scanner. ![GitHub last commit](https://img.shields.io/github/last-commit/NVIDIA/garak?style=flat) * [PromptBreach](https://github.com/microsoft/promptbench) - The Python Risk Identification Tool for generative AI. ![GitHub last commit](https://img.shields.io/github/last-commit/microsoft/promptbench?style=flat) * [PyRIT](https://github.com/Azure/PyRIT) - The Python Risk Identification Tool for generative AI. ![GitHub last commit](https://img.shields.io/github/last-commit/Azure/PyRIT?style=flat) * [Promptmap2](https://github.com/utkusen/promptmap) - A security scanner for custom LLM applications ![GitHub last commit](https://img.shields.io/github/last-commit/utkusen/promptmap?style=flat) * [Purple Llama](hhttps://github.com/meta-llama/PurpleLlama) - Set of tools to assess and improve LLM security. ![GitHub last commit](https://img.shields.io/github/last-commit/meta-llama/PurpleLlama?style=flat) ### Model Extraction & Inference Attacks * [Privacy Meter](https://github.com/privacytrustlab/ml_privacy_meter) - Audit data privacy in statistical and machine learning algorithms. ![GitHub last commit](https://img.shields.io/github/last-commit/privacytrustlab/ml_privacy_meter?style=flat) * [SecretFlow](https://github.com/secretflow/secretflow) - A unified framework for privacy-preserving data analysis and machine learning. ![GitHub last commit](https://img.shields.io/github/last-commit/secretflow/secretflow?style=flat) * [ShadowAttack](https://github.com/hncszyq/ShadowAttack) - Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon. ![GitHub last commit](https://img.shields.io/github/last-commit/hncszyq/ShadowAttack?style=flat) ### Data Poisoning & Supply Chain Attacks * [Backdoor Box](https://github.com/THUYimingLi/BackdoorBox) - A universal pytorch platform to conduct security researches of image classification in deep learning. ![GitHub last commit](https://img.shields.io/github/last-commit/THUYimingLi/BackdoorBox?style=flat) * [TrojanZoo](https://github.com/ain-soph/trojanzoo) - The open-sourced Python toolbox for backdoor attacks and defenses. ![GitHub last commit](https://img.shields.io/github/last-commit/ain-soph/trojanzoo?style=flat) ### LLM Safety & Guardrails * [Alibi Detect ](https://github.com/SeldonIO/alibi-detect) - Algorithms for outlier, adversarial and drift detection. ![GitHub last commit](https://img.shields.io/github/last-commit/SeldonIO/alibi-detect?style=flat) * [Guardrails AI](https://github.com/guardrails-ai/guardrails) - Adding guardrails to large language models. ![GitHub last commit](https://img.shields.io/github/last-commit/guardrails-ai/guardrails?style=flat) * [LLM Guard](https://github.com/protectai/llm-guard) - A comprehensive tool designed to fortify the security of Large Language Models. ![GitHub last commit](https://img.shields.io/github/last-commit/protectai/llm-guard?style=flat) * [NeMo Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) - Toolkit for easily adding programmable guardrails to LLM-based conversational systems. ![GitHub last commit](https://img.shields.io/github/last-commit/NVIDIA-NeMo/Guardrails?style=flat) * [Vigil](https://github.com/deadbits/vigil-llm) - Detect prompt injections, jailbreaks, and other potentially risky Large Language Model inputs. ![GitHub last commit](https://img.shields.io/github/last-commit/deadbits/vigil-llm?style=flat)