Growing the Community of AI Hackers with the Generative Red Team
No ratings
Presented at DEFCON 31 by
We’re running the largest live AI hacking event ever in the AI village this year. Anthropic, Google, HuggingFace, Meta, NVIDIA, OpenAI, and Stability, have all provided models to attack and Scale AI have built the platform. This event is orders of magnitude bigger than any previous AI red team effort. There are observers from the White House, NIST, NSF, and the EU coming to learn from hackers. We built this event to grow the community that knows how to effectively evaluate Large Language Models as it is much more than prompt injections and jailbreaks. AI works fundamentally differently to traditional software and only forms a part of a product. Trust and Security of AI in a system thus has to work fundamentally differently to traditional software. This is especially true for generative AI systems. The core difference is AI is a stochastic component of software and is allowed to make a small amount of mistakes. This changes bug hunting, reporting, and payouts. Come to this talk to hear about how and why we organized this, and the history of algorithmic & bias bounties that led up to the largest one ever at DEFCON 31. We’ll also give you some tips to help you in the contest. REFERENCES: We Need Bug Bounties for Bad Algorithms - Amit Elazari - https://www.vice.com/en/article/8xky...bad-algorithms Introducing Twitter’s first algorithmic bias bounty challenge - Rumman Chowdhury & Jutta Williams - https://blog.twitter.com/engineering...unty-challenge Sharing learnings from the first algorithmic bias bounty challenge - Kyra Yee & Irene Font Peradejordi - https://blog.twitter.com/engineering...unty-challenge Bias Buccaneers - Rumman Chowdhury, Jutta Williams, Subho Majumdar, Scott Steinhardt, Ben Colman - https://www.biasbuccaneers.org/ An Algorithmic Framework for Bias Bounties - Ira Globus-Harris, Michael Kearns, Aaron Roth - https://arxiv.org/abs/2201.10408 Machine Learning Security Evasion Competition - Hyrum Anderson, et al. - https://mlsec.io/ https://cujo.com/announcing-the-winn...n-competition/ MITRE ATLAS - Ram Shankar Siva Kumar, et al - https://atlas.mitre.org/ The Spherical Cow of ML Security - Sven Cattell - http://aivillage.org/adversarial%20ml/spherical-cow/ The Case for a Hippocratic Oath for Connected Medical Devices: Viewpoint - Beau Woods, Andrea Coravos, and Joshua David Corman - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6444210/ Announcing OpenAI’s Bug Bounty Program - https://openai.com/blog/bug-bounty-program Microsoft Malware Classification Challenge - Royi Ronen, Marian Radu, Corina Feuerstein, Elad Yom-Tov, Mansour Ahmadi - https://www.kaggle.com/c/malware-classification https://arxiv.org/abs/1802.10135