Poisoning Web-Scale Training Datasets is Practical

No ratings

Presented at BlackHat USA 2023 by

Many popular deep learning models often rely on massive, distributed datasets gathered from the internet. Due to licensing and other such factors, these datasets are usually stored as a list of URLs where training samples can be retrieved. However, domains expire and could be purchased by a malicious actor. This issue not only affects StableDiffusion, but also Large-Language Models such as ChatGPT that are trained on internet sourced data. In this talk, we will explore the strategies used by model builders to create large datasets, and then show two attacks that exploit these mechanics. Our first attack abuses a well-known issue, domain squatting. We take advantage of the ever-changing nature of online content and domain control to make sure a dataset annotator's first look at the dataset differs from what's downloaded by other users. In our second attack, frontrunning poisoning, we zero in on massive datasets that take snapshots of crowd-sourced content, like Wikipedia, where all an attacker needs is a brief window of time to inject malicious samples.Together these attacks represent a significant challenge to the current state of trust as it relates to foundational models and the datasets they're trained on. These attacks are possible today, and only require a Malicious Actor control 0.01% of a dataset.