Indirect Prompt Injection Into LLMs Using Images and Sounds

No ratings

Presented at BlackHat Europe 2023 by

Multi-modal Large Language Models (LLMs) are advanced artificial intelligence models that can produce contextually rich responses that combine inputs of various types (text, audio, pictures). As a result, Bard already relies on such architecture, and the next generation of ChatGPT is expected to rely on them as well.In this talk, we demonstrate how images and audio samples can be used for indirect prompt and instruction injection against (unmodified and benign) multi-modal LLMs. An attacker generates an adversarial perturbation corresponding to the prompt and blends it into an image or audio recording. When the user asks the (unmodified, benign) model about the perturbed image or audio, the perturbation steers the model to output the attacker-chosen text and/or make the subsequent dialog follow the attacker's instruction.We will discuss possible threat models (and their significance with respect to existing threat models), the two types of attacks that can be applied (targeted output attack and dialog poisoning), and the method's technical implementation. We will further demonstrate targeted output attacks and dialog poisoning attacks against the two most popular open-source multi-modal LLMs (LLaVa and PandaGPT) and show how we created dedicated images and audio samples that caused the two LLMs to: (1) manipulate the user to visit a compromised website (phishing), (2) manipulate the user to provide personally identifiable information (information gathering), (3) distribute manipulated content, (4) answer like characters from films (pirate, Harry Potter).Speaker is presenting remotely.