"What did you say?'': Extracting unintentional secrets from predictive text learning systems

No ratings

Presented at CyberScience2020 2020 by

Gwyn Legg

As a primary form of communication, text is used widely for online communications, including e-mailconversations, mobile text messaging, chatroom and forum discussions. Modern systems include facilities such aspredictive text, recently implemented using deep learning algorithms, to estimate the next word to be written basedon previous historical entries. However, we often enter sensitive information such as passwords using the same inputdevices - namely, smartphone soft keyboards. In this paper, we explore the problem of deep learning models whichmemorize sensitive training data, and how secrets can be extracted from predictive text models. We propose ageneral black-box attack algorithm to accomplish this for all kinds of memorised sequences, discuss mitigations andcountermeasures, and explore how this attack vector could be deployed on an Android or iOS mobile device platformsas part of target reconnaissance.