Leaky Deltas: Exposing Data Updates in Natural Language Models

Please note: This seminar will be run in hybrid mode. Please join in person in 42-115 in Prentice Lecture theatre or via zoom https://uqz.zoom.us/j/85629419739

Abstract

To continuously improve quality and reflect changes in data, machine learning applications must regularly retrain and update their core models. This seminar will show that a differential analysis of language model snapshots before and after an update can reveal a surprising amount of detailed information about changes in the training data. To demonstrate the leakage due to updates of natural language models, we develop two metrics — differential score and differential rank. The extent of leakage analysis that is possible using these metrics across a range of models and datasets will be discussed as well as the privacy implications of the findings, mitigation strategies, and evaluation of their effect.

The first part of the talk is based on joint work with Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Victor Rühle, Andrew Paverd, Boris Köpf and Marc Brockschmidt. https://arxiv.org/pdf/1912.07942.pdf

Speaker

A/Prof Olya Ohrimenko

Olya Ohrimenko is an Associate Professor at The University of Melbourne which she joined in 2020. Prior to that she was a Principal Researcher at Microsoft Research in Cambridge, UK, where she started as a Postdoctoral Researcher in 2014. Her research interests include privacy and integrity of machine learning algorithms, data analysis tools and cloud computing, including topics such as differential privacy, verifiable and data-oblivious computation, trusted execution environments, side-channel attacks and mitigations. Recently Olya has worked with the Australian Bureau of Statistics and National Bank Australia. She has received solo and joint research grants from Facebook and Oracle and is currently a PI on an AUSMURI grant. Olya holds a Ph.D. degree from Brown University and a B.CS. (Hons) degree from the University of Melbourne. See https://people.eng.unimelb.edu.au/oohrimenko/ for more information.

Host

Dr Guido Zuccon 

Dr Guido Zuccon is an Associate Professor at the University of Queensland, Information Technology and Electrical Engineering School, an ARC DECRA Fellow (2018-2020), an Affiliate Associate Professor at the UQ Centre for Health Services Research, Faculty of Medicine, and a Visiting Principal Scientist at Queensland Health. He leads the Information Engineering Lab (ielab), a research team working in Information Retrieval and Health Data Science.

Guido's main research interests are Information Retrieval, Health Search, Formal Models of Search and Search Interaction, and Health Data Science. He has successfully attracted funding from the ARC via an ARC Discovery Early Career Research Award Fellowship and an ARC Discoverty Project. His research has also been funded by Google (Google Research Awards program), Grain Research and Development Corporation (GRDC), Microsoft (Microsoft Azure for Research Award), the CSIRO (research gifts and PhD Students Top-up scholarships), the Australian Academy of Science (FASIC program), the European Science Foundation, and Neusoft Corporation.