To continuously improve quality and reflect changes in data, machine learning applications must regularly retrain and update their core models. This seminar will show that a differential analysis of language model snapshots before and after an update can reveal a surprising amount of detailed information about changes in the training data. To demonstrate the leakage due to updates of natural language models, we develop two metrics — differential score and differential rank. The extent of leakage analysis that is possible using these metrics across a range of models and datasets will be discussed as well as the privacy implications of the findings, mitigation strategies, and evaluation of their effect.
