Eugene is an Assistant Professor at UMass Amherst CICS. Eugene's work focuses on security and privacy in emerging AI-based systems under real-life conditions and attacks. He completed his PhD at Cornell Tech advised by Vitaly Shmatikov and Deborah Estrin. Eugene's research was recognized by Apple Scholars in AI/ML and Digital Life Initiative fellowships and Usenix Security Distinguished Paper Award. He received an engineering degree from Baumanka and worked at Cisco as a software engineer. Eugene has extensive industry experience (Cisco, Amazon, Apple) and spends part of his time as a Research Scientist at Google.

Security: He worked on backdoor attacks in federated learning and proposed new frameworks Backdoors101 and Mithridates, and a new attack on generative language models covered by VentureBeat and The Economist. Recent work includes studies on vulnerabilities in multi-modal systems: instruction injections, adversarial illusions and adding biases to text-to-image models.

Privacy: Eugene worked on the Air Gap privacy protection for LLM Agents and operationalizing Contextual Integrity. He worked on aspects of differential privacy including fairness trade-offs, applications to location heatmaps, and tokenization methods for private federated learning. Additionally he helped to build the Ancile system that enforces use-based privacy of user data.

Eugene grew up in Tashkent and plays water polo.

Announcement 1: I am looking for PhD students (apply) and post-docs to work on attacks on LLM agents and generative models. Please reach out over email!

Announcement 2: We will be holding a seminar CS 692PA on Privacy and Security for GenAI models, please sign up if you are interested.

Papers

• format_paintOperationalizing Contextual Integrity in Privacy-Conscious Assistants, Preprint'24
  Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi, Itay Yona, Ilia Shumailov, Aneesh Pappu, Chongyang Shi, Laura Weidinger, Robert Stanforth, Leonard Berrada, Pushmeet Kohli, Po-Sen Huang, Borja Balle

  Contextual Integrity can be applied to help personal assistants protect user privacy, we study how to instruct LLMs to follow CI.

  Work done at Googleandroid. [PDF]
• image_searchInjecting Bias in Text-To-Image Models via Composite-Trigger BackdoorsPreprint'24
  Ali Naseh, Jaechul Roh, Eugene Bagdasaryan, Amir Houmansadr

  Can text-to-image models be impacted by the adversary injected biases?

  [PDF]
• blur_onUnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI Preprint'24
  Ilia Shumailov, Jamie Hayes, Eleni Triantafillou, Guillermo Ortiz-Jimenez, Nicolas Papernot, Matthew Jagielski, Itay Yona, Heidi Howard, Eugene Bagdasaryan

  Unlearning large concepts can either be easily recovered or will impact model performance.

  Work done at Googleandroid. [PDF]
• integration_instructionsSoft Prompts Go Hard: Steering Visual Language Models with Hidden Meta-InstructionsPreprint'24
  Tingwei Zhang, Collin Zhang, John X. Morris, Eugene Bagdasaryan, Vitaly Shmatikov

  How an adversary can inject hidden instructions into a visual language model?

  [PDF]
• flutter_dashAir Gap: Protecting Privacy-Conscious Conversational AgentsCCS'24
  Eugene Bagdasaryan, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, Daniel Ramage

  Can you make an agent that knows when to share the user data depending on context? Can it be protected from adversaries trying to extract that data.

  Work done at Googleandroid. [PDF]
• scienceoffMithridates: Boosting Natural Resistance to Backdoor Learning CCS'24
  Eugene Bagdasaryan, Vitaly Shmatikov

  A novel audit method for poisoning attacks.

  [PDF] [Code]
• auto_fix_high emoji_events Adversarial Illusions in Multi-Modal Embeddings Usenix Security'24
  Tingwei Zhang, Rishi Jha, Eugene Bagdasaryan, Vitaly Shmatikov

  We propose adversarial alignment in cross-modal settings. Distinguished paper award.

  [PDF], [Code].
• forum(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs Preprint'23
  Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, Vitaly Shmatikov

  A new prompt injection attack that makes LLMs talk like pirates when they see an image or an audio.

  [PDF] [Code]
• mapTowards Sparse Federated Analytics: Location Heatmaps under Distributed Differential Privacy with Secure Aggregation PETS'22
  Eugene Bagdasaryan, Peter Kairouz, Stefan Mellem, Adrià Gascón, Kallista Bonawitz, Deborah Estrin, and Marco Gruteser

  A new algorithm for building heatmaps with local-like differential privacy.

  Work done at Google. [PDF] [Updated Code]
• campaignSpinning Language Models: Risks of Propaganda-as-a-Service and Countermeasures S&P'22
  Eugene Bagdasaryan and Vitaly Shmatikov

  We discover new capabilities of large language models to express attacker-chosen opinions on certain topics while performing tasks like summarization, translation, and language generation.

  [PDF] [Code]
• spellcheckTraining a Tokenizer for Free with Private Federated Learning FL4NLP@ACL'22
  Eugene Bagdasaryan, Congzheng Song, Rogier van Dalen, Matt Seigel, and Áine Cahill

  Tokenization is an important part of training a good language model, however in private federated learning where user data are not available generic tokenization methods reduce performance. We show how to obtain a good tokenizer without spending additional privacy budget.

  Work done at Apple. Best paper runner-up award. [PDF]
• visibility_off Blind Backdoors in Deep Learning Models Usenix Security'21
  Eugene Bagdasaryan and Vitaly Shmatikov

  We propose a novel attack that injects complex and semantic backdoors without access to the training data or the model and evades all known defenses.

  [PDF] [Code]
• smartphoneHow To Backdoor Federated Learning AISTATS'20
  Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov

  We introduce a constrain-and-scale attack, a form of data poisoning, that can stealthily inject a backdoor into one of the participating models during a single round of Federated Learning training. This attack can avoid proposed defenses and propagate the backdoor to a global server that will distribute the compromised model to other participants.

  [PDF] [Code]
• local_hospitalSalvaging Federated Learning by Local Adaptation Preprint
  Tao Yu, Eugene Bagdasaryan, and Vitaly Shmatikov

  Recovering participants' performance on their data when using federated learning with robustness and privacy techniques.

  [Paper] [Code]
• doneAncile: Enhancing Privacy for Ubiquitous Computing with Use-Based Privacy WPES'19
  Eugene Bagdasaryan, Griffin Berlstein, Jason Waterman, Eleanor Birrell, Nate Foster, Fred B. Schneider, and Deborah Estrin

  A novel platform that enables control over application's data usage with language level policies and implementing use-based privacy.

  [PDF] [Code] [Slides]
• faceDifferential Privacy Has Disparate Impact on Model Accuracy NeurIPS'19
  Eugene Bagdasaryan and Vitaly Shmatikov

  This project discusses a new trade off between privacy and fairness. We observe that training a Machine Learning model with Differential Privacy reduces accuracy on underrepresented groups.

  [NeurIPS, 2019], [Code].
• memoryX-containers: Breaking down barriers to improve performance and isolation of cloud-native containers ASPLOS'19
  Zhiming Shen, Zhen Sun, Gur-Eyal Sela, Eugene Bagdasaryan, Christina Delimitrou, Robbert Van Renesse, and Hakim Weatherspoon

  A fast and compact cloud-native implementation of containers.

  [PDF].
• extensionOpenrec: A modular framework for extensible and adaptable recommendation algorithms WSDM'18
  Longqi Yang, Eugene Bagdasaryan, Joshua Gruenstein, Cheng-Kang Hsieh, and Deborah Estrin

  An open and modular Python framework that supports extensible and adaptable research in recommender systems.

  [PDF], [Code].