Yangsibo Huang

yangsibo [at] princeton (dot) edu

I’m a research scientist at Google NYC. Before that, I was a Ph.D. student and Wallace Memorial Fellow at Princeton University, co-advised by Prof. Kai Li and Prof. Sanjeev Arora.

I work at the intersection of machine learning, systems, and policy. My research explores how and why machine learning systems may go wrong, through the lenses of privacy and copyright violation, as well as security and safety concerns. Recently, I am also interested in model memorization and its implications on capabilities.

✰ Awards

• 🏆 The best paper award at the Secure and Trustworthy LLMs at ICLR’24 (for the work Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications that I co-lead)
• 🏅 Rising Star in EECS in 2023
• 🏅 Wallace Memorial Fellowship (2023-2024)
• 📃 Winner of Meta’s Privacy-Enhancing Technologies call for proposals in 2022

🎙 Recent Talks

• Open Technical Questions in GenAI Copyright
  • 08/08/2024, Open AGI Symposium @ Columbia University
  • 10/16/2024, Workshop on Alignment, Trust, Watermarking, and Copyright Issues in LLMs @ Simons Institute
• Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy
  • 09/24/2024, Unlearning Society @ Google DeepMind
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning
  • 08/18/2024, PPML Workshop @ Crypto 2024
• Identifying, Understanding, and Mitigating Failure Modes of Safety Alignment in Large Language Models
  • 04/24/2024, VITA-group Talk Series @ UT Austin
  • 04/30/2024, ML Series @ Microsoft Research, New England
• Detecting Pretraining Data from Large Language Models
  • 12/15/2023, Workshop on Regulatable ML @ NeurIPS 2023

🗞️ News

• 📊 [09/2024] We released ConceptMix, new benchmark that evaluates how well text-to-image models can generate images that accurately combine multiple visual concepts. Interestingly, we found that image generation models struggle with combining >3 visual concepts (e.g., “red,” “fluffy,” “squared,” “smartphone”) & we attribute this to their training data.
• 📃 [08/2024] Collecting, using, and sharing human feedback on models brings up new privacy and copyright concerns. We discuss these issues, along with others, in this recent work: The Future of Open Human Feedback.
• 📊 [07/2024] We released a new LLM safety benchmark, SORRY-Bench, designed to systematically evaluate how well LLMs refuse unsafe requests across 45 fine-grained harmful categories.
• 🐱 [06/2024] How easy is it for current image/video generation systems to output copyrighted characters like Mario and Batman (which leads to legal risks)? How can we prevent them from doing so? Check out our CopyCat!
• 📃 [03/2024] We release an open letter advocating for A Safe Harbor for AI Evaluation and Red Teaming. It has been signed by 300+ researchers and reported by The Washington Post, VentureBeat, AIPwn, and Computerworld.
• 📙 [01/2024] Our white paper on advancing Differential Privacy’s deployment in real-world applications got accepted by Harvard Data Science Review.

✎ Selected Publications

Please refer to full publications or my Google Scholar profile for the full list. “(α)” stands for alphabetical order (with “⁺” standing for lead), “*” stands for equal contribution.

㋡ Experiences

• [10/22-12/23] Google AI, Research Intern, hosted by Chiyuan Zhang and Badih Ghazi
• [05/22-09/22] Meta AI, Research Intern, hosted by Seyi Feyisetan and Alexandre Sablayrolles
• [09/18-04/19] Harvard Medical School, Research Intern, supervised by Prof. Quanzheng Li
• [06/18-08/18] David Geffen School of Medicine at UCLA, Research Intern, supervised by Prof. Ke Sheng

♥ Service

• Area Chair for
  • Theoretical Foundations of Foundation Models (co-located with ICML 2024)
• Program Committee member for
  • Generative AI + Law (GenLaw) ’24 (co-located with ICML 2024)
  • Privacy Regulation and Protection in Machine Learning (co-located with ICLR 2024)
  • Federated Learning and Analytics in Practice (co-located with ICML 2023)
  • Interpretable Machine Learning in Healthcare (co-located with ICML 2021, 2022)
  • Computer Vision for Automated Medical Diagnosis (co-located with ICCV 2021)
• Reviewer for ICML (2022, 2023, 2024), NeurIPS (2021, 2022, 2023), COLM (2024)

ꐕ MISC

• In my spare time, I mainly stay with my four cats 😺😻😼😽.
• I enjoy reading books about psychology.