Sanjeev Arora

Evaluating Gradient Inversion Attacks and Defenses in Federated Learning

arXiv (Cornell University) · 2021 · 123 citations

Gradient inversion attack (or input recovery from gradient) is an emerging threat to the security and privacy preservation of Federated learning, whereby malicious eavesdroppers or participants in the protocol can recover (partially) the clients' private data. This paper evaluates existing attacks and defenses. We find that some attacks make strong assumptions about the setup. Relaxing such assumptions can substantially weaken these attacks. We then evaluate the benefits of three proposed defens…

Advancing science- and evidence-based AI policy

Science · 2025-07-31 · 10 citations

Policy must be informed by, but also facilitate the generation of, scientific evidence.

ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty

arXiv (Cornell University) · 2024-08-26 · 1 citations

Compositionality is a critical capability in Text-to-Image (T2I) models, as it reflects their ability to understand and combine multiple concepts from text descriptions. Existing evaluations of compositional capability rely heavily on human-designed text prompts or fixed templates, limiting their diversity and complexity, and yielding low discriminative power. We propose ConceptMix, a scalable, controllable, and customizable benchmark which automatically evaluates compositional generation abilit…

Can Models Learn Skill Composition from Examples?

arXiv (Cornell University) · 2024-09-29 · 1 citations

As large language models (LLMs) become increasingly advanced, their ability to exhibit compositional generalization -- the capacity to combine learned skills in novel ways not encountered during training -- has garnered significant attention. This type of generalization, particularly in scenarios beyond training data, is also of great interest in the study of AI safety and alignment. A recent study introduced the SKILL-MIX evaluation, where models are tasked with composing a short paragraph demo…

Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision

arXiv (Cornell University) · 2026-04-13

Current post-training methods in verifiable settings fall into two categories. Reinforcement learning (RLVR) relies on binary rewards, which are broadly applicable and powerful, but provide only sparse supervision during training. Distillation provides dense token-level supervision, typically obtained from an external teacher or using high-quality demonstrations. Collecting such supervision can be costly or unavailable. We propose Self-Distillation Zero (SD-Zero), a method that is substantially…