Yike Wang

The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games

arXiv (Cornell University) · 2021 · 591 citations

Proximal Policy Optimization (PPO) is a ubiquitous on-policy reinforcement learning algorithm but is significantly less utilized than off-policy learning algorithms in multi-agent settings. This is often due to the belief that PPO is significantly less sample efficient than off-policy methods in multi-agent systems. In this work, we carefully study the performance of PPO in cooperative multi-agent settings. We show that PPO-based multi-agent algorithms achieve surprisingly strong performance in…

Control Synthesis from Linear Temporal Logic Specifications using\n Model-Free Reinforcement Learning

2019 · 95 citations

We present a reinforcement learning (RL) framework to synthesize a control\npolicy from a given linear temporal logic (LTL) specification in an unknown\nstochastic environment that can be modeled as a Markov Decision Process (MDP).\nSpecifically, we learn a policy that maximizes the probability of satisfying\nthe LTL formula without learning the transition probabilities. We introduce a\nnovel rewarding and path-dependent discounting mechanism based on the LTL\nformula such that (i) an optimal po…

Adversarial Training for Large Neural Language Models

arXiv (Cornell University) · 2020 · 91 citations

Generalization and robustness are both key desiderata for designing machine learning methods. Adversarial training can enhance robustness, but past work often finds it hurts generalization. In natural language processing (NLP), pre-training large neural language models such as BERT have demonstrated impressive gain in generalization for a variety of tasks, with further improvement from adversarial fine-tuning. However, these models are still vulnerable to adversarial attacks. In this paper, we s…

Delayed Interactions in Active Agents: Stability and Formations

ArXiv.org · 2025-08-30

Active agents with time-delayed interactions arise naturally in various real-world systems, such as biological systems, transportation networks and robotic swarms. Such systems are typically modeled as Delay Differential Equations (DDEs) that incorporate inertial effects. In this paper, we investigate the stability of pattern formation of active agents with inertia and time delays, in both uncoupled and coupled scenarios. We derive and analyze a high-dimensional linear DDE model that characteriz…

Control Synthesis in Partially Observable Environments for Complex Perception-Related Objectives

ArXiv.org · 2025-06-27

Perception-related tasks often arise in autonomous systems operating under partial observability. This work studies the problem of synthesizing optimal policies for complex perception-related objectives in environments modeled by partially observable Markov decision processes. To formally specify such objectives, we introduce \emph{co-safe linear inequality temporal logic} (sc-iLTL), which can define complex tasks that are formed by the logical concatenation of atomic propositions as linear ineq…