Lerrel Pinto

About

Lerrel Pinto is a roboticist at Meta Superintelligence Labs (MSL), where he leads efforts in building frontier robotics models to achieve personal superintelligence in the physical world. He is also the co-founder and CEO of Assured Robot Intelligence (ARI), which was acquired by Meta in May 2026. Previously, he co-founded Fauna Robotics, acquired by Amazon. In academia, he runs the General Robotics & AI Lab (GRAIL) at NYU, focusing on robot learning, sensory representation learning, action and behavior modeling, reinforcement learning for adaptation, and the development of affordable, open-source robots. His work has been recognized with numerous awards including the Samsung AI researcher of the year award, Sloan Fellowship, Packard Fellowship, CIFAR Fellowship, TR35, IEEE RAS Early Career, and NSF CAREER awards.

Research topics

• Computer Science
• Artificial Intelligence
• Machine Learning
• Mathematics
• Human–computer interaction
• Programming language
• Computer vision
• Computer graphics (images)
• Algorithm

Selected publications

• Reinforcement Learning with Augmented Data

  arXiv (Cornell University) · 2020 · 246 citations

  Learning from visual observations is a fundamental yet challenging problem in Reinforcement Learning (RL). Although algorithmic advances combined with convolutional neural networks have proved to be a recipe for success, current methods are still lacking on two fronts: (a) data-efficiency of learning and (b) generalization to new environments. To this end, we present Reinforcement Learning with Augmented Data (RAD), a simple plug-and-play module that can enhance most RL algorithms. We perform th…

  DOI

• Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation

  arXiv (Cornell University) · 2020 · 19 citations

  Senior authorCorresponding

  Vision-based robotics often separates the control loop into one module for perception and a separate module for control. It is possible to train the whole system end-to-end (e.g. with deep RL), but doing it "from scratch" comes with a high sample complexity cost and the final result is often brittle, failing unexpectedly if the test environment differs from that of training. We study the effects of using mid-level visual representations (features learned asynchronously for traditional computer v…

  DOI

• DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning

  arXiv (Cornell University) · 2024-11-07 · 5 citations

  preprintOpen accessSenior author

  The ability to predict future outcomes given control actions is fundamental for physical reasoning. However, such predictive models, often called world models, remains challenging to learn and are typically developed for task-specific solutions with online policy learning. To unlock world models' true potential, we argue that they should 1) be trainable on offline, pre-collected trajectories, 2) support test-time behavior optimization, and 3) facilitate task-agnostic reasoning. To this end, we p…

  PublisherOA PDFDOI

• Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments

  2025-05-19 · 4 citations

  article

  Robot models, particularly those trained with large amounts of data, have recently shown a plethora of real-world manipulation and navigation capabilities. Several independent efforts have shown that given sufficient training data in an environment, robot policies can generalize to demonstrated variations in that environment. However, needing to finetune robot models to every new environment stands in stark contrast to models in language or vision that can be deployed zero-shot for open-world pr…

  PublisherDOI

• P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies

  2025-05-19 · 3 citations

  article

  Developing generalizable robot policies that can robustly handle varied environmental conditions and object instances remains a fundamental challenge in robot learning. While considerable efforts have focused on collecting large robot datasets and developing policy architectures to learn from such data, naïvely learning from visual inputs often results in brittle policies that fail to transfer beyond the training data. This work presents Prescriptive Point Priors for Policies or P3-PO, a novel f…

  PublisherDOI

Frequent coauthors

• Pieter Abbeel

  University of California, Berkeley

  43 shared

• Abhinav Gupta

  21 shared

• Abhinav Gupta

  21 shared

• Dhiraj Gandhi

  Meta (Israel)

  18 shared

• Nur Muhammad Mahi Shafiullah

  16 shared

• Soumith Chintala

  11 shared

• Siddhant Haldar

  11 shared

• Jyothish Pari

  9 shared

Education

• Ph.D., Robotics

  Unknown

• M.S., Robotics

  Unknown

• B.S., Robotics

  Unknown

Similar researchers at New York University

• Jiawei Zhangh-76
• Foster Provosth-60
• Panagiotis Ipeirotish-50