About

Svetlana Lazebnik is a Professor and Willett Faculty Scholar at the Siebel School of Computing and Data Science at the University of Illinois Urbana-Champaign. Her research areas include Artificial Intelligence, with recent courses taught related to Deep Learning for Computer Vision and introductory deep learning courses. She is recognized for her contributions to computer vision, as evidenced by her honors such as being named a University Scholar and her recognition as a distinguished speaker discussing generative image models for virtual try-on and stylization. Lazebnik's work focuses on advancing understanding and applications in computer vision, contributing to the academic community through teaching and research.

Research topics

• Natural Language Processing
• Artificial Intelligence
• Computer Science
• Theoretical computer science

Selected publications

• An Automatic and Quantitative Post-Earthquake Rapid Building Assessment Framework from Exterior Drift-Sensitive Nonstructural Components using Instance Segmentation

  SSRN Electronic Journal · 2026-01-01

  preprintOpen access
  PublisherDOI

• CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance

  ArXiv.org · 2026-03-12

  articleOpen access

  We introduce CRAFT hand, a tendon-driven anthropomorphic hand with hybrid hard-soft compliance for contact-rich manipulation. The design is based on a simple idea: contact is not uniform across the hand. Impacts concentrate at joints, while links carry most of the load. CRAFT places soft material at joints and keeps links rigid, and uses rollingcontact joint surfaces to keep flexion on repeatable motion paths. Fifteen motors mounted on the fingers drive the hand through tendons, keeping the form factor compact and the fingers light. In structural tests, CRAFT improves strength and endurance while maintaining comparable repeatability. In teleoperation, CRAFT improves handling of fragile and low-friction items, and the hand covers 33/33 grasps in the Feix taxonomy. The full design costs under $600 and will be released open-source with visionbased teleoperation and simulation integration. Project page: http://craft-hand.github.io/

  PublisherOA PDF

• CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance

  arXiv (Cornell University) · 2026-03-12

  preprintOpen access

  We introduce CRAFT hand, a tendon-driven anthropomorphic hand with hybrid hard-soft compliance for contact-rich manipulation. The design is based on a simple idea: contact is not uniform across the hand. Impacts concentrate at joints, while links carry most of the load. CRAFT places soft material at joints and keeps links rigid, and uses rollingcontact joint surfaces to keep flexion on repeatable motion paths. Fifteen motors mounted on the fingers drive the hand through tendons, keeping the form factor compact and the fingers light. In structural tests, CRAFT improves strength and endurance while maintaining comparable repeatability. In teleoperation, CRAFT improves handling of fragile and low-friction items, and the hand covers 33/33 grasps in the Feix taxonomy. The full design costs under $600 and will be released open-source with visionbased teleoperation and simulation integration. Project page: http://craft-hand.github.io/

  PublisherDOI

• Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping

  arXiv (Cornell University) · 2025-10-10

  preprintOpen access

  Multimodal large language models (MLLMs) often miss small details and spatial relations in cluttered scenes, leading to errors in fine-grained perceptual grounding. We introduce AttWarp, a lightweight method that allocates more resolution to query-relevant content while compressing less informative areas, all while preserving global context. At test time, the approach uses an MLLM's cross-modal attention to perform rectilinear warping of the input image, reallocating spatial resolution toward regions the model deems important, without changing model weights or architecture. This attention-guided warping preserves all original image information but redistributes it non-uniformly, so small objects and subtle relationships become easier for the same model to read while the global layout remains intact. Across five benchmarks (TextVQA, GQA, DocVQA, POPE, MMMU) and four MLLMs (LLaVA, Qwen-VL, InternVL, and InstructBLIP), AttWarp consistently improves accuracy, strengthens compositional reasoning, and reduces hallucinations, outperforming four competitive baselines that manipulate raw images at test time. Together, these results show that attention-guided warping prioritizes information relevant to the query while preserving context, and that the same MLLMs perform better when given such warped inputs.

  PublisherOA PDFDOI

• A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

  2025-05-19 · 8 citations

  article

  Task specification for robotic manipulation in open-world environments is challenging, requiring flexible and adaptive objectives that align with human intentions and can evolve through iterative feedback. We introduce Iterative Keypoint Reward (IKER), a visually grounded, Python-based reward function that serves as a dynamic task specification. Our framework leverages VLMs to generate and refine these reward functions for multi-step manipulation tasks. Given RGB-D observations and free-form language instructions, we sample keypoints in the scene and generate a reward function conditioned on these keypoints. IKER operates on the spatial relationships between keypoints, leveraging commonsense priors about the desired behaviors, and enabling precise SE(3) control. We reconstruct real-world scenes in simulation and use the generated rewards to train reinforcement learning (RL) policies, which are then deployed into the real world-forming a real-to-sim-to-real loop. Our approach demonstrates notable capabilities across diverse scenarios, including both prehensile and non-prehensile tasks, showcasing multi-step task execution, spontaneous error recovery, and on-the-fly strategy adjustments. The results highlight IKER's effectiveness in enabling robots to perform multi-step tasks in dynamic environments through iterative reward shaping. Project Page: https://iker-robot.github.io/

  PublisherDOI

• A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

  ArXiv.org · 2025-02-12

  preprintOpen access

  Task specification for robotic manipulation in open-world environments is challenging, requiring flexible and adaptive objectives that align with human intentions and can evolve through iterative feedback. We introduce Iterative Keypoint Reward (IKER), a visually grounded, Python-based reward function that serves as a dynamic task specification. Our framework leverages VLMs to generate and refine these reward functions for multi-step manipulation tasks. Given RGB-D observations and free-form language instructions, we sample keypoints in the scene and generate a reward function conditioned on these keypoints. IKER operates on the spatial relationships between keypoints, leveraging commonsense priors about the desired behaviors, and enabling precise SE(3) control. We reconstruct real-world scenes in simulation and use the generated rewards to train reinforcement learning (RL) policies, which are then deployed into the real world-forming a real-to-sim-to-real loop. Our approach demonstrates notable capabilities across diverse scenarios, including both prehensile and non-prehensile tasks, showcasing multi-step task execution, spontaneous error recovery, and on-the-fly strategy adjustments. The results highlight IKER's effectiveness in enabling robots to perform multi-step tasks in dynamic environments through iterative reward shaping.

  PublisherOA PDFDOI

• UnZipLoRA: Separating Content and Style from a Single Image

  2025-10-19

  preprintOpen accessSenior author

  This paper introduces UnZipLoRA, a method for decomposing an image into its constituent subject and style, represented as two distinct LoRAs (Low-Rank Adaptations). Unlike existing personalization techniques that focus on either subject or style in isolation, or require separate training sets for each, UnZipLoRA disentangles these elements from a single image by training both the LoRAs simultaneously. UnZipLoRA ensures that the resulting LoRAs are compatible, i.e., they can be seamlessly combined using direct addition. UnZipLoRA enables independent manipulation and recontextualization of subject and style, including generating variations of each, applying the extracted style to new subjects, and recombining them to reconstruct the original image or create novel variations. To address the challenge of subject and style entanglement, UnZipLoRA employs a novel prompt separation technique, as well as column and block separation strategies to accurately preserve the characteristics of subject and style, and ensure compatibility between the learned LoRAs. Evaluation with human studies and quantitative metrics demonstrates UnZipLoRA's effectiveness compared to other state-of-the-art methods, including DreamBooth-LoRA, Inspiration Tree, and B-LoRA.

  PublisherOA PDFDOI

• Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images

  2025-02-26 · 9 citations

  articleSenior author

  Most virtual try-on research is motivated to serve the fashion business by generating images to demonstrate garments on studio models at a lower cost. However, virtual try-on should be a broader application that also allows customers to visualize garments on themselves using their own casual photos, known as in-the-wild try-on. Unfortunately, the existing methods, which achieve plausible results for studio try-on settings, perform poorly in the in-the-wild context. This is because these methods often require paired images (garment images paired with images of people wearing the same garment) for training. While such paired data is easy to collect from shopping websites for studio settings, it is difficult to obtain for in-the-wild scenes. In this work, we fill the gap by (1) introducing a Street-TryOn benchmark to support in-the-wild virtual try-on applications and (2) proposing a novel method to learn virtual try-on from a set of in-the-wild person images directly without requiring paired data. We tackle the unique challenges, including warping garments to more diverse human poses and rendering more complex backgrounds faithfully, by a novel DensePose warping correction method combined with diffusion-based conditional inpainting. Our experiments show competitive performance for standard studio try-on tasks and SOTA performance for street try-on and cross-domain try-on tasks.

  PublisherDOI

• Combining Multiple Cues for Visual Madlibs Question Answering

  UNC Libraries · 2024-08-14

  articleOpen accessSenior author
  PublisherDOI

• In Memoriam: Xiaoou Tang

  International Journal of Computer Vision · 2024-05-16

  articleOpen access
  PublisherOA PDFDOI

Recent grants

• CAREER: Similarity-based Representation of Large-scale Image Collections

  NSF · $372k · 2012–2015

• RI: Medium: Collaborative Research: Text-to-Image Reference Resolution for Image Understanding and Manipulation

  NSF · $550k · 2016–2021

Frequent coauthors

• Cordelia Schmid

  40 shared

• Jean Ponce

  Département d'Informatique

  31 shared

• Arun Mallya

  Nvidia (United States)

  25 shared

• Alexander G. Schwing

  24 shared

• Bryan A. Plummer

  20 shared

• C. Schmid

  18 shared

• Unnat Jain

  Carnegie Mellon University

  18 shared

• Andrew Fitzgibbon

  16 shared

Labs

• Siebel School of Computing and Data SciencePI

Education

• Ph.D., Computer Science

  University of California, Berkeley

  1999

• M.S., Computer Science

  University of California, Berkeley

  1994

• B.S., Computer Science

  University of Illinois at Urbana-Champaign

  1991

Awards & honors

• University Scholar Honor