About

Steve Marschner is a Professor in the Computer Science Department at Cornell University, with affiliations in the Field of Applied Mathematics and the Field of Electrical and Computer Engineering. He also serves as the Associate Dean for Research at the Bowers College of Computing and Information Science. Marschner earned his Ph.D. in Computer Science from Cornell University in 1998, with a thesis focused on Inverse Rendering for Computer Graphics, under the advisement of Donald P. Greenberg, Charles Van Loan, and Leonard Gross. He holds a Sc.B. in Mathematics-Computer Science from Brown University, completed in 1993. Throughout his career, Marschner has held various academic and research positions, including roles as Assistant Professor, Associate Professor, and Professor at Cornell University, as well as visiting and guest professorships at institutions such as ETH Zürich. He has also worked in industry research positions, including at Microsoft Research and NVIDIA Research, where he is currently a Distinguished Graphics Researcher. His research contributions are centered on computer graphics, particularly in rendering techniques, appearance modeling, and the simulation of materials and light transport. His work encompasses areas such as wave optics, differentiable rendering, and the modeling of complex materials like cloth, feathers, and woven textiles. Marschner's extensive publication record reflects his focus on advancing the understanding and practical application of rendering and material appearance in computer graphics. His research integrates mathematical and computational methods to address challenges in realistic image synthesis and the physical simulation of materials, contributing significantly to both academic knowledge and technological innovation in graphics and visualization.

Research topics

• Computer Science
• Artificial Intelligence
• Computer graphics (images)
• Mechanical engineering
• Engineering
• Engineering drawing
• Programming language
• Computer vision
• Physics
• Optics
• Algorithm
• Mathematics
• Materials science

Selected publications

• ArchSym: Detecting 3D-Grounded Architectural Symmetries in the Wild

  ArXiv.org · 2026-04-24

  articleOpen access

  Symmetry detection is a fundamental problem in computer vision, and symmetries serve as powerful priors for downstream tasks. However, existing learning-based methods for detecting 3D symmetries from single images have been almost exclusively trained and evaluated on object-centric or synthetic datasets, and thus fail to generalize to real-world scenes. Furthermore, due to the inherent scale ambiguity of monocular inputs, which makes localizing the 3D plane an ill-posed problem, many existing works only predict the plane's orientation. In this paper, we address these limitations by presenting the first framework for detecting 3D-grounded reflectional symmetries from single, in-the-wild RGB images, focusing on architectural landmarks. We introduce two key innovations: (1) a scalable data annotation pipeline to automatically curate a large-scale dataset of architectural symmetries, ArchSym, from SfM reconstructions by leveraging cross-view image matching; and building on the dataset, (2) a single-view symmetry detector that accurately localizes symmetries in 3D by parameterizing them as signed distance maps defined relative to predicted scene geometry. We validate our symmetry annotation pipeline against geometry-based alternatives and demonstrate that our symmetry detector significantly outperforms state-of-the-art baselines on our new benchmark.

  PublisherOA PDF

• ArchSym: Detecting 3D-Grounded Architectural Symmetries in the Wild

  arXiv (Cornell University) · 2026-04-24

  preprintOpen access

  Symmetry detection is a fundamental problem in computer vision, and symmetries serve as powerful priors for downstream tasks. However, existing learning-based methods for detecting 3D symmetries from single images have been almost exclusively trained and evaluated on object-centric or synthetic datasets, and thus fail to generalize to real-world scenes. Furthermore, due to the inherent scale ambiguity of monocular inputs, which makes localizing the 3D plane an ill-posed problem, many existing works only predict the plane's orientation. In this paper, we address these limitations by presenting the first framework for detecting 3D-grounded reflectional symmetries from single, in-the-wild RGB images, focusing on architectural landmarks. We introduce two key innovations: (1) a scalable data annotation pipeline to automatically curate a large-scale dataset of architectural symmetries, ArchSym, from SfM reconstructions by leveraging cross-view image matching; and building on the dataset, (2) a single-view symmetry detector that accurately localizes symmetries in 3D by parameterizing them as signed distance maps defined relative to predicted scene geometry. We validate our symmetry annotation pipeline against geometry-based alternatives and demonstrate that our symmetry detector significantly outperforms state-of-the-art baselines on our new benchmark.

  PublisherDOI

• Seeing Fast and Slow: Learning the Flow of Time in Videos

  arXiv (Cornell University) · 2026-04-23

  articleOpen access

  How can we tell whether a video has been sped up or slowed down? How can we generate videos at different speeds? Although videos have been central to modern computer vision research, little attention has been paid to perceiving and controlling the passage of time. In this paper, we study time as a learnable visual concept and develop models for reasoning about and manipulating the flow of time in videos. We first exploit the multimodal cues and temporal structure naturally present in videos to learn, in a self-supervised manner, to detect speed changes and estimate playback speed. We then show that these learned temporal reasoning models enable us to curate the largest slow-motion video dataset to date from noisy in-the-wild sources. Such slow-motion footage, typically filmed by high-speed cameras, contains substantially richer temporal detail than standard videos. Using this data, we further develop models capable of temporal control, including speed-conditioned video generation, which produces motion at specified playback speed, and temporal super-resolution, which tranforms low-FPS, blurry videos into high-FPS sequences with fine-grained temporal details. Our findings highlight time as a manipulable, perceptual dimension in video learning, opening doors to temporally controllable video generation, temporal forensics detection, and potentially richer world-models that understand how events unfold over time.

  PublisherOA PDF

• Seeing Fast and Slow: Learning the Flow of Time in Videos

  arXiv (Cornell University) · 2026-04-23

  preprintOpen access

  How can we tell whether a video has been sped up or slowed down? How can we generate videos at different speeds? Although videos have been central to modern computer vision research, little attention has been paid to perceiving and controlling the passage of time. In this paper, we study time as a learnable visual concept and develop models for reasoning about and manipulating the flow of time in videos. We first exploit the multimodal cues and temporal structure naturally present in videos to learn, in a self-supervised manner, to detect speed changes and estimate playback speed. We then show that these learned temporal reasoning models enable us to curate the largest slow-motion video dataset to date from noisy in-the-wild sources. Such slow-motion footage, typically filmed by high-speed cameras, contains substantially richer temporal detail than standard videos. Using this data, we further develop models capable of temporal control, including speed-conditioned video generation, which produces motion at specified playback speed, and temporal super-resolution, which tranforms low-FPS, blurry videos into high-FPS sequences with fine-grained temporal details. Our findings highlight time as a manipulable, perceptual dimension in video learning, opening doors to temporally controllable video generation, temporal forensics detection, and potentially richer world-models that understand how events unfold over time.

  PublisherDOI

• Fiber-level Woven Fabric Capture from a Single Microscopic Image

  ACM Transactions on Graphics · 2026-05-18

  article

  Accurately rendering the appearance of fabrics is challenging, due to their complex 3D microstructures and specialized optical properties. If we model the geometry and optics of fabrics down to the fiber level, we can achieve unprecedented rendering realism, but this raises the difficulty of authoring the fiber-level assets. Existing approaches can obtain fiber-level geometry with special devices (e.g., CT) or hand-designed procedural pipelines. In this paper, we propose a method to capture fiber-level geometry and appearance of woven fabrics using a single low-cost microscope image. This may seem like an impossible task: a single image from a low-cost microscope looks very different from the final rendering we would like to achieve, and the information contained in it may seem minimal. We propose a novel fiber parameter estimation pipeline in a coarse-to-fine manner, establishing a subset of parameters step by step. At the core of our pipeline are differentiable procedural geometric and appearance models for woven fabrics at the fiber level, enabling both geometry and appearance to be optimized simultaneously. We first use a simple neural network to predict initial parameters, then we optimize the parameters of procedural fiber geometry and an approximated shading model via differentiable rasterization to match the microscope photo more accurately. Finally, we refine the fiber appearance parameters via differentiable path tracing, converging to accurate fiber optical parameters, which are suitable for physically-based light simulations to produce high-quality rendered results. We believe that our method is the first to utilize differentiable rendering at the microscopic level, supporting physically-based scattering from explicit fiber assemblies. Our fabric parameter estimation achieves high-quality re-rendering of measured woven fabric samples in both distant and close-up views. We also propose a patch-space fiber geometry procedural generation method and a two-scale path tracing framework for efficient rendering of fabric scenes.

  PublisherDOI

• HairFormer: Transformer-Based Dynamic Neural Hair Simulation

  ArXiv.org · 2025-07-16

  preprintOpen accessSenior author

  Simulating hair dynamics that generalize across arbitrary hairstyles, body shapes, and motions is a critical challenge. Our novel two-stage neural solution is the first to leverage Transformer-based architectures for such a broad generalization. We propose a Transformer-powered static network that predicts static draped shapes for any hairstyle, effectively resolving hair-body penetrations and preserving hair fidelity. Subsequently, a dynamic network with a novel cross-attention mechanism fuses static hair features with kinematic input to generate expressive dynamics and complex secondary motions. This dynamic network also allows for efficient fine-tuning of challenging motion sequences, such as abrupt head movements. Our method offers real-time inference for both static single-frame drapes and dynamic drapes over pose sequences. Our method demonstrates high-fidelity and generalizable dynamic hair across various styles, guided by physics-informed losses, and can resolve penetrations even for complex, unseen long hairstyles, highlighting its broad generalization.

  PublisherOA PDFDOI

• Quadric-Based Silhouette Sampling for Differentiable Rendering

  ACM Transactions on Graphics · 2025-07-27

  articleSenior author

  Physically based differentiable rendering has established itself as key to inverse rendering, in which scenes are recovered from images through gradient-based optimization. Taking the derivative of the rendering equation is made difficult by the presence of discontinuities in the integrand at object silhouettes. To obtain correct derivatives w.r.t. changing geometry, accounting e.g. for changing penumbras or silhouettes in glossy reflections, differentiable renderers must compute an integral over these silhouettes. Prior work proposed importance sampling of silhouette edges for a given shading point. The main challenge is to efficiently reject parts of the mesh without silhouettes during sampling, which has been done using top-down traversal of a tree. Inaccuracies of this existing rejection procedure result in many samples with zero contribution. Thus, variance remains high and subsequent work has focused on alternatives such as area sampling or path space differentiable rendering. We propose an improved rejection test. It reduces variance substantially, which makes edge sampling in a unidirectional path tracer competitive again. Our rejection test relies on two approximations to the triangle planes of a mesh patch: A bounding box in dual space and dual quadrics. Additionally, we improve the heuristics used for stochastic traversal of the tree. We evaluate our method in a unidirectional path tracer and achieve drastic improvements over the original edge sampling and outperform methods based on area sampling.

  PublisherDOI

• ObjectCarver: Semi-Automatic Segmentation, Reconstruction and Separation of 3D Objects

  2025-03-25

  article

  Implicit neural fields have made remarkable progress in reconstructing 3D surfaces from multiple images; however, they encounter challenges when it comes to separating individual objects within a scene. Previous approaches to this problem require ground-truth segmentation masks and introduce floating artifacts in occluded parts of the scene. We address these challenges with ObjectCarver. Object-Carver requires no ground-truth segmentation; all it needs is just a few user clicks in a single view. ObjectCarver also introduces a new loss function that prevents floaters and avoids inappropriate carving-out due to occlusion. Finally, ObjectCarver uses a simple initialization technique that significantly speeds up the process while preserving geometric details. We demonstrate qualitatively and quantitatively on multiple datasets (including a new dataset and benchmark with complete ground-truth) that ObjectCarver produces more accurate reconstructions of each object while minimizing artifacts.

  PublisherDOI

• Synergistic diffraction and multibounce reflection interference within single iridescent microstructures

  Newton · 2025-04-15

  preprintOpen access

  <h2>Summary</h2> Multibounce reflection interference produces structural coloration due to interference between light rays that travel along different paths of total internal reflection in microscale cavities. As feature dimensions decrease, a ray-based framework may become inaccurate as contributions from wave-based phenomena, such as diffraction, become increasingly important. We employ Fourier plane microscopy to collect angle-resolved interference spectra from individual 10-μm-scale microstructures and reveal how edge diffraction, synergistically coupled with multibounce total internal reflection, contributes to far-field interference. Hemicylinders, particularly those with lower contact angles, exhibit interference signatures that cannot be fully explained by the conventional multibounce ray-based framework but align with full-wave simulations. By collecting angularly resolved interference spectra from selected regions within isolated hemicylinders, we identify the contributions from light entering and exiting along different paths. We develop an extended ray model that incorporates edge diffraction and use experiments to guide estimation of the relative intensity contributions of diffracted ray trajectories.

  PublisherOA PDFDOI

• Accurate Differential Operators for Hybrid Neural Fields

  2025-06-10 · 1 citations

  article

  Neural fields have become widely used in various fields, from shape representation to neural rendering, and for solving partial differential equations (PDEs). With the advent of hybrid neural field representations like Instant NGP that leverage small MLPs and explicit representations, these models train quickly and can fit large scenes. Yet in many applications like rendering and simulation, hybrid neural fields can cause noticeable and unreasonable artifacts. This is because they do not yield accurate spatial derivatives needed for these downstream applications. In this work, we propose two ways to circumvent these challenges. Our first approach is a post hoc operator that uses local polynomial fitting to obtain more accurate derivatives from pre-trained hybrid neural fields. Additionally, we also propose a self-supervised fine-tuning approach that refines the hybrid neural field to yield accurate derivatives directly while preserving the initial signal. We show applications of our method to rendering, collision simulation, and solving PDEs. We observe that using our approach yields more accurate derivatives, reducing artifacts and leading to more accurate simulations in downstream applications.

  PublisherDOI

Recent grants

• CHS: Medium: Collaborative Research: Integrated Simulation of Cloth Mechanics and Appearance for Predictive Virtual Prototyping

  NSF · $966k · 2015–2020

• HCC: Large: Collaborative Research: Beyond Flat Images: Acquiring, Processing and Fabricating Visually Rich Material Appearance

  NSF · $1000k · 2010–2015

• CAREER: Modeling the Properties and Appearance of Materials

  NSF · $412k · 2004–2010

• Accurate and Efficient Visual Simulation of Fiber-based Mechanical Structures

  NSF · $314k · 2007–2011

• CHS: Medium: Collaborative Research: Fast Photorealistic Computer Graphics Rendering of Non-Smooth Surfaces

  NSF · $588k · 2017–2022

Frequent coauthors

• Peter Shirley

  36 shared

• Michael Ashikhmin

  21 shared

• Wenzel Jakob

  École Polytechnique Fédérale de Lausanne

  20 shared

• Markus Groß

  Walt Disney (Switzerland)

  20 shared

• Michael Gleicher

  University of Wisconsin–Madison

  19 shared

• Garrett Johnson

  19 shared

• Tamara Munzner

  19 shared

• Peter Willemsen

  University of Minnesota, Duluth

  19 shared

Awards & honors

• Sloan Research Fellowship
• ACM SIGGRAPH Academy
• ACM Fellow
• Academy of Motion Picture Arts and Sciences Technical Achiev…