About

The research in our lab uses advanced data science techniques to understand how water, plants, geology and climate interact in a tightly coupled system – and how humans are changing this system.

Research topics

• Computer Science
• Human–computer interaction
• Art
• Artificial Intelligence
• Linguistics
• Engineering
• Neuroscience
• Computer graphics (images)
• Communication
• Telecommunications
• Cognitive psychology
• Cognitive science
• Architectural engineering
• Aesthetics
• Multimedia
• Philosophy
• Programming language
• Psychology
• Distributed computing
• Visual arts
• Geography

Selected publications

• Parsing the Senses to Explore Complex Systems

  2025-04-07

  book-chapter1st authorCorresponding

  We live and navigate in a complex system. The earth is made up of a myriad of complex systems, the biosphere, atmosphere, and lithosphere, as well as complex systems generated by humans’ interactions with the planet – socio and economic complex systems. We understand the world through our sensory systems. We have intuitively learned how to navigate through these systems by parsing our senses to understand and navigate through this world. We see and feel the sun, but typically do not hear or smell the sun. We feel and hear the wind, but typically do not see the wind directly, but perceive it through our vision with the movement of trees and other objects. Using the computational platform, the senses can be mapped to certain frequencies. Can we systematically parse all our senses through frequency relationships that will allow us to explore very complex systems by seeing, hearing, and interacting with the data? This chapter will explore these questions by using the computational platform to understand and interact with complex systems ranging from the atomic to the galactic. Creating new worlds through immersive, interactive installations using the combination of advanced science research and artistic practice will lead to compelling new art forms and may lead to new scientific discoveries.

  PublisherDOI

• Spatial Orchestra: Locomotion Music Instruments through Spatial Exploration

  2024-05-11 · 6 citations

  articleOpen access

  Spatial Orchestra demonstrates how easy it is to play musical instruments using basic input like natural locomotion, which is accessible to most. Unlike many musical instruments, our work allows individuals of all skill levels to effortlessly create music by walking into virtual bubbles. Our Augmented Reality experience involves interacting with ever-shifting sound bubbles that the user engages with by stepping into color-coded bubbles within the assigned area using a standalone AR headset. Each bubble corresponds to a cello note, and omits sound from the center of the bubble, and lets the user hear and express in spatial audio, effectively transforming participants into musicians. This interactive element enables users to explore the intersection of spatial awareness, musical rhythm that extends to bodily expression through playful movements and dance-like gestures within the bubble-filled environment. This unique experience illuminates the intricate relationship between spatial awareness and the art of musical performance.

  PublisherOA PDFDOI

• Interdisciplinary Translations: Sensory Perception as a Universal Language

  arXiv (Cornell University) · 2024

  Senior authorCorresponding
  • Computer Science
  • Linguistics
  • Psychology

  This paper investigates sensory perception's pivotal role as a universal communicative bridge across varied cultures and disciplines, and how it manifests its value in the study of media art, human computer interaction and artificial intelligence. By analyzing its function in non-verbal communication through interactive systems, and drawing on the interpretive model in translation studies where "sense" acts as a mediation between two languages, this paper illustrates how interdisciplinary communication in media art and human-computer interaction is afforded by the abstract language of human sensory perception. Specific examples from traditional art, interactive media art, HCI, communication, and translation studies demonstrate how sensory feedback translates and conveys meaning across diverse modalities of expression and how it fosters connections between humans, art, and technology. Pertaining to this topic, this paper analyzes the impact of sensory feedback systems in designing interactive experiences, and reveals the guiding role of sensory perception in the design philosophy of AI systems. Overall, the study aims to broaden the understanding of sensory perception's role in communication, highlighting its significance in the evolution of interactive experiences and its capacity to unify art, science, and the human experience.

  PublisherOA PDFDOI

• Parasitic Signals: Multimodal Sonata for Real-time Interactive Simulation of the SARS-CoV-2 Virus

  2023-10-22

  articleSenior author

  This project aims to transform the nano-scale of a striking biological phenomenon, the relationship between the SARS-CoV-2 virus and human molecules, into an interactive audiovisual simulation. In this work, Atomic Force Microscopy (AFM) touching and imaging a single molecule measures the interaction between the spike protein of SARS-CoV-2 and human cellular proteins and measures the dynamic of the spike protein. We create a comprehensive scientific model based on diverse datasets and theories presenting a real-time interactive complex system with efficient rendering and sonification using a single C++ platform. This project invites the audience into an immersive space where they can control the behavior of biomolecules, allowing them to intuitively perceive biological properties. This project is not only a demonstration of scientific data but also attempts to look at the interspecies relationship in parasitism which particularly deals with our current and post-pandemic life with coronavirus and how we might control our coexistence in a virtual space.

  PublisherDOI

• Composing and Performing Complex Systems: From the Quantum to the Cosmological

  Leonardo · 2021

  1st authorCorresponding
  • Computer Science
  • Computer Science
  • Distributed computing

  Abstract This paper discusses the creation and development of a large distributed immersive multimedia computation system and environment based on the discipline of orchestral music composition, concert hall design, and performance. Just as the orchestra evolved through mechanical engineering to become a large distributed multiuser instrument whose information can be transmitted by either a client-server model (i.e. orchestra-conductor) or a client-to-client model, as in an instrumental ensemble, large-scale distributed multimedia computational platforms can be modeled in the same way, facilitating the users as performers of the system. Multiple researchers can mine large, complex data sets to uncover relationships of interest in their spatiotemporal structures.

  PublisherDOI

• MYRIOI

  2020 · 1 citations

  1st authorCorresponding
  • Computer Science
  • Computer Science
  • Human–computer interaction

  What would it be like to have a shared VR experience and to be present---to really feel presence together---in immersive worlds unimagined, from the atomic to the cosmic?

  PublisherDOI

• Mediating Public Space: Art and Technology That Goes Beyond the Frame Art Gallery

  Leonardo · 2020

  • Computer Science
  • Visual arts
  • Art

  Algotecton (from algorithm and tecton-carpentry, articulation) is a site-specific generative sculpture inspired by the Weaire-Phelan structure, a mathematical construct that approximates the geometry of foam. It comprises 16 interlocking polyhedra fabricated using advanced parametric modeling and computer numerical control (CNC) technologies. Evocative of different natural formations-a crystalline structure, a kelp forest, a molecular compound-the sculpture responds to Kendall Buster's Parabiosis II piece at the street level of the Washington Convention Center in Washington, D.C. Algotecton harnesses state-of-theart computational design and fabrication techniques to give material expression to mathematical concepts, invite discovery and playfully transform people's perception of space and form.

  PublisherOA PDFDOI

• Structure in Charles Wuorinen's String Trio

  Perspectives of New Music · 2018-01-01

  article1st authorCorresponding

  Typescript (Photocopy) --- Thesis (Ph. D.)--University of Rochester, 1984.

  PublisherDOI

• PROBABLY/POSSIBLY?

  2017-10-19 · 1 citations

  article1st authorCorresponding

  This research project is based on 32 years of Kuchera-Morin's research and practice in spatio-temporal music composition and media arts. The project is an immersive interactive visual, sonic computational instrument presented as an installation, which includes the development of an open-source computational language, and Kuchera-Morin's immersive interactive visual/sonic composition PROBABLY/POSSIBLY? Using the mathematics of quantum mechanics, the immersive instrument and computational language facilitates the creation of new, unique visual/sonic art forms. This project allows the artist to drive scientific and technological research for creative expression. This same technology is giving physicists insight into higher dimensional representation. The immersive visual/sonic instrument and language is based on the time-dependent Schrödinger equation splitting a hydrogen-like atom's electron in superposition in various orbitals. The immersive media composition, PROBABLY/POSSIBLY? can be interactively performed using our multimodal computational platform and open source language. The instrument/installation can also be used to compose and perform a number of art works based on the time-dependent Schrödinger equation

  PublisherDOI

• Seeing and Hearing the Eigenvectors of a Fluid

  2017-01-01

  article
  Publisher

Recent grants

• Artistic Group Performance as a Model for Novel Collaborative Multimodal Human-Computer Interfaces

  NSF · $194k · 2007–2009

• EAGER: A Computational Framework Integrating Methods from Music Composition and Sketching for Large Scale Scientific Data Visualizations in the 3D Immersive Allosphere

  NSF · $300k · 2010–2013

Frequent coauthors

• Matthew Wright

  Rochester Institute of Technology

  15 shared

• Lance Putnam

  10 shared

• Charles Roberts

  10 shared

• Tobias Höllerer

  10 shared

• Graham Wakefield

  9 shared

• Stephen Pope

  University of Pretoria

  6 shared

• Dan Overholt

  Aalborg University

  5 shared

• Andrés Cabrera

  University of California, Santa Barbara

  5 shared

Labs

• AlloSphere Research FacilityPI

Awards & honors

• Seattle Public Library 'Making Visible the Invisible' perman…
• John Simon Guggenheim Fellowship in Visual Arts (2016)
• Creative Capital Foundation support
• Daniel Langlois Foundation for the Arts, Science and Technol…
• Canada Council for the Arts support