About

Shubham Jain received her PhD from WINLAB at Rutgers University in 2017, under the supervision of Dr. Marco Gruteser. Her research interests are in smart environments, wearable health sensing frameworks, and cyber-physical systems (CPS). Her expertise lies in leveraging pervasive sensing devices toward developing multimodal data integration and interpretation frameworks. Her research enables mobile devices to expand their role and innovate new services, ranging from continuous health monitoring to large-scale video analytics and pedestrian safety. Her research in pedestrian safety received the Large Organization Recognition Award at AT&T Connected Intersections challenge and has been featured in various media outlets including the Wall Street Journal.

Research topics

• Computer Science
• Artificial Intelligence
• Computer vision
• Machine Learning
• Transport engineering
• Mathematics
• Engineering
• Speech recognition

Selected publications

• AI-Powered Predictive Maintenance in 6G RAN: Enhancing Reliability

  Journal of Quantum Science and Technology. · 2025-01-03

  articleSenior author

  The rapid evolution of 6G networks is on the verge of revolutionizing wireless communication with unprecedented speed, reduced latency, and connectivity. With the increased complexity of RAN in 6G, maintaining the network's reliability and minimizing network downtime is of utmost importance. The introduction of predictive maintenance powered by Artificial Intelligence has offered a new horizon in enhancing the performance and reliability of 6G RANs. This paper will present the integration of AI algorithms, including machine and deep learning, in predicting potential failures and the maintenance requirement of the 6G RAN infrastructure. By analyzing the data extracted from network sensors, real-time monitoring, and past performance trends, the AI can accurately detect emerging issues and suggest timely interventions that can avoid service interruptions that involve higher costs and reduce the need for physical inspection. The paper will further suggest how predictive maintenance in AI-powered modes can optimize network resource utilization, improve the lifespan of the system, and enhance the efficiency of the 6G network in general. Continuing advancements in AI technology will see 6G RAN self-heal, adapt to changing conditions, and enable autonomous predictive diagnostics. This transformative approach will not only reduce operational costs but also ensure the seamless operation of 6G networks, which are fundamental for the development of smart cities and autonomous vehicles, among other mission-critical applications relying on stable and efficient wireless communication. It concludes with the fact that AI can shape the future of 6G networks in establishing a new paradigm in the maintenance and management of next-generation communication infrastructures

  PublisherDOI

• YOPO-Nav: Visual Navigation using 3DGS Graphs from One-Pass Videos

  ArXiv.org · 2025-12-10

  preprintOpen access

  Visual navigation has emerged as a practical alternative to traditional robotic navigation pipelines that rely on detailed mapping and path planning. However, constructing and maintaining 3D maps is often computationally expensive and memory-intensive. We address the problem of visual navigation when exploration videos of a large environment are available. The videos serve as a visual reference, allowing a robot to retrace the explored trajectories without relying on metric maps. Our proposed method, YOPO-Nav (You Only Pass Once), encodes an environment into a compact spatial representation composed of interconnected local 3D Gaussian Splatting (3DGS) models. During navigation, the framework aligns the robot's current visual observation with this representation and predicts actions that guide it back toward the demonstrated trajectory. YOPO-Nav employs a hierarchical design: a visual place recognition (VPR) module provides coarse localization, while the local 3DGS models refine the goal and intermediate poses to generate control actions. To evaluate our approach, we introduce the YOPO-Campus dataset, comprising 4 hours of egocentric video and robot controller inputs from over 6 km of human-teleoperated robot trajectories. We benchmark recent visual navigation methods on trajectories from YOPO-Campus using a Clearpath Jackal robot. Experimental results show YOPO-Nav provides excellent performance in image-goal navigation for real-world scenes on a physical robot. The dataset and code will be made publicly available for visual navigation and scene representation research.

  PublisherOA PDFDOI

• A Landmark-Aware Visual Navigation Dataset for Map Representation Learning

  2025-03-04

  article

  Map representations learned by expert demonstrations have shown promising research value. However, the field of visual navigation still faces challenges due to the lack of real-world human-navigation datasets that can support efficient, supervised, representation learning of environments. We present a Landmark-Aware Visual Navigation (LAVN) dataset to allow for supervised learning of human-centric exploration policies and map building. We collect RGBD observation and human point-click pairs as a human annotator explores virtual and real-world environments with the goal of full coverage exploration of the space. The human annotators also provide distinct landmark examples along each trajectory, which we intuit will simplify the task of map or graph building and localization. These human point-clicks serve as direct supervision for waypoint prediction when learning to explore in environments. Our dataset covers a wide spectrum of scenes, including rooms in indoor environments, as well as walkways outdoors. We releaseour dataset with detailed documentation at https://huggingface.co/datasets/visnavdataset/lavn (DOI: l0.57967/hf/2386) and a plan for long-term preservation.

  PublisherDOI

• Non-iterative sparse signal recovery algorithms for sparse matrices and their guarantees

  Journal of the Franklin Institute · 2025-10-08

  article
  PublisherDOI

• Real-time Emotion Recognition through Articulatory Motion and Speech

  2025-02-12

  articleSenior author

  We demonstrate a real-time emotion recognition system that combines articulatory motion with audio features. Our ear-worn system tracks jaw movements and facial muscle vibrations during speech to achieve robust emotion classification. The system processes three distinct signals: jaw motion, facial muscle vibrations, and bone-borne vibrations, combining them with audio features for emotion detection. We demonstrate the system's ability to recognize six emotions (happy, sad, anger, fear, disgust, neutral) in real-time with 93% accuracy while robust to body motion artifacts. Our live demonstration allows audience members to observe how different emotional expressions manifest in jaw motion patterns through real-time visualization of emotion classification results.

  PublisherDOI

• Advanced image security through chaotic system-based encryption and fisher-yates matrix shuffling

  Optik · 2025-03-13 · 9 citations

  article1st authorCorresponding
  PublisherDOI

• Deterministic construction of unimodular tight frames consisting orthogonal blocks via block preserving operators

  Signal Processing · 2024-12-04 · 3 citations

  article
  PublisherDOI

• Hand Gesture Recognition for Blind Users by Tracking 3D Gesture Trajectory

  2024-05-11 · 5 citations

  articleOpen access

  Hand gestures provide an alternate interaction modality for blind users and can be supported using commodity smartwatches without requiring specialized sensors. The enabling technology is an accurate gesture recognition algorithm, but almost all algorithms are designed for sighted users. Our study shows that blind user gestures are considerably diferent from sighted users, rendering current recognition algorithms unsuitable. Blind user gestures have high inter-user variance, making learning gesture patterns difcult without large-scale training data. Instead, we design a gesture recognition algorithm that works on a 3D representation of the gesture trajectory, capturing motion in free space. Our insight is to extract a micro-movement in the gesture that is user-invariant and use this micro-movement for gesture classifcation. To this end, we develop an ensemble classifer that combines image classifcation with geometric properties of the gesture. Our evaluation demonstrates a 92% classifcation accuracy, surpassing the next best state-of-the-art which has an accuracy of 82%.

  PublisherOA PDFDOI

• Poster Unvoiced: Designing an Unvoiced User Interface using Earables and LLMs

  2024-11-04

  articleOpen accessSenior author

  This poster presents the design and implementation of Unvoiced, a silent speech interaction system. Unvoiced transforms subtle jaw movements into rich speech spectrograms, enabling seamless and private device interaction. Our system captures low-frequency jaw motion signals using ear-worn IMUs and translates them into high-fidelity mel-spectrograms through cross-modal translation techniques. By incorporating phonetic, contextual, and syntactic information, Unvoiced generates high-fidelity spectrograms that existing speech recognition systems can process. In our evaluation with 19 users across four common tasks, Unvoiced achieved a remarkable >94% task completion rate and <9% Word Error Rate (WER) for over 90% of phrases, maintaining robust performance even in noisy conditions.

  PublisherOA PDFDOI

• Enabling Accessible and Ubiquitous Interaction in Next-Generation Wearables: An Unvoiced Speech Approach

  2024-12-04 · 2 citations

  articleSenior author

  As wearable devices increase, there's a growing need for intuitive, private, and accessible interaction methods. This position paper builds on the research on unvoiced speech interaction and authentication to propose a vision for interaction in next-generation wearables. This paper draws upon our previous work on unvoiced speech interfaces that leverage jaw movements and facial vibrations for command recognition and user authentication. We argue that unvoiced speech interaction can provide a robust, privacy-preserving, and noise-resistant alternative to traditional interfaces, enhancing accessibility and offering discrete interaction in public spaces. We discuss the potential integration of these systems into commercial devices and explore gesture-based interactions as an alternative to touch. Additionally, we discuss the future direction of unvoiced speech interfaces. This paper sets the stage for implementing unvoiced speech and gesture-based interaction in mainstream wearables in our daily interactions with technology.

  PublisherDOI

Recent grants

• CPS: Small: Performance Monitoring Cyber-Physical System for Emerging Fitness Spaces

  NSF · $500k · 2019–2021

• CAREER: Closed-loop Health Behavior Interventions in Multi-device Environments

  NSF · $589k · 2023–2028

• SCC-PG: CrossGuard: Enhancing Pedestrian Experience at Intersections

  NSF · $150k · 2020–2021

• CNS Core: Medium: Collaborative: Reality-Aware Networks

  NSF · $192k · 2020–2024

• CPS: Small: Performance Monitoring Cyber-Physical System for Emerging Fitness Spaces

  NSF · $438k · 2020–2025

Frequent coauthors

• Marco Gruteser

  19 shared

• Abrar Alali

  Old Dominion University

  15 shared

• Kristin Dana

  9 shared

• Ashwin Ashok

  9 shared

• Bryan Bo Cao

  7 shared

• Prerna Khanna

  Stony Brook University

  7 shared

• Phuc Nguyen

  University of Massachusetts Amherst

  6 shared

• Akhlesh Kumar

  Central Forensic Science Laboratory

  6 shared

Awards & honors

• Large Organization Recognition Award at AT&T Connected Inter…