About

Tapomayukh "Tapo" Bhattacharjee is an Assistant Professor in the Computer Science department at Cornell University. His research is focused on enabling robots to assist people with mobility limitations in performing activities of daily living. He emphasizes the importance of efficient and safe physical and social interactions between robots and their immediate environments as a key to achieving this goal. His work spans multiple fields including human-robot interaction, haptic perception, and robot manipulation, aiming to improve the capabilities of assistive robots through these interdisciplinary approaches.

Research topics

• Artificial Intelligence
• Computer Science
• Human–computer interaction
• Machine Learning
• Computer Security
• Simulation
• Psychology
• Engineering
• Mathematics

Selected publications

• A Human-in-the-Loop Confidence-Aware Failure Recovery Framework for Modular Robot Policies

  ArXiv.org · 2026-02-10

  articleOpen accessSenior author

  Robots operating in unstructured human environments inevitably encounter failures, especially in robot caregiving scenarios. While humans can often help robots recover, excessive or poorly targeted queries impose unnecessary cognitive and physical workload on the human partner. We present a human-in-the-loop failure-recovery framework for modular robotic policies, where a policy is composed of distinct modules such as perception, planning, and control, any of which may fail and often require different forms of human feedback. Our framework integrates calibrated estimates of module-level uncertainty with models of human intervention cost to decide which module to query and when to query the human. It separates these two decisions: a module selector identifies the module most likely responsible for failure, and a querying algorithm determines whether to solicit human input or act autonomously. We evaluate several module-selection strategies and querying algorithms in controlled synthetic experiments, revealing trade-offs between recovery efficiency, robustness to system and user variables, and user workload. Finally, we deploy the framework on a robot-assisted bite acquisition system and demonstrate, in studies involving individuals with both emulated and real mobility limitations, that it improves recovery success while reducing the workload imposed on users. Our results highlight how explicitly reasoning about both robot uncertainty and human effort can enable more efficient and user-centered failure recovery in collaborative robots. Supplementary materials and videos can be found at: http://emprise.cs.cornell.edu/modularhil

  PublisherOA PDF

• A Human-in-the-Loop Confidence-Aware Failure Recovery Framework for Modular Robot Policies

  2026-03-10

  articleSenior author

  Robots operating in unstructured human environments inevitably encounter failures, especially in robot caregiving scenarios. While humans can often help robots recover, excessive or poorly targeted queries impose unnecessary cognitive and physical workload on the human partner. We present a human-in-the-loop failure-recovery framework for modular robotic policies, where a policy is composed of distinct modules such as perception, planning, and control, any of which may fail and often require different forms of human feedback. Our framework integrates calibrated estimates of module-level uncertainty with models of human intervention cost to decide which module to query and when to query the human. It separates these two decisions: a module selector identifies the module most likely responsible for failure, and a querying algorithm determines whether to solicit human input or act autonomously. We evaluate several module-selection strategies and querying algorithms in controlled synthetic experiments, revealing trade-offs between recovery efficiency, robustness to system and user variables, and user workload. Finally, we deploy the framework on a robot-assisted bite acquisition system and demonstrate, in studies involving individuals with both emulated and real mobility limitations, that it improves recovery success while reducing the workload imposed on users. Our results highlight how explicitly reasoning about both robot uncertainty and human effort can enable more efficient and user-centered failure recovery in collaborative robots. Supplementary materials and videos can be found at: emprise.cs.cornell.edu/modularhil.

  PublisherDOI

• WAFFLE: A Wearable Approach to Bite Timing Estimation in Robot-Assisted Feeding

  2026-03-10

  article

  Millions of people around the world need assistance with feeding. Robotic feeding systems offer the potential to enhance autonomy and quality of life for individuals with impairments and reduce caregiver workload. However, their widespread adoption has been limited by technical challenges such as estimating bite timing, the appropriate moment for the robot to transfer food to a user’s mouth. In this work, we introduce WAFFLE: Wearable Approach For Feeding with LEarned Bite Timing, a system that accurately predicts bite timing by leveraging wearable sensor data to be highly reactive to natural user cues such as head movements, chewing, and talking. We train a supervised regression model on bite timing data from 14 participants and incorporate a user-adjustable assertiveness threshold to convert predictions into proceed or stop commands. In a study with 15 participants without motor impairments with the Obi feeding robot, WAFFLE performs statistically on par with or better than baseline methods across measures of feeling of control, robot understanding, and workload, and is preferred by the majority of participants for both individual and social dining. We further demonstrate WAFFLE’s generalizability in a study with 2 participants with motor impairments in their home environments using a Kinova 7DOF robot. Our findings support WAFFLE’s effectiveness in enabling natural, reactive bite timing that generalizes across users, robot hardware, robot positioning, feeding trajectories, foods, and both individual and social dining contexts. Videos are located at https://sites.google.com/view/bitetiming/.

  PublisherDOI

• CareEval: Evaluating Large Language Models for Decision-Making in Physical Robot Caregiving

  2026-03-12

  articleSenior author

  We present CareEval, a benchmark for evaluating the physical caregiving decision-making abilities of Large Language Models. Developed with a licensed occupational therapist expert in caregiving and validated by eight clinical stakeholders, it contains 100 realistic scenarios spanning all six basic Activities of Daily Living. Instead of testing general reasoning, CareEval assesses whether model responses account for key physical caregiving factors, such as user function, agency, intent, communication, and safety, and align with expert practice. Across several state-of-the-art LLMs, the best model only scores 53.1%, revealing substantial gaps in current models’ ability to reason about physical caregiving. We release 80 of the CareEval scenarios and all prompts through our website: https://emprise.cs.cornell.edu/care-eval/.

  PublisherDOI

• Dataset for CLAMP: Crowdsourcing a LArge-scale in-the-wild haptic dataset with an open-source device for Multimodal robot Perception

  Harvard Dataverse · 2026-01-01

  datasetOpen accessSenior author

  This is the dataset for the publication "CLAMP: Crowdsourcing a LArge-scale in-the-wild haptic dataset with an open-source device for Multimodal robot Perception". Refer to the project website for more details: https://emprise.cs.cornell.edu/clamp/

  PublisherDOI

• A Human-in-the-Loop Confidence-Aware Failure Recovery Framework for Modular Robot Policies

  Open MIND · 2026-02-10

  preprintSenior author

  Robots operating in unstructured human environments inevitably encounter failures, especially in robot caregiving scenarios. While humans can often help robots recover, excessive or poorly targeted queries impose unnecessary cognitive and physical workload on the human partner. We present a human-in-the-loop failure-recovery framework for modular robotic policies, where a policy is composed of distinct modules such as perception, planning, and control, any of which may fail and often require different forms of human feedback. Our framework integrates calibrated estimates of module-level uncertainty with models of human intervention cost to decide which module to query and when to query the human. It separates these two decisions: a module selector identifies the module most likely responsible for failure, and a querying algorithm determines whether to solicit human input or act autonomously. We evaluate several module-selection strategies and querying algorithms in controlled synthetic experiments, revealing trade-offs between recovery efficiency, robustness to system and user variables, and user workload. Finally, we deploy the framework on a robot-assisted bite acquisition system and demonstrate, in studies involving individuals with both emulated and real mobility limitations, that it improves recovery success while reducing the workload imposed on users. Our results highlight how explicitly reasoning about both robot uncertainty and human effort can enable more efficient and user-centered failure recovery in collaborative robots. Supplementary materials and videos can be found at: http://emprise.cs.cornell.edu/modularhil

  DOI

• CLAMP: Crowdsourcing a LArge-scale in-the-wild haptic dataset with an open-source device for Multimodal robot Perception

  ArXiv.org · 2025-05-27

  preprintOpen accessSenior author

  Robust robot manipulation in unstructured environments often requires understanding object properties that extend beyond geometry, such as material or compliance-properties that can be challenging to infer using vision alone. Multimodal haptic sensing provides a promising avenue for inferring such properties, yet progress has been constrained by the lack of large, diverse, and realistic haptic datasets. In this work, we introduce the CLAMP device, a low-cost (<\$200) sensorized reacher-grabber designed to collect large-scale, in-the-wild multimodal haptic data from non-expert users in everyday settings. We deployed 16 CLAMP devices to 41 participants, resulting in the CLAMP dataset, the largest open-source multimodal haptic dataset to date, comprising 12.3 million datapoints across 5357 household objects. Using this dataset, we train a haptic encoder that can infer material and compliance object properties from multimodal haptic data. We leverage this encoder to create the CLAMP model, a visuo-haptic perception model for material recognition that generalizes to novel objects and three robot embodiments with minimal finetuning. We also demonstrate the effectiveness of our model in three real-world robot manipulation tasks: sorting recyclable and non-recyclable waste, retrieving objects from a cluttered bag, and distinguishing overripe from ripe bananas. Our results show that large-scale, in-the-wild haptic data collection can unlock new capabilities for generalizable robot manipulation. Website: https://emprise.cs.cornell.edu/clamp/

  PublisherOA PDFDOI

• Oral temperature as an indicator of disease in pre-weaned dairy calves

  American Association of Bovine Practitioners Conference Proceedings · 2025-03-26

  articleOpen access

  Diarrheal and respiratory diseases pose significant threats to pre-weaned dairy calves. Early detection affords prompt inter­vention, thereby minimizing disease spread and severity and improving welfare and performance. Change in core body tem­perature can be an early indicator of illness. In calves, this is typically measured via rectal temperature (RT) which is time intensive, stressful to the calf, and invasive. Our study explored the potential of measuring oral temperature (OT) as an alterna­tive indicator of fever in dairy calves with a goal of informing the design of novel health monitoring sensors.

  PublisherOA PDFDOI

• To Ask or not to Ask: Human-in-the-loop Contextual Bandits with Applications in Robot-Assisted Feeding

  2025-05-19 · 4 citations

  articleOpen accessSenior author

  Robot-assisted bite acquisition involves picking up food items with varying shapes, compliance, sizes, and textures. Fully autonomous strategies may not generalize efficiently across this diversity. We propose leveraging feedback from the care recipient when encountering novel food items. However, frequent queries impose a workload on the user. We formulate human-in-the-loop bite acquisition within a contextual bandit framework and introduce LINUCB-QG, a method that selectively asks for help using a predictive model of querying workload based on query types and timings. This model is trained on data collected in an online study involving 14 participants with mobility limitations, 3 occupational therapists simulating physical limitations, and 89 participants without limitations. We demonstrate that our method better balances task performance and querying workload compared to autonomous and always-querying baselines and adjusts its querying behavior to account for higher workload in users with mobility limitations. We validate this through experiments in a simulated food dataset and a user study with 19 participants, including one with severe mobility limitations. Please check out our project website at: emprise.cs.cornell.edu/hilbiteacquisition/.

  PublisherOA PDFDOI

• Detection of Neonatal Calf Diarrhea Using Suckle Pressure and Machine Learning

  Agriculture · 2025-08-28

  articleOpen access

  Neonatal calf diarrhea (NCD) remains one of the most prevalent and economically burdensome health challenges in preweaned calves, leading to compromised growth, increased morbidity, and high mortality rates worldwide. While traditional methods such as physical examination and clinical health scoring are widely used, they often require trained personnel, are resource-intensive, and are prone to subjectivity, which limits their scalability in large dairy operations. This observational cohort study investigated the feasibility of using suckle pressure measurement combined with machine learning (ML) techniques for NCD detection. A total of 51 female Holstein calves on a commercial dairy farm were enrolled at birth and health scored daily from 1 to 21 days of age. Suckle pressures were measured at 1, 3, 5, 7, 10, 14, and 21 days, as well as daily following NCD diagnosis until fecal consistency returned to normal. Pressure measurements were captured using impression film-wrapped nipples, producing 349 images, of which 54 were from calves diagnosed with NCD. Image features, including pixel density, color saturation, entropy, and histogram-based features, were extracted for analysis. Multiple ML classifiers—Support Vector Machine, K-Nearest Neighbors, Random Forest, Gradient Boosting, and Easy Ensemble (EE)—were applied to detect NCD status based on image features. The EE classifier achieved the best detection performance, with an accuracy of 0.90, precision of 0.64, and recall of 0.82, effectively handling data imbalance. Notably, the results also demonstrated that NCD onset could be predicted up to one day prior to clinical manifestation by training classifiers on pre-symptomatic suckle pressure data and testing on post-onset data. The EE classifier also outperformed other models in this early prediction window, with an accuracy of 0.74, precision of 0.67, and recall of 0.70. The results of our preliminary study suggest that suckle pressure may offer a novel, non-invasive approach for precision health monitoring in dairy systems, enabling timely intervention to reduce disease severity, improve calf health, and minimize economic losses.

  PublisherOA PDFDOI

Frequent coauthors

• Siddhartha S Srinivasa

  3535 shared

• Ethan Kroll Gordon

  University of Washington

  3047 shared

• Matt Schmittle

  University of Washington

  1976 shared

• Gilwoo Lee

  University of Washington

  1760 shared

• Ryan Feng

  University of Michigan–Ann Arbor

  1622 shared

• Shivaum Kumar

  University of Washington

  1580 shared

• Youngsun Kim

  Kyung Hee University Medical Center

  1476 shared

• Shivaum Kumar

  University of Washington

  1450 shared

Labs

• EmPRISE LabPI

  Enabling robots to assist people with mobility limitations with activities of daily living.

Education

• Ph.D., robotics

  Georgia Institute of Technology

• Other, Computer Science and Engineering

  University of Washington

Awards & honors

• TRI Young Faculty Researcher Award '24
• NSF CAREER Award '23
• AFCEA 40 under 40 Award '22
• Best Paper and Student Paper Award Finalist at ICRA’25
• Best Systems Paper Award Finalist at HRI'24