About

Thanassis Rikakis is a professor whose work explores collaborative knowledge and innovation in 21st-century education. His professional website includes sections on his biography, CV, education, faculty positions, administrative roles, research, grants, publications, projects, labs, curriculum development, and teaching, with a particular emphasis on music and collaborative learning. His research focuses on advancing educational practices through innovative approaches to collaboration and knowledge sharing, contributing to the development of new educational paradigms and methodologies.

Research topics

• Artificial Intelligence
• Computer Science
• Human–computer interaction
• Simulation
• Computer vision
• Medicine
• Geology
• Engineering
• Physical therapy
• Cognitive science
• Neuroscience
• Physical medicine and rehabilitation
• Psychology

Selected publications

• A methodology for integrating AI into embodied human intelligence for the performance of complex tasks

  Frontiers in Artificial Intelligence · 2026-03-04

  articleOpen accessSenior author

  We propose a theory and methodology for designing human-artificial intelligence (AI) collaboration in complex, embodied tasks. The theory distinguishes human embodied intelligence from computational intelligence and identifies synergies in which AI enhances—rather than replicates or replaces—human performance. We represent observable structures of expert performance as a nested network with four interdependent layers: Environment (space and tools), Activity (what is done), Goals (what is aimed for), and Meaning (how performance is interpreted), all connected by dynamic four-layer edges. A bidirectional Dynamic Bayesian Network (DBN) computes this representation across temporal scales: instants, actions, complete performances, and sequences. The DBN informs the design of digital tools (from sensors to data structures and AI modules) that capture human performance and extract features, descriptors, and predictions that enhance the observability and analysis of performance. During task performance, a top-down pass predicts expert orientation—current goals and interpretations—and drives a search policy that selects where to look. A bottom-up pass processes action-conditioned computational observations and filters them through a gated pipeline to produce new candidates for four-layer connectivity (c4). After expert validation, candidates update the network, sharpening DBN posteriors, reducing entropy, and thereby enhancing human performance. We instantiated this framework in automated physical rehabilitation assessment through a 12-month deployment with 10 clinicians and 105 stroke survivors. Co-design cycles developed and enriched a four-layer DBN representation of rehabilitation assessment and informed the design of a computational ensemble for automated assessment. The computational ensemble achieved 90.8% agreement with clinicians at the exercise level, 93.1% at the segment level, and 90.6% at the movement quality level. Clinicians validated automated assessments at high rates and reported improved confidence and efficiency when leveraging ensemble insights for therapy assessment and planning. This portable methodology and theory can be applied to the embodied performance of complex tasks across multiple applications.

  PublisherOA PDFDOI

• Automated ARAT Scoring Using Multimodal Video Analysis, Multi-View Fusion, and Hierarchical Bayesian Models: A Clinician Study

  ArXiv.org · 2025-05-03

  preprintOpen accessSenior author

  Manual scoring of the Action Research Arm Test (ARAT) for upper extremity assessment in stroke rehabilitation is time-intensive and variable. We propose an automated ARAT scoring system integrating multimodal video analysis with SlowFast, I3D, and Transformer-based models using OpenPose keypoints and object locations. Our approach employs multi-view data (ipsilateral, contralateral, and top perspectives), applying early and late fusion to combine features across views and models. Hierarchical Bayesian Models (HBMs) infer movement quality components, enhancing interpretability. A clinician dashboard displays task scores, execution times, and quality assessments. We conducted a study with five clinicians who reviewed 500 video ratings generated by our system, providing feedback on its accuracy and usability. Evaluated on a stroke rehabilitation dataset, our framework achieves 89.0% validation accuracy with late fusion, with HBMs aligning closely with manual assessments. This work advances automated rehabilitation by offering a scalable, interpretable solution with clinical validation.

  PublisherOA PDFDOI

• Automatic Temporal Segmentation for Post-Stroke Rehabilitation: A Keypoint Detection and Temporal Segmentation Approach for Small Datasets

  2025-02-28

  article

  Rehabilitation is essential and critical for post-stroke patients, addressing both physical and cognitive aspects. Stroke predominantly affects older adults, with 75% of cases occurring in individuals aged 65 and older, under-scoring the urgent need for tailored rehabilitation strategies in aging populations. Despite the critical role therapists play in evaluating rehabilitation progress and ensuring the effectiveness of treatment, current assessment methods can often be subjective, inconsistent, and time-consuming, leading to delays in adjusting therapy protocols. This study aims to address these challenges by providing a solution for con-sistent and timely analysis. Specifically, we perform temporal segmentation of video recordings to capture detailed activities during stroke patients' rehabilitation. The main application scenario motivating this study is the clinical as-sessment of daily tabletop object interactions, which are crucial for post-stroke physical rehabilitation. To achieve this, we present a framework that leverages the biomechan-ics of movement during therapy sessions. Our solution di-vides the process into two main tasks: 2D keypoint detection to track patients' physical movements, and 1 D time-series temporal segmentation to analyze these movements over time. This dual approach enables automated labeling with only a limited set of real-world data, addressing the challenges of variability in patient movements and limited dataset availability. By tackling these issues, our method shows strong potential for practical deployment in physical therapy settings, enhancing the speed and accuracy of rehabilitation assessments.

  PublisherDOI

• Data Acquisition Through Participatory Design for Automated Rehabilitation Assessment

  arXiv (Cornell University) · 2025-01-02

  preprintOpen access

  Through participatory design, we are developing a computational system for the semi-automated assessment of the Action Research Arm Test (ARAT) for stroke rehabilitation. During rehabilitation assessment, clinicians rate movement segments and components in the context of overall task performance. Clinicians change viewing angles to assess particular components. Through studies with clinicians, we develop a system that includes: a) unobtrusive multi-camera capture, b) a segmentation interface for non-expert segmentors, and c) a rating interface for expert clinicians. Five clinicians independently captured 1800 stroke survivor videos with <5$\%$ errors. Three segmentors have segmented 760 of these videos, averaging 20 seconds per segment. They favor the recommended camera view $>$ 90\%. Multiple clinicians have rated the segmented videos while reporting minimal problems. The complete data will be used for training an automated segmentation and rating system that empowers the clinicians as the ratings will be compatible with clinical practice and intuition.

  PublisherOA PDFDOI

• Automatic Temporal Segmentation for Post-Stroke Rehabilitation: A Keypoint Detection and Temporal Segmentation Approach for Small Datasets

  ArXiv.org · 2025-02-27

  preprintOpen access

  Rehabilitation is essential and critical for post-stroke patients, addressing both physical and cognitive aspects. Stroke predominantly affects older adults, with 75% of cases occurring in individuals aged 65 and older, underscoring the urgent need for tailored rehabilitation strategies in aging populations. Despite the critical role therapists play in evaluating rehabilitation progress and ensuring the effectiveness of treatment, current assessment methods can often be subjective, inconsistent, and time-consuming, leading to delays in adjusting therapy protocols. This study aims to address these challenges by providing a solution for consistent and timely analysis. Specifically, we perform temporal segmentation of video recordings to capture detailed activities during stroke patients' rehabilitation. The main application scenario motivating this study is the clinical assessment of daily tabletop object interactions, which are crucial for post-stroke physical rehabilitation. To achieve this, we present a framework that leverages the biomechanics of movement during therapy sessions. Our solution divides the process into two main tasks: 2D keypoint detection to track patients' physical movements, and 1D time-series temporal segmentation to analyze these movements over time. This dual approach enables automated labeling with only a limited set of real-world data, addressing the challenges of variability in patient movements and limited dataset availability. By tackling these issues, our method shows strong potential for practical deployment in physical therapy settings, enhancing the speed and accuracy of rehabilitation assessments.

  PublisherOA PDFDOI

• Enacting Transdisciplinary Values for a Postdigital World: The Challenge-Based Reflective Learning (CBRL) Framework

  Postdigital Science and Education · 2024-06-22 · 6 citations

  articleOpen accessSenior author

  Abstract Traditional disciplinary and interdisciplinary educational models often fall short in enabling students to transform problems and solutions for real-world needs. They restrict learners’ ability to deconstruct problems and innovate beyond their subject-based expertise, hindering the development of reflective practice in new and unknown situations across domains. This paper introduces the Challenge-Based Reflective Learning (CBRL) framework that emphasizes context-driven, challenge-based experiential learning process. It presents a novel approach to understanding cross-boundary interactions and learning, overcoming the limitations of traditional, discipline-bounded models involving inter- and trans-disciplinarity. CBRL cultivates reflective practice by nurturing domain-general competencies and domain-specific skills inherent in concrete human experiences. This paper translates reflective practice theories into actionable methods for higher education, demonstrating their application at the Iovine and Young Academy at the University of Southern California—a school that integrates technology, arts and design, and business and entrepreneurship through its reflective, challenge-driven learning approach. The case study outlines a four-year college curriculum that flexibly incorporates student interests and societal challenges across domains. This paper enhances the scholarship of reflective practice and transdisciplinary education and research, discussing the implications for cultivating new kinds of expertise needed in a postdigital era.

  PublisherOA PDFDOI

• Advances in Computer Vision for Home-Based Stroke Rehabilitation

  2024-05-29 · 1 citations

  book-chapter

  This chapter examines the application of computer vision (CV) and deep learning (DL) techniques in the design and development of stroke rehabilitation systems. We begin by reviewing motion-capture-based solutions and discussing their inherent challenges and limitations. Next, we explore the requirements for next-generation home-based rehabilitation systems using simple RGB camera sensors, focusing on two main challenges: (1) the scarcity of high-quality data in healthcare settings, and (2) the limited explainability and usability of CV techniques due to their black-box nature. Addressing the crucial interaction between healthcare providers and rehabilitation systems, we present a series of experiments in collaboration with multiple hospitals to demonstrate the efficacy of a cyber–human intelligent system design in overcoming these challenges. Furthermore, we outline essential design principles for building low-cost, minimally intrusive rehabilitation systems that can be deployed in patients’ homes. Finally, we discuss the potential of 3D CV advances in designing the next generation of rehabilitation systems and review future opportunities in this domain.

  PublisherDOI

• A Hierarchical Bayesian Model for Cyber-Human Assessment of Movement in Upper Extremity Stroke Rehabilitation

  IEEE Transactions on Neural Systems and Rehabilitation Engineering · 2024-01-01 · 2 citations

  articleOpen access

  The evidence-based quantification of the relation between changes in movement quality and functionality can assist clinicians in achieving more effective structuring or adapting of therapy. In this paper, clinicians rated task, segment, and composite movement feature performance for 478 videos of stroke survivors executing upper extremity therapy tasks. We used the clinician ratings to develop a Hierarchical Bayesian Model (HBM) with task, segment, and composite layers for computing the statistical relation of movement quality changes to function. The model was enhanced through a detailed correlation graph (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Delta _{\textit {HBM}}$ </tex-math></inline-formula>) that links computationally extracted kinematics with clinician-rated composite features for different task-segment combinations. Utilizing the weights and correlation graphs, we finally derive reverse cascading probabilities of the proposed HBM from kinematics to composite features, segments, and tasks. In a test involving 98 cases where clinician ratings differed, the HBM resolved 95% of these discrepancies. The model effectively aligned kinematic data with specific task-segment combinations in over 90% of cases. Once the HBM is expanded and refined through additional data it can be used for the automated calculation of statistical relations between changes in kinematics and performance of functional tasks and the generation of therapy assessment recommendations for clinicians. While our work primarily focuses on the upper extremities of stroke survivors, the HBM can be adapted to many other neurorehabilitation contexts.

  PublisherDOI

• ASAR Dataset and Computational Model for Affective State Recognition During ARAT Assessment for Upper Extremity Stroke Survivors

  2023-10-09 · 2 citations

  articleOpen access

  Stroke is one of the leading global causes of disability and many stroke survivors must deal with challenging movement impairment daily. Stroke rehabilitation therapy is expensive, physically, and emotionally challenging, can be difficult to access, and is time-consuming for patients and clinicians alike. As a result, there is an opportunity to develop automated or semi-automated systems to assist clinicians in efficient and accurate stroke patient movement assessment. In this work, we present the ASAR (Affective State for ARAT Rehab) dataset where we use custom label classes (neutral, engagement, and pain) to annotate video data of stroke survivors performing the standardized Action Research Arm Test (ARAT) for upper extremity assessment. 106 patients participated in this study and each patient performed, or attempted to perform, each of the 19 ARAT assessment tasks. In our preliminary analysis, we annotated the frames of 126 videos for 10 patients. Using a multimodal model, we achieved an accuracy of 0.77 ± 0.06 in per-frame state recognition on the ASAR dataset. This affective information will potentially increase the interpretability of clinician’s scores of the stroke survivor’s ARAT performance and thus could assist in the design and implementation of semi-automated systems for stroke assessment and rehabilitation.

  PublisherOA PDFDOI

• A Hierarchical Bayesian Model for Cyber-Human Assessment of Rehabilitation Movement

  medRxiv · 2022-05-27

  preprintOpen access

  Abstract Background The evidence-based quantification of the relation between changes in movement quality and functionality can assist clinicians in achieving more effective structuring or adaptations of therapy. Facilitating this quantification through computational tools can also result in the generation of large-scale data sets that can inform automated assessment of rehabilitation. Interpretable automated assessment can leave more time for clinicians to focus on treatment and allow for remotely supervised therapy at the home. Methods In our first experiment, we developed a rating process and accompanying computational tool to assist clinicians in following a standardized movement assessment process relating functionality to movement quality. We conducted three studies with three different versions of the computational rating tool. Clinicians rated task, segment, and movement feature performance for 440 videos in which stroke survivors executed standardized upper extremity therapy tasks related to functional activities. In our second experiment, we used the 440 rated videos, in addition to 140 videos of unimpaired subjects performing the same tasks, to improve our previously developed automated assessment ensemble model that automatically generates segmentation times and task ratings across impaired and unimpaired movement. The automated assessment ensemble integrates expert knowledge constraints into data driven training though a combination of HMM, transformer, MSTCN++, and decision tree computational modules. In our third experiment, we used the therapist and automated ratings to develop a four-layer Hierarchical Bayesian Model (HBM) for computing the statistical relation of movement quality changes to functionality. We first calculated conditional layer probabilities using clinician ratings of task, segment, and movement features. We increased the granularity of observation of the HBM by formulating Δ HBM , a correlation graph between kinematics and movement composite features. Finally, we used k-means clustering on the Δ HBM to identify three clusters of features among the 16 movement composite and 20 kinematic features and used the centroid of these clusters as the weights of the input data to our computational assessment ensemble. Results We evaluated the efficacy of our rating interface in terms of inter-rater reliability (IRR) across tasks, segments, and movement features. The third version of the interface produced an average IRR of 67%, while the time per session (TPS) was the lowest of the three studies. By analyzing the ratings, we were able to identify a small number of movement features that have the highest probability of predicting functional improvement. We evaluated the performance of our automated assessment model using 60% impaired and 40% unimpaired movement data and achieved a frame-wise segmentation accuracy of 87.85±0.58 and a block-segmentation accuracy of 98.46±1.6. We also demonstrated the performance of our proposed HBM in correlation to clinician’s ratings with a correlation over 90%. The HBM also generates a correlation graph, Δ HBM that relates 16 composite movement features to the 20 kinematic features. We can thus integrate the HBM into the computational assessment ensemble to perform automated and integrated movement quality and functionality assessment that is driven by computationally extracted kinematics. Conclusions Combining standardized clinician ratings of videos with knowledge based and data driven computational analysis of rehabilitation movement allows the expression of an HBM that increases the observability of the relation of movement quality to functionality and enables the training of computational algorithms for automated assessment of rehabilitation movement. While our work primarily focuses on the upper extremity of stroke survivors, the models can be adopted to many other neurorehabilitation contexts.

  PublisherOA PDFDOI

Recent grants

• SCH:INT: Collaborative Research: Semi-Automated Rehabilitation in the Home

  NSF · $1.1M · 2020–2022

• Collaborative Research: EAGER: A Virtual eXchange to Support Networks of Creativity and Innovation Amongst Science, Engineering, Arts and Design (XSEAD)

  NSF · $62k · 2013–2014

• Collaborative Research: EAGER: A Virtual eXchange to Support Networks of Creativity and Innovation Amongst Science, Engineering, Arts and Design (XSEAD)

  NSF · $259k · 2011–2013

Frequent coauthors

• Hari Sundaram

  23 shared

• Yinpeng Chen

  19 shared

• Aisling Kelliher

  University of Southern California

  17 shared

• Steven L. Wolf

  Emory University

  17 shared

• Nicole Lehrer

  German Center for Lung Research

  14 shared

• Margaret Duff

  13 shared

• Todd Ingalls

  12 shared

• Jiping He

  Beijing Institute of Technology

  9 shared