A Multiple Compiler Framework for Improved Performance

Lecture notes in computer science · 2025-10-31

Classification using hyperdimensional computing: a review with comparative analysis

Artificial Intelligence Review · 2025-03-17 · 11 citations

Abstract Hyperdimensional computing (HD), also known as vector symbolic architectures (VSA), is an emerging and promising paradigm for cognitive computing. At its core, HD/VSA is characterized by its distinctive approach to compositionally representing information using high-dimensional randomized vectors. The recent surge in research within this field gains momentum from its computational efficiency stemming from low-resolution representations and ability to excel in few-shot learning scenarios. Nonetheless, the current literature is missing a comprehensive comparative analysis of various methods since each of them uses a different benchmark to evaluate its performance. This gap obstructs the monitoring of the field’s state-of-the-art advancements and acts as a significant barrier to its overall progress. To address this gap, this review not only offers a conceptual overview of the latest literature but also introduces a comprehensive comparative study of HD/VSA classification methods. The exploration starts with an overview of the strategies proposed to encode information as high-dimensional vectors. These vectors serve as integral components in the construction of classification models. Furthermore, we evaluate diverse classification methods as proposed in the existing literature. This evaluation encompasses techniques such as retraining and regenerative training to augment the model’s performance. To conclude our study, we present a comprehensive empirical study. This study serves as an in-depth analysis, systematically comparing various HD/VSA classification methods using two benchmarks, the first being a set of seven popular datasets used in HD/VSA and the second consisting of 121 datasets being the subset from the UCI Machine Learning repository. To facilitate future research on classification with HD/VSA, we open-sourced the benchmarking and the implementations of the methods we review. Since the considered data are tabular, encodings based on key-value pairs emerge as optimal choices, boasting superior accuracy while maintaining high efficiency. Secondly, iterative adaptive methods demonstrate remarkable efficacy, potentially complemented by a regenerative strategy, depending on the specific problem. Furthermore, we show how HD/VSA is able to generalize while training with a limited number of training instances. Lastly, we demonstrate the robustness of HD/VSA methods by subjecting the model memory to a large number of bit-flips. The results illustrate that the model’s performance remains reasonably stable until the occurrence of 40% of bit flips, where the model’s performance is drastically degraded. Overall, this study performed a thorough performance evaluation on different methods and, on the one hand, a positive trend was observed in terms of improving classification performance but, on the other hand, these developments could often be surpassed by off-the-shelf methods. This calls for better integration with the broader machine learning literature; the developed benchmarking framework provides practical means for doing so.

Learning encoding phasors with fractional power encoding

2025-03-28

Hyperdimensional Computing (HDC), also Hyperdimensional Computing (HD), also known as Vector Symbolic Architectures (VSA), operates in high-dimensional vector spaces for symbolic processing. Within HD, Fractional Power Encoding (FPE), also referred to as Random Fourier Features, is a method that creates randomized data representations by exponentiating random base vectors. In this paper, we evaluate the application of FPE for classification tasks. We focus on the Fourier Holographic Reduced Representation (FHRR) within this framework. We investigate the use of various probability distributions for sampling to generate the basis hypervectors for FPE. Our findings indicate that distributions from the generalized hyperbolic family, such as Laplace, Cauchy, Student-t, and Gaussian, are more effective in capturing information during encoding. Furthermore, we propose two novel models for classification. Each model begins by determining an optimal bandwidth to achieve high accuracy. The first model, inspired by Fourier Series, generates an encoding by accumulating different bandwidths. Both models employ Stochastic Gradient Descent (SGD) to learn encodings tailored to specific applications. This approach enables the models to generate application-specific kernels, resulting in higher accuracy. Our models outperforms the current state-of-the-art in Hyperdimensional Computing in 121 datasets on accuracy, memory usage, training time and inference time allowing for achieving better results in embedded devices. Furthermore our results outperform Support Vector Machines (SVM) on accuracy.

Molecular Classification Using Hyperdimensional Graph Classification

2024-06-30 · 4 citations

Our work introduces an innovative approach to graph learning by leveraging Hyperdimensional Computing. Graphs serve as a widely embraced method for conveying information, and their utilization in learning has gained significant attention. This is notable in the field of chemoinformatics, where learning from graph representations plays a pivotal role. An important application within this domain involves the identification of cancerous cells across diverse molecular structures. We propose an HDC-based model that demonstrates comparable Area Under the Curve results when compared to state-of-the-art models like Graph Neural Networks (GNNs) or the Weisfieler-Lehman graph kernel (WL). Moreover, it outperforms previously proposed hyperdimensional computing graph learning methods. Furthermore, it achieves noteworthy speed enhancements, boasting a 40x acceleration in the training phase and a 15x improvement in inference time compared to GNN and WL models. This not only underscores the efficacy of the HDC-based method, but also highlights its potential for expedited and resource-efficient graph learning.

Always-Sparse Training by Growing Connections with Guided Stochastic Exploration

arXiv (Cornell University) · 2024-01-12

The excessive computational requirements of modern artificial neural networks (ANNs) are posing limitations on the machines that can run them. Sparsification of ANNs is often motivated by time, memory and energy savings only during model inference, yielding no benefits during training. A growing body of work is now focusing on providing the benefits of model sparsification also during training. While these methods greatly improve the training efficiency, the training algorithms yielding the most accurate models still materialize the dense weights, or compute dense gradients during training. We propose an efficient, always-sparse training algorithm with excellent scaling to larger and sparser models, supported by its linear time complexity with respect to the model width during training and inference. Moreover, our guided stochastic exploration algorithm improves over the accuracy of previous sparse training methods. We evaluate our method on CIFAR-10/100 and ImageNet using ResNet, VGG, and ViT models, and compare it against a range of sparsification methods.

Enhanced Detection of Transdermal Alcohol Levels Using Hyperdimensional Computing on Embedded Devices

arXiv (Cornell University) · 2024-03-18 · 1 citations

Alcohol consumption has a significant impact on individuals' health, with even more pronounced consequences when consumption becomes excessive. One approach to promoting healthier drinking habits is implementing just-in-time interventions, where timely notifications indicating intoxication are sent during heavy drinking episodes. However, the complexity or invasiveness of an intervention mechanism may deter an individual from using them in practice. Previous research tackled this challenge using collected motion data and conventional Machine Learning (ML) algorithms to classify heavy drinking episodes, but with impractical accuracy and computational efficiency for mobile devices. Consequently, we have elected to use Hyperdimensional Computing (HDC) to design a just-in-time intervention approach that is practical for smartphones, smart wearables, and IoT deployment. HDC is a framework that has proven results in processing real-time sensor data efficiently. This approach offers several advantages, including low latency, minimal power consumption, and high parallelism. We explore various HDC encoding designs and combine them with various HDC learning models to create an optimal and feasible approach for mobile devices. Our findings indicate an accuracy rate of 89\%, which represents a substantial 12\% improvement over the current state-of-the-art.

Molecular Classification Using Hyperdimensional Graph Classification

arXiv (Cornell University) · 2024-03-18 · 2 citations

Our work introduces an innovative approach to graph learning by leveraging Hyperdimensional Computing. Graphs serve as a widely embraced method for conveying information, and their utilization in learning has gained significant attention. This is notable in the field of chemoinformatics, where learning from graph representations plays a pivotal role. An important application within this domain involves the identification of cancerous cells across diverse molecular structures. We propose an HDC-based model that demonstrates comparable Area Under the Curve results when compared to state-of-the-art models like Graph Neural Networks (GNNs) or the Weisfieler-Lehman graph kernel (WL). Moreover, it outperforms previously proposed hyperdimensional computing graph learning methods. Furthermore, it achieves noteworthy speed enhancements, boasting a 40x acceleration in the training phase and a 15x improvement in inference time compared to GNN and WL models. This not only underscores the efficacy of the HDC-based method, but also highlights its potential for expedited and resource-efficient graph learning.

Hyperdimensional computing: a framework for stochastic computation and symbolic AI

Journal Of Big Data · 2024-10-24 · 9 citations

Abstract Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures (VSA), is a neuro-inspired computing framework that exploits high-dimensional random vector spaces. HDC uses extremely parallelizable arithmetic to provide computational solutions that balance accuracy, efficiency and robustness. The majority of current HDC research focuses on the learning capabilities of these high-dimensional spaces. However, a tangential research direction investigates the properties of these high-dimensional spaces more generally as a probabilistic model for computation. In this manuscript, we provide an approachable, yet thorough, survey of the components of HDC. To highlight the dual use of HDC, we provide an in-depth analysis of two vastly different applications. The first uses HDC in a learning setting to classify graphs. Graphs are among the most important forms of information representation, and graph learning in IoT and sensor networks introduces challenges because of the limited compute capabilities. Compared to the state-of-the-art Graph Neural Networks, our proposed method achieves comparable accuracy, while training and inference times are on average 14.6× and 2.0× faster, respectively. Secondly, we analyse a dynamic hash table that uses a novel hypervector type called circular-hypervectors to map requests to a dynamic set of resources. The proposed hyperdimensional hashing method has the efficiency to be deployed in large systems. Moreover, our approach remains unaffected by a realistic level of memory errors which causes significant mismatches for existing methods.

Smartwatch-Based Prediction of Transdermal Alcohol Levels Using Hyperdimensional Computing

2024-11-10

Excessive alcohol consumption was responsible for 6% of global deaths in 2023. To encourage healthier drinking habits and enhance user awareness of their current condition, just-in-time interventions prove to be a suitable approach for informing users about their current state of intoxication. Current methods for determining blood alcohol content are intrusive and many also invasive, requiring users to use breathalizers or actively engage in urine or blood tests. In this study, we introduce an application utilizing Hyperdimensional Computing to predict if a user is under the influence of alcohol, achieving an accuracy of 93.5% on average. Furthermore, this application is designed to run on both smartphones and smartwatches, enabling full on device computation and online learning through a C implementation utilizing vectorial operations. The application has shown to be very efficient, having a training time per instance of 13.2 and 1.25ms on smartwatch and smartphone respectively and inference time of 6.8 and 1.1ms. Moreover the energy consumption of the running application is negligible compared to the energy usage of the idle device.