About

Vasileios P. Kemerlis is an Associate Professor of Computer Science at Brown University, specializing in software, hardware, and systems security. His research encompasses a broad range of topics including operating system kernel protection, software hardening, fuzz testing, and information flow tracking. He leads the Secure Systems Lab (SSL) and serves as the Academic Director of Brown's Master of Science in Cybersecurity program. Professor Kemerlis has been actively involved in teaching courses related to software security and exploitation, computer security, and advanced topics in software security over multiple years. His work focuses on advancing the security of computing systems through innovative research in kernel isolation, memory protection, system call filtering, and defenses against various classes of attacks such as speculative execution and deserialization vulnerabilities.

Research topics

• Computer Science
• Operating system
• Distributed computing
• Programming language
• Parallel computing

Selected publications

• TENNOR: Trustworthy Execution for Neural Networks through Obliviousness and Retrievals

  arXiv (Cornell University) · 2026-05-08

  preprintOpen access

  Training wide neural networks on sensitive data in untrusted cloud environments requires simultaneously achieving computational efficiency and rigorous privacy guarantees. Sparsification techniques, essential for scalable training of wide layers, expose input-dependent memory-access patterns (i.e., leakage) that are visible and can be exploited by a host OS/hypervisor, even when computation is protected by a Trusted Execution Environment. We present TENNOR, a system that resolves this tension by co-designing the neural network training pipeline with doubly oblivious primitives, eliminating access-pattern leakage while also utilizing adaptive sparsification. TENNOR recasts sparse neuron activation as a locality-sensitive hashing (LSH) retrieval problem, reducing secure sparsification to doubly oblivious accesses over an LSH data structure. To eliminate the prohibitive storage cost of ``multi-table'' LSH, we introduce Multi-Probe Winner-Take-All (MP-WTA): the first multi-probe scheme for rank-based LSH, achieving a 50x reduction in (hash table) memory while preserving model accuracy. We evaluate TENNOR on extreme multi-label classification benchmarks with output layers of up to 325K neurons inside an Intel TDX Trusted Domain, achieving speedups of 13x--470x over a Path ORAM baseline and reducing a 208-hour run to about 26 minutes.

  PublisherDOI

• TENNOR: Trustworthy Execution for Neural Networks through Obliviousness and Retrievals

  ArXiv.org · 2026-05-08

  articleOpen access

  Training wide neural networks on sensitive data in untrusted cloud environments requires simultaneously achieving computational efficiency and rigorous privacy guarantees. Sparsification techniques, essential for scalable training of wide layers, expose input-dependent memory-access patterns (i.e., leakage) that are visible and can be exploited by a host OS/hypervisor, even when computation is protected by a Trusted Execution Environment. We present TENNOR, a system that resolves this tension by co-designing the neural network training pipeline with doubly oblivious primitives, eliminating access-pattern leakage while also utilizing adaptive sparsification. TENNOR recasts sparse neuron activation as a locality-sensitive hashing (LSH) retrieval problem, reducing secure sparsification to doubly oblivious accesses over an LSH data structure. To eliminate the prohibitive storage cost of ``multi-table'' LSH, we introduce Multi-Probe Winner-Take-All (MP-WTA): the first multi-probe scheme for rank-based LSH, achieving a 50x reduction in (hash table) memory while preserving model accuracy. We evaluate TENNOR on extreme multi-label classification benchmarks with output layers of up to 325K neurons inside an Intel TDX Trusted Domain, achieving speedups of 13x--470x over a Path ORAM baseline and reducing a 208-hour run to about 26 minutes.

  PublisherOA PDF

• NOPoutNG: Improving the Effectiveness of Hardware-assisted Control-flow Integrity via Dynamic Landing Pad Elision

  2025-12-08

  articleSenior author
  PublisherDOI

• P <scp>ickle</scp> B <scp>all</scp> : Secure Deserialization of Pickle-based Machine Learning Models

  2025-11-19 · 1 citations

  article
  PublisherDOI

• PickleBall: Secure Deserialization of Pickle-based Machine Learning Models (Extended Report)

  ArXiv.org · 2025-08-21

  preprintOpen access

  Machine learning model repositories such as the Hugging Face Model Hub facilitate model exchanges. However, bad actors can deliver malware through compromised models. Existing defenses such as safer model formats, restrictive (but inflexible) loading policies, and model scanners have shortcomings: 44.9% of popular models on Hugging Face still use the insecure pickle format, 15% of these cannot be loaded by restrictive loading policies, and model scanners have both false positives and false negatives. Pickle remains the de facto standard for model exchange, and the ML community lacks a tool that offers transparent safe loading. We present PickleBall to help machine learning engineers load pickle-based models safely. PickleBall statically analyzes the source code of a given machine learning library and computes a custom policy that specifies a safe load-time behavior for benign models. PickleBall then dynamically enforces the policy during load time as a drop-in replacement for the pickle module. PickleBall generates policies that correctly load 79.8% of benign pickle-based models in our dataset, while rejecting all (100%) malicious examples in our dataset. In comparison, evaluated model scanners fail to identify known malicious models, and the state-of-art loader loads 22% fewer benign models than PickleBall. PickleBall removes the threat of arbitrary function invocation from malicious pickle-based models, raising the bar for attackers to depend on code reuse techniques.

  PublisherOA PDFDOI

• IUBIK: Isolating User Bytes in Commodity Operating System Kernels via Memory Tagging Extensions

  2025-05-12

  articleSenior author

  Hardening OS kernels against memory errors is generally addressed by protecting security-critical data against corruption and disclosure. However, establishing a sound model for identifying sensitive memory objects in need of protection is hard, leading to emergent attack vectors that can be abused by attackers. In this paper, we propose rethinking how OS kernels are hardened by introducing IUBIK for compartmentalizing kernel memory. IUBIK prevents kernel exploitation by segregating attacker-controlled data-frequently used to manipulate security-critical data-in shadow memory, preventing it from interacting with sensitive kernel objects. To achieve this, IUBIK uses MTE: a recent hardware feature, available in ARM CPUs, which allows mitigating exploits based on both spatial and temporal memory-errors, efficiently. We ensure that segregated objects do not contain sensitive fields, such as pointers, by rewriting their struct definitions. Moreover, we develop a profiling framework that explores the kernel codebase in-depth and records code sites where attacker-controlled objects are allocated, allowing IUBIK to isolate them; our profiler recorded 292 privileged and 212 non-privileged allocation sites for a diverse set of workloads. Finally, we evaluate an implementation of IUBIK for the Linux kernel, across a suite of micro- and macro-benchmarks, demonstrating that our prototype incurs no runtime overhead in most tests and negligible additional memory consumption.

  PublisherDOI

• ISLAB: Immutable Memory Management Metadata for Commodity Operating System Kernels

  2024-06-28 · 4 citations

  articleSenior author

  Kernel memory allocators maintain several metadata structures optimized for efficiently managing system memory. However, existing implementations adopt either weak or no protection at all to ensure the integrity of said metadata in the presence of memory errors. In this paper, we first demonstrate how existing memory hardening schemes fall short against several in-kernel memory corruption scenarios. We then present ISLAB: a set of novel (slab-based) heap hardening techniques that aim to ensure the integrity of the memory managed by the kernel, and minimize the incurred runtime, and memory, overhead. ISLAB prevents memory corruption exploits by segregating metadata from within corruptible memory objects into shadow memory. It also relies on a novel SMAP-assisted memory isolation framework, called kSMAP, to protect allocator metadata against adversaries with stronger memory access capabilities. We implemented and evaluated ISLAB atop SLUB, the default slab allocator in Linux, and equipped it with kSMAP to protect process credentials, a popular target in kernel exploitation. Our experiments show that ISLAB incurs no runtime overhead in realistic benchmarks, and moderate overhead in stress tests. Lastly, we show how ISLAB's approach can be generalized to protect the integrity of other kernel subsystems that use corruptible metadata for memory management, such as linked lists.

  PublisherDOI

• Safeslab: Mitigating Use-After-Free Vulnerabilities via Memory Protection Keys

  2024-12-02 · 3 citations

  articleOpen accessSenior author

  Restricting dangling pointers from accessing freed memory is a promising technique for mitigating use-after-free vulnerabilities in memory-unsafe programming languages. However, existing solutions suffer from high performance overheads, as they rely on conventional page table manipulation to make dangling pointers inaccessible. In this paper, we present Safeslab: a heap-hardening extension that aims to mitigate use-after-free vulnerabilities via a novel and efficient address aliasing approach. Safeslab assigns multiple virtual aliases to each memory page in the system, and manages their access rights via the recently introduced Memory Protection Keys hardware extension, which is designed to provide a fast alternative to page tables for memory management. This allows Safeslab to drastically reduce the number of page table modifications, while blocking dangling pointers efficiently. We integrated Safeslab into the Linux kernel, replacing its default heap allocator (SLUB). The results of our experimental evaluation with real-world benchmarks show that Safeslab incurs a negligible runtime overhead of up to 4% and moderate memory waste.

  PublisherOA PDFDOI

• QUACK: Hindering Deserialization Attacks via Static Duck Typing

  2024-01-01 · 4 citations

  articleOpen access

  Managed languages facilitate convenient ways for serializing objects, allowing applications to persist and transfer them easily, yet this feature opens them up to attacks.By manipulating serialized objects, attackers can trigger a chained execution of existing code segments, using them as gadgets to form an exploit.Protecting deserialization calls against attacks is cumbersome and tedious, leading to many developers avoiding deploying defenses properly.We present QUACK, a framework for automatically protecting applications by fixing calls to deserialization APIs.This "binding" limits the classes allowed for usage in the deserialization process, severely limiting the code available for (ab)use as part of exploits.QUACK computes the set of classes that should be allowed using a novel static duck typing inference technique.In particular, it statically collects all statements in the program code that manipulate objects after they are deserialized, and puts together a filter for the list of classes that should be available at runtime.We have implemented QUACK for PHP and evaluated it on a set of applications with known CVEs, and popular applications crawled from GitHub.QUACK managed to fix the applications in a way that prevented any attempt at automatically generating an exploit against them, by blocking, on average, 97% of the application's code that could be used as gadgets.We submitted a sample of three fixes generated by QUACK as pull requests, and their developers merged them.

  PublisherDOI

• Eclipse: Preventing Speculative Memory-error Abuse with Artificial Data Dependencies

  2024-12-02 · 2 citations

  articleOpen accessSenior author

  Historically, researchers have treated memory safety-based and speculative execution attacks as two separate domains. Recent work has introduced Speculative Memory-error Abuse (SMA) attacks, which combine memory corruption vulnerabilities with Spectre-like primitives. Using SMA, an attacker can leak sensitive program information and defeat a wide variety of memory-corruption mitigations, including (K)ASLR, software-based XOM, and even ARM PA, eventually carrying out an end-to-end (architecturally-visible) exploit. We present Eclipse: a novel protection scheme against SMA attacks. Eclipse works by propagating artificial data dependencies onto sensitive data, preventing the CPU from using attacker-controlled data during speculative execution. We demonstrate that Eclipse provides comprehensive protection against speculative-probing and Pacman-style attacks, two prominent examples of Speculative Memory-error Abuse attacks that target both the x86(-64) and ARM architectures. We evaluate the performance of Eclipse on x86-64 and demonstrate that it introduces minimal overhead, compared to alternative hardening approaches, incurring ≈0%--9.5% slowdown on SPEC CPU 2017, up to 8.6% slowdown in real-world applications, and negligible overhead in the Linux kernel.

  PublisherOA PDFDOI

Recent grants

• CAREER: Countering Emerging Software Threats with Adaptive Hardening, Debloating, and Hardware-assisted Protection

  NSF · $383k · 2023–2028

Frequent coauthors

• Angelos D. Keromytis

  Georgia Institute of Technology

  26 shared

• Michalis Polychronakis

  Stony Brook University

  14 shared

• Georgios Portokalidis

  IMDEA Software

  14 shared

• Vaggelis Atlidakis

  Brown University

  11 shared

• Alexander J. Gaidis

  Brown University

  10 shared

• Kent Williams-King

  Brown University

  10 shared

• Simha Sethumadhavan

  Columbia University

  9 shared

• Nicholas DeMarinis

  John Brown University

  8 shared

Education

• PhD, Computer Science

  Columbia University

  2015

• MPhil, Computer Science

  Columbia University

  2013

• MSc, Computer Science

  Columbia University

  2010

• BSc, Computer Science

  Athens University of Economics and Business

  2006