Ryan Marcus

Stage: Query Execution Time Prediction in Amazon Redshift

2024-05-23 · 18 citations

Query performance (e.g., execution time) prediction is a critical component of modern DBMSes. As a pioneering cloud data warehouse, Amazon Redshift relies on an accurate execution time prediction for many downstream tasks, ranging from high-level optimizations, such as automatically creating materialized views, to low-level tasks on the critical path of query execution, such as admission, scheduling, and execution resource control. Unfortunately, many existing execution time prediction technique…

Learned Offline Query Planning via Bayesian Optimization

Proceedings of the ACM on Management of Data · 2025-06-17 · 4 citations

Analytics database workloads often contain queries that are executed repeatedly. Existing optimization techniques generally prioritize keeping optimization cost low, normally well below the time it takes to execute a single instance of a query. If a given query is going to be executed thousands of times, could it be worth investing significantly more optimization time? In contrast to traditional online query optimizers, we propose an offline query optimizer that searches a wide variety of plans…

Low Rank Learning for Offline Query Optimization

Proceedings of the ACM on Management of Data · 2025-06-17 · 2 citations

Recent deployments of learned query optimizers use expensive neural networks and ad-hoc search policies. To address these issues, we introduce LimeQO, a framework for offline query optimization leveraging low-rank learning to efficiently explore alternative query plans with minimal resource usage. By modeling the workload as a partially observed, low-rank matrix, we predict unobserved query plan latencies using purely linear methods, significantly reducing computational overhead compared to neur…

ScaleLLM: A Technique for Scalable LLM-augmented Data Systems

2025-06-17 · 1 citations

Large language models (LLMs) offer powerful semantic insights for data analytics, but row-by-row LLM calls quickly become prohibitively expensive in large datasets. We introduce ScaleLLM, a novel system that substantially reduces both latency and cost on text classification tasks. ScaleLLM couples LLM-generated labels on a small subset of data with a lightweight machine learning model for large-scale inference. This approach provides significant speed-ups-up to 37×-while maintaining accuracy clo…

BFTBrain: Adaptive BFT Consensus with Reinforcement Learning

arXiv (Cornell University) · 2024-08-12 · 1 citations

This paper presents BFTBrain, a reinforcement learning (RL) based Byzantine fault-tolerant (BFT) system that provides significant operational benefits: a plug-and-play system suitable for a broad set of hardware and network configurations, and adjusts effectively in real-time to changing fault scenarios and workloads. BFTBrain adapts to system conditions and application needs by switching between a set of BFT protocols in real-time. Two main advances contribute to BFTBrain's agility and performa…