Discrete Scan Statistics for Higher-Order Markovian Sequences

2024-01-01

Preface

Handbook of statistics · 2024 · 1 citations

• Mathematics

Fitting sparse Markov models through a collapsed Gibbs sampler

Computational Statistics · 2022 · 2 citations

• Computer Science
• Artificial Intelligence
• Machine Learning

Equivalence relations and inference for sparse Markov models

Handbook of statistics · 2022

• Computer Science
• Artificial Intelligence
• Machine Learning

Distributions of pattern statistics in sparse Markov models

Annals of the Institute of Statistical Mathematics · 2019-04-05 · 5 citations

Computation of exact probabilities associated with overlapping pattern occurrences

Wiley Interdisciplinary Reviews Computational Statistics · 2019-07-05 · 3 citations

Abstract Searching for patterns in data is important because it can lead to the discovery of sequence segments that play a functional role. The complexity of pattern statistics that are used in data analysis and the need of the sampling distribution of those statistics for inference renders efficient computation methods as paramount. This article gives an overview of the main methods used to compute distributions of statistics of overlapping pattern occurrences, specifically, generating functions, correlation functions, the Goulden‐Jackson cluster method, recursive equations, and Markov chain embedding. The underlying data sequence will be assumed to be higher‐order Markovian, which includes sparse Markov models and variable length Markov chains as special cases. Also considered will be recent developments for extending the computational capabilities of the Markov chain‐based method through an algorithm for minimizing the size of the chain's state space, as well as improved data modeling capabilities through sparse Markov models. An application to compute a distribution used as a test statistic in sequence alignment will serve to illustrate the usefulness of the methodology. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Pattern Recognition Data: Types and Structure > Categorical Data Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms

Minimal auxiliary Markov chains through sequential elimination of states

Communications in Statistics - Simulation and Computation · 2018-02-09 · 7 citations

When using an auxiliary Markov chain to compute the distribution of a pattern statistic, the computational complexity is directly related to the number of Markov chain states. Theory related to minimal deterministic finite automata have been applied to large state spaces to reduce the number of Markov chain states so that only a minimal set remains. In this paper, a characterization of equivalent states is given so that extraneous states are deleted during the process of forming the state space, improving computational efficiency. The theory extends the applicability of Markov chain based methods for computing the distribution of pattern statistics.

Discrete Scan Statistics for Higher-Order Markovian Sequences

2018-11-05

Multiple window discrete scan statistic for higher-order Markovian sequences

Journal of Applied Statistics · 2015-01-25 · 3 citations

Accurate and efficient methods to detect unusual clusters of abnormal activity are needed in many fields such as medicine and business. Often the size of clusters is unknown; hence, multiple (variable) window scan statistics are used to identify clusters using a set of different potential cluster sizes. We give an efficient method to compute the exact distribution of multiple window discrete scan statistics for higher-order, multi-state Markovian sequences. We define a Markov chain to efficiently keep track of probabilities needed to compute p -values for the statistic. The state space of the Markov chain is set up by a criterion developed to identify strings that are associated with observing the specified values of the statistic. Using our algorithm, we identify cases where the available approximations do not perform well. We demonstrate our methods by detecting unusual clusters of made free throw shots by National Basketball Association players during the 2009-2010 regular season.

Faster exact distributions of pattern statistics through sequential elimination of states

Annals of the Institute of Statistical Mathematics · 2015-09-18 · 9 citations