Simple analytical models for estimating the queue lengths from probe vehicles at traffic signals: A combinatorial approach for nonparametric models

Expert Systems with Applications · 2024-04-24 · 6 citations

Partial-Information Q-Learning for General Two-Player Stochastic Games

arXiv (Cornell University) · 2023-02-21 · 1 citations

In this article we analyze a partial-information Nash Q-learning algorithm for a general 2-player stochastic game. Partial information refers to the setting where a player does not know the strategy or the actions taken by the opposing player. We prove convergence of this partially informed algorithm for general 2-player games with finitely many states and actions, and we confirm that the limiting strategy is in fact a full-information Nash equilibrium. In implementation, partial information offers simplicity because it avoids computation of Nash equilibria at every time step. In contrast, full-information Q-learning uses the Lemke-Howson algorithm to compute Nash equilibria at every time step, which can be an effective approach but requires several assumptions to prove convergence and may have runtime error if Lemke-Howson encounters degeneracy. In simulations, the partial information results we obtain are comparable to those for full-information Q-learning and fictitious play.

A Mean-Field Control Problem and Application to Inter-Bank Lending and Borrowing

Dynamic Systems and Applications · 2021-07-01

Optimal asset allocation with restrictions on liquidity

Stochastic Analysis and Applications · 2021-08-17

An optimal asset allocation problem involving restrictions on liquidity is studied in this article. The portfolio consists of liquid and illiquid asset. The portfolio is only allowed to rebalance at particular times. An investor tries to maximize the total utility of a hyperbolic absolute risk aversion function depending on the consumption, which is sourced only from the liquid asset. The optimal policies of the consumption, investment, and allocation are derived. A numerical approximation scheme is developed to show the optimal allocation policy in our model is path-dependent. Paths of the value function and other optimal controls are illustrated to validate our results.

A Portfolio Rebalancing Problem

Dynamic Systems and Applications · 2021-09-05

Bayesian Parameter Estimations for Grey System Models in Online Traffic Speed Predictions

arXiv (Cornell University) · 2021-08-15

This paper presents Bayesian parameter estimation for first order Grey system models' parameters (or sometimes referred to as hyperparameters). There are different forms of first-order Grey System Models. These include $GM(1,1)$, $GM(1,1| \cos(ωt)$, $GM(1,1| \sin(ωt)$, and $GM(1,1| \cos(ωt), \sin(ωt)$. The whitenization equation of these models is a first-order linear differential equation of the form \[ \frac{dx}{dt} + a x = f(t) \] where $a$ is a parameter and $f(t) = b$ in $GM(1,1|)$ , $f(t) = b_1\cos(ωt) + b_2$ in $GM(1,1| cos(ωt)$, $f(t) = b_1\sin(ωt)+b_2$ in $GM(1,1| \sin(ωt)$, $f(t) = b_1\sin(ωt) + b_2\cos(ωt) + b_3$ in $GM(1,1| \cos(ωt), \sin(ωt)$, $f(t) = b x^2$ in Grey Verhulst model (GVM), and where $b, b_1, b_2$, and $b_3$ are parameters. The results from Bayesian estimations are compared to the least square estimated models with fixed $ω$. We found that using rolling Bayesian estimations for GM parameters can allow us to estimate the parameters in all possible forms. Based on the data used for the comparison, the numerical results showed that models with Bayesian parameter estimations are up to 45\% more accurate in mean squared errors.

Study of Optimal Power Control by Cooperative Game for Satellite Communication Subsystems

2022 7th International Conference on Communication and Electronics Systems (ICCES) · 2021 · 2 citations

• Computer Science
• Computer Science
• Telecommunications

The multiple user terminals in a satellite transponder's communication channel would be competing for limited radio resources to meet their selfish data rates. The interference among users also limits the performance of a transponder. The spectrum and power control system of a transponder needs to coordinate all users to reduce interference and maximize the overall performance of its channel. This paper investigates optimal control strategies for a transponder to allocate power among competing user terminals. Because of centralization of this type of channel, a discrete cooperative game model is set up to study how to coordinate its competitive users. The game model is then decomposed to a nonlinear optimization problem, so its KKT condition is analyzed and solved analytically and numerically to get optimal solutions under symmetric conditions. At last, an optimal power control scheme in terms of the channel gain and the power noise ratio is provided.

A novel kernel-free least squares twin support vector machine for fast and accurate multi-class classification

Knowledge-Based Systems · 2021 · 45 citations

• Artificial Intelligence
• Computer Science
• Artificial Intelligence

Bayesian Parameter Estimations for Grey System Models in Online Traffic\n Speed Predictions

arXiv (Cornell University) · 2021-08-15

This paper presents Bayesian parameter estimation for first order Grey system\nmodels' parameters (or sometimes referred to as hyperparameters). There are\ndifferent forms of first-order Grey System Models. These include $GM(1,1)$,\n$GM(1,1| \\cos(\\omega t)$, $GM(1,1| \\sin(\\omega t)$, and $GM(1,1| \\cos(\\omega\nt), \\sin(\\omega t)$. The whitenization equation of these models is a\nfirst-order linear differential equation of the form \\[ \\frac{dx}{dt} + a x =\nf(t) \\] where $a$ is a parameter and $f(t) = b$ in $GM(1,1|)$ , $f(t) =\nb_1\\cos(\\omega t) + b_2$ in $GM(1,1| cos(\\omega t)$, $f(t) = b_1\\sin(\\omega\nt)+b_2$ in $GM(1,1| \\sin(\\omega t)$, $f(t) = b_1\\sin(\\omega t) + b_2\\cos(\\omega\nt) + b_3$ in $GM(1,1| \\cos(\\omega t), \\sin(\\omega t)$, $f(t) = b x^2$ in Grey\nVerhulst model (GVM),\n and where $b, b_1, b_2$, and $b_3$ are parameters. The results from Bayesian\nestimations are compared to the least square estimated models with fixed\n$\\omega$. We found that using rolling Bayesian estimations for GM parameters\ncan allow us to estimate the parameters in all possible forms. Based on the\ndata used for the comparison, the numerical results showed that models with\nBayesian parameter estimations are up to 45\\% more accurate in mean squared\nerrors.\n

A kernel-free double well potential support vector machine with applications

European Journal of Operational Research · 2020 · 45 citations

• Computer Science
• Artificial Intelligence
• Computer Science