A LIMITING DESCRIPTION OF CONTROL IN A GAUSSIAN ONE-ARMED BANDIT PROBLEM

A. V. Kolnogorov

Abstract


We consider a limiting description of control in the Gaussian one-armed bandit problem in application to batch processing of big data, if estimates of unknown mathematical expectation and variance of one-step incomes are performed during data processing within batches. This description is given by a second-order partial differential equation in which the estimate of the unknown variance is present as a constant parameter. This result means that when processing big data, an arbitrarily accurate estimate of the unknown variance can be obtained at a relatively arbitrarily short initial stage, and then used for control.

Keywords


Gaussian one-armed bandit; batch processing; Bayesian approach; invariant description

Full Text:

PDF

References


Kolnogorov A.V. Optimization of Two-Alternative Batch Processing with Parameter Estimation Based on Data Inside Batches. Journal of Computational and Engineering Mathematics, 2023, vol. 10, no. 4, pp. 40-50. DOI: 10.14529/jcem230403

Kolnogorov A.V.

Invariant Description of Control in a Gaussian One-Armed Bandit

Problem. Bulletin of the South Ural State University. Series: Mathematical Modelling, Programming and Computer Software, 2024, vol. 17, no. 1, pp. 27-36. DOI: 10.14529/mmp240103

Sragovich V.G. Mathematical Theory of Adaptive Control, Singapore, World Sci., 2006.

Slivkins A. Introduction to Multi-Armed Bandits, arXiv:1904.07272v7, 2022.

Johnson S.M., Karlin S. On Sequential Designs for Maximizing the Sum of $n$ Observations. The Annals of Mathematical Statistics, 1956, vol. 27, pp. 1060-1074. DOI: 10.1214/aoms/1177728073

Chernoff H., Ray, S.N. A Bayes Sequential Sampling Inspection Plan. The Annals of Mathematical Statistics, 1965, vol. 36, pp. 1387-1407. DOI: 10.1214/aoms/1177699898

Kolnogorov A.V. A Limiting Description in a Gaussian One-Armed Bandit Problem with Both Unknown Parameters. Bulletin of the South Ural State University. Series: Mathematical Modelling, Programming and Computer Software, 2025, vol. 18, no. 1, pp. 35-45. DOI: 10.14529/mmp250103


Refbacks

  • There are currently no refbacks.


 Save