UCB-Type Learning Algorithms with Kaplan–Meier Estimator for Lost-Sales Inventory Models with Lead Times.

This article focuses on developing efficient upper confidence bound (UCB)-type learning algorithms incorporating the Kaplan–Meier (KM) estimator for optimizing periodic-review lost-sales inventory systems with lead times. It introduces two algorithms: UCB-KM-CBS for the capped base-stock (CBS) policy, which involves two parameters and lacks convexity, and UCB-KM-BS for the base-stock (BS) policy. The UCB-KM-CBS algorithm uses a novel combination of simulation along the base-stock dimension with the KM estimator to handle censored demand and averaging along the order cap dimension, achieving a regret bound tight in the planning horizon but with exponential dependence on lead time. In contrast, UCB-KM-BS attains a regret bound linear in lead time, matching the best existing results for base-stock policies. Extensive numerical experiments demonstrate that both algorithms outperform prior methods, and guidance is provided on selecting between them based on limited demand data.