Offline Reinforcement Learning for Human-Guided Human-Machine Interaction with Private Information.

This article focuses on developing an offline reinforcement learning (RL) framework for human-guided human-machine interaction modeled as a two-player turn-based cooperative game with private information. The key challenge addressed is the presence of unobserved confounding due to private information inaccessible to the machine (Alice) about the human (Bob), which biases standard RL methods, alongside distributional mismatch in offline data. To overcome this, the authors treat the other player's previous action as an instrumental variable (IV), allowing for nonparametric identification and off-policy evaluation of policy pairs despite invalid IV conditions where exclusion restrictions fail. They propose a pessimistic policy learning algorithm that leverages these IV-based evaluations to find an optimal policy pair, proving convergence rates under mild assumptions and demonstrating effectiveness through simulation in a video recommendation system example. The work bridges causal inference and cooperative game theory, offering a foundation for future extensions to multiagent settings and real-world applications.