Dynamical Linear Reward Systems under Competitive Horizon Criteria

We consider reward systems defined as iterative decision-making processes, where a player selects an action from the unit interval, and the environment responds by choosing a reward function from a known set of functions. The goal of the player is to accumulate rewards that exceed a given threshold in minimal time, and the performance is measured via regret with respect to an optimal player who knows the entire sequence of reward functions in advance. The central challenge lies in the dynamical nature of the reward system: each time step may involve a different reward function, requiring the player's policy to adapt over time and making the regret an infinite-letter optimization problem. Our main result is an explicit expression for the optimal regret in the case of two linear reward functions that have opposing slopes. Moreover, we show that the optimal regret is achieved by a piecewise-constant action sequence, where both the transition times and action values exhibit special structural properties. These properties seem fundamental and may extend to classes of nonlinear reward functions. Finally, we highlight the implications of our solution in the context of communication, particularly, in characterizing the capacity of arbitrarily varying channels (AVCs) under competitive performance criteria.