Publications

2026

1. 

   Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward

   Ş.K. Alkır^†, N. Saldi, B. Anahtarci, and 1 more author

   Preprint, , Jun 2026

   Abs DOI Bib PDF

   We study inverse reinforcement learning for discrete-time, infinite-horizon mean-field games (MFGs) under an average-reward criterion. Expert demonstrations are assumed to arise from a stationary mean-field equilibrium under an unknown reward, and the goal is to recover a policy explaining the observed behaviour via the maximum causal entropy principle. We formulate the inverse problem by enforcing consistency with the expert mean-field term and long-run feature expectations, treating two reward classes within a unified occupation-measure framework. For finite-dimensional linear rewards, we give a convex dual reformulation with an explicit log-partition objective, and prove smoothness and curvature properties justifying constant-step-size gradient descent. For infinite-dimensional RKHS rewards, we develop a Lagrangian relaxation whose inner-maximising policy is characterised by a soft Bellman equation. The main obstacle is the absence of a discount-factor contraction. We resolve this by introducing a minorisation-based sub-stochastic kernel that yields a strict contraction of the soft Bellman operator. We establish Fréchet differentiability and Lipschitz smoothness of the log-likelihood score, leading to a gradient ascent algorithm with convergence guarantees. Two numerical examples, a malware-spread MFG and an RKHS-based consumer-choice model, show that the recovered policies closely match expert behaviour.

   @article{AlkırSal25,
     title = {Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward},
     author = {Alkır, Ş.K. and Saldi, N. and Anahtarci, B. and D., Kariksiz C.},
     month = jun,
     year = {2026},
     journal = {Preprint},
     location = {},
     doi = {https://doi.org/10.48550/arXiv.2606.16759}
   }

2025

1. Inverse Reinforcement Learning for Mean-field Games with Average Reward Criterion

   Ş.K. Alkır^† and N. Saldi

   2025 IEEE 64th Conference on Decision and Control (CDC), Rio de Janeiro, Brazil, Dec 2025

   Abs DOI Bib PDF

   We study the inverse reinforcement learning (IRL) problem for discrete-time, infinite-horizon mean-field games (MFG) with an average-reward criterion. Unlike the forward setting, where the reward function is known, IRL assumes access only to expert demonstrations that are optimal under some unknown reward function. The objective is to recover the reward structure that explains these expert behaviors. Our approach is based on the maximum causal entropy principle, which selects the least biased policy among those consistent with the observed demonstrations. We show that the resulting non-convex formulation is equivalently reformulated as a convex optimization problem over occupation measures. Furthermore, we establish that the dual objective is smooth and strongly convex over compact sets and derive a variational representation using a log-partition formulation. Finally, we propose a first-order algorithm for solving the dual problem and recovering an entropy-maximizing equilibrium policy.

   @article{AlkSal25,
     title = {Inverse Reinforcement Learning for Mean-field Games with Average Reward Criterion},
     author = {Alkır, Ş.K. and Saldi, N.},
     month = dec,
     year = {2025},
     pages = {7272-7277},
     journal = {2025 IEEE 64th Conference on Decision and Control (CDC)},
     location = {Rio de Janeiro, Brazil},
     doi = {10.1109/CDC57313.2025.11312818}
   }