Deep Reinforcement Learning in Agent-Based Model AgriPoliS to Simulate Strategic Land Market Interactions

Abstract

AgriPoliS (Agricultural Policy Simulator) is an Agent-Based Model for simulating the dynamic evolution of agricultural regions with a particular emphasis on the structural transformations influenced by economic, ecological, and societal factors. This study introduces a Reinforcement Learning (RL) framework to empower agents within AgriPoliS with strategic decision-making capabilities, specifically in the context of land market bidding. Traditional agents within AgriPoliS make decisions through Mixed Integer Programming (MIP) optimizations, inherently limited by myopic considerations of the current year's conditions. In this framework one agent within AgriPoliS is defined as the RL agent, with AgriPoliS itself serving as the learning environment. The RL agent's bidding coefficient are formulated as actions, with the state space comprising various agent and regional variables, including liquidity, farm endowments, remaining land, and neighbouring farms. We adopt the cumulative equity capital of the agent at the end of the simulation as the reward from the environment, reflecting a holistic perspective towards stable and sustainable development. Utilizing a policy gradient algorithm, we update the RL agent's policy network to optimize bidding strategies. Training results demonstrate that the RL-enhanced agent consistently outperforms its non-learning counterpart, exhibiting strategically stable bidding behavior. To assess the robustness of the algorithm, extensive training runs are conducted with varying hyperparameters, including policy network initialization, learning rate, and exploration noise levels. Our findings underscore the efficacy of RL in enhancing strategic decision-making in agricultural land markets, showcasing superior performance compared to fixed bidding strategies.