Dual-value reinforcement learning with delayed local welfare feedback in spatial public goods games

Cooperation is difficult to sustain in public-goods dilemmas because contributing can reduce an individual’s immediate payoff, while benefits to the surrounding group may emerge only after several rounds. We introduce a dual-value reinforcement-learning model for a spatial public goods game that evaluates these two consequences separately. One value system learns from the individual payoff obtained after each action, whereas the other learns from local welfare accumulated over multiple rounds. The two evaluations are combined only when an action is selected. Within the tested settings, numerical simulations show that cooperation is best supported when neither evaluation fully dominates. A moderate welfare-feedback horizon produces higher cooperation, fewer strategy changes, and stronger agreement between the two value systems than one-step or excessively long feedback. The results further show that increasing the influence or duration of social evaluation does not improve cooperation indefinitely. Stable cooperation instead depends on coordinating immediate individual incentives with delayed neighborhood-level consequences, rather than replacing payoff-oriented learning with social valuation.