Cooperative Multi Agent Learning System

Abstract

Coordination in cooperative multi-agent systems is an important problem in multi-agent learning and has been studied a lot in the literature. In this work, we investigate the multi-agent coordination problems in cooperative environments under the networked multi-agent learning framework using some social network structures and will try to improve coordination efficiency. A networked multi-agent learning framework consists of a population of agents where each agent interacts with another agent randomly selected from its neighborhood in each round. Each agent updates its learning policy through repeated interactions with its neighbors via both individual learning and social learning. It is not clear a priori whether all agents are able

These agents may be computer programs, robots, or even humans. They can be used to solve problems that are difficult or impossible for an individual agent or a monolithic system to solve[19].

A cooperative multi-agent system (CMAS) is composed of a set of autonomous agents that interact with one another in a shared environment. In order to successfully interact, these agents in MAS will thus need the ability to cooperate, coordinate, and negotiate with other resident agents, in much the same manner we cooperate, coordinate, and negotiate with other people in our daily lives. One fundamental property of an agent in a multi-agent system is its ability of adaptively adjusting its behaviors in response to other agents in order to achieve effective coordination on desirable outcomes since the outcome not only depends on the action it takes but also the actions were taken by other agents that it interacts with. In cooperative MASs, the agents share common interests (e.g., the same reward function), thus the increase in individual's benefit also leads to the increase of the benefits of the whole group. Hao and Leung[6] were the first who proposed a multi-agent social learning framework to investigate multi-agent coordination problem in cooperative games assuming that the agents' interactions are random. In their recent work, they considered

Two different types of learners (IALs and JALs) based on the traditional Q-learning algorithm were introduced by incorporating both heuristics of optimal assumption and Frequency Maximum Q value(FMQ)[13] strategy. They concluded that for deterministic cooperative games, both IALs and JALs could effectively learn to coordinate with optimal joint actions without significant performance difference, however, when it comes to stochastic cooperative games, JALs usually can achieve much better performance than IALs, since it could better distinguish between the stochasticity of the game itself and the stochastic explorations of the interacting