Abstract

In this paper, a new algorithm based on case base reasoning and reinforcement learning is proposed to increase the rate convergence of the reinforcement learning algorithms in multi-agent systems. In the propose method, we investigate how making improved action selection in reinforcement learning (RL) algorithm. In the proposed method, the new combined model using case base reasoning systems and a new optimized function has been proposed to select the action, which has led to an increase in algorithms based on Q-learning. The algorithm mentioned has been used for solving the problem of cooperative Markov’s games as one of the models of Markov based multi-agent systems. The results of experiments have shown that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy.

References

• R. A. C. Branchi, R. Raquel, R. L. D. Mantaras, ” Imroving Reinforcement Learning by using Case Based Heuristics”, Proceeding of the Int. Conference on Case Based Learning 2009 (ICCBR 2009), Springer , 2009.
• N. Vlassis, “A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence”, 2007, Morgan and Claypool Publishers.
• C. Boutilier, "Sequential optimality and coordination in multi-agent systems", in: Proceedings of the 16th International joint conference on Artificial intelligence, 1999 , Vol. 1, Morgan Kaufmann Publishers Inc., Stockholm, Sweden.
• L. Bosniu, R. Babuska, and B. Schutter, "A Comprehensive Survey of Multiagent Reinforcement Learning", IEEE Transaction on System, Man, Cybern, 2008 ,vol. 38, pp. 156-171.
• B. Masoumi, M. R. Meybodi, “Speeding up learning automata based multi agent systems using the concepts of stigmergy and entropy”, Journal of Expert Systems with Applications, July 2011, Vol 38, Issue 7, PP. 8105-8118.
• M. Lauer and M. Riedmiller, "An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems", in The 17th International Conference on Machine Learning San Francisco, CA, USA, 2000: Morgan Kaufmann Publishers Inc, pp. 535 – 542.
• J. Hu, M. Wellman, "Nash Q-Learning for General-Sum Stochastic Games", Journal of Machine Learning Research, , 2003, vol. 4, pp. 1039-1069.
• X. Wang and T. Sandholm, "Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games", in Advances in Neural Information Processing Systems, 2002, vol. 15: MIT Press, pp. 1571-1578, 2002,
• M. Song , G. Gu and G. Zhang , “ Pareto-Q Learning Algorithm for Cooperative Agents in General Sum Games”, In Multiagent Systems and Applications , 2005, Vol.3690 : Springer, Berlin/Heidelberg , pp.576-578.
• L. Matignon , G. J. Lauent and N. L. Front-part , “ Hysteretic Q-Learning: An Algorithm for Decentralized Reinforcement Learning in Cooperative Multi-agent Teams “ , In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems IROS , San Diego , CA , USA, Nov. 2007, PP.64-69.
• F. S. Melo, M. I. Ribeiro, “Reinforcement Learning with Function Approximation for Cooperative Navigation Tasks”, IEEE International Conference on Robotics and A Utomation Pasadena, CA, USA, May 2008, pp. 3321-2237.
• M. Lauer and M. Riedmiller, “Reinforcement Learning for Stochastic cooperative Multi-agent Systems”, In Proceeding of AAMAS 2004, New York, NY, ACM Press, pp. 1514-1515.
• R. A. C. Bianchi, C. H. C. Ribeiro, A. H. R. Costa, “ Accelerating autonomous learning by using a heuristic selection of actions”, Journal of Heuristis , , 2008, Vol. 2, pp.135-168.
• R. A. C. Bianchi, C. H. C. Ribeiro, A. H. R. Costa, ”Heuristic selection of actions in multi agent reinforcement learning”, 20th International conference on Artificial Intelligence, India , Jan 2007, pp.690-695.
• L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley and Sons, New York, 1994.
• R. S. Sutton, A. G. Barto, “Reinforcement Learning : An Introduction”, MIT Press, 1998.
• J. F. Nash, “Non-cooperative Games”, Annals of Mathematics, , 1951, Vol. 54, pp. 286–295.
• A. M. Fink, Equilibrium in a Stochastic N-person Game, Journal of Science in Hiroshima University, Series A-I, 1964, Vol. 28, pp. 89–93.
• A. Aamodt; E. Plaza, "Case-Based Reasoning: Foundational Issues", Methodological Variations and System Approaches AI Communications, IOS Press, 1994, Vol. 7, No. 1, pp. 39-59.
• R. Bergman; "Engineering Applications of Case Based Reasoning", Journal of Engineering Applications of Artificial Intelligence, 1999 , Vol. 12, pp.805.
• Gabel, T. And Riedmiller, M., “CBR for state value function Approximation in Reinforcement Learning”, Proceeding of the Inter. Conference on Case Based Learning 2005 (ICCBR 2005) , Springer , Chicago, USA.
• Y. Shoham and K. Leyton-Brown , “Multiagent Systems: Algorithmic , Game theoretic and Logical Foundation “ ,2009.