AAAI22-RLG

Schedule

Start Time PST EST CET			Type	Duration	Title & Speaker / Author(s)

7:00	10:00	16:00	Intro	10 min	Workshop overview Viliam Lisy, Martin Schmid, Noam Brown
7:10	10:10	16:10	Mini Tutorial	30 min	Noam Brown
7:40	10:40	16:40	Invited Talk	40 min	Multiagent learning: from games to sports, and back Karl Tuyls (DeepMind, University of Liverpool)
8:20	11:20	17:20	Posters	60 min	Poster Session #1

Break (20 min)

9:40	12:40	18:40	Talk	20 min	Provably Efficient Decentralized Communication for Multi-Agent RL Justin Lidard, Udari Madhushani and Naomi Leonard
10:00	13:00	19:00	Talk	20 min	Subgame solving without common knowledge Brian Zhang and Tuomas Sandholm
10:20	13:20	19:20	Invited Talk	40 min	~~Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination~~ Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning Peter Stone (The University of Texas at Austin and Sony AI)
11:00	14:00	20:00	HIGC	10 min	Hidden Information Games Competition Michal Sustr

Break (30 min)

11:40	14:40	20:40	Talk	20 min	Multi-Agent Learning for Iterative Dominance Elimination: Intrinsic Barriers and New Algorithms Jibang Wu, Haifeng Xu and Fan Yao
12:00	15:00	21:00	Talk	20 min	Computing Strategies of American Football via Counterfactual Regret Minimization Yuki Shimano, Kenshi Abe, Atsushi Iwasaki and Kazunori Ohkawara
12:20	15:20	21:20	Invited Talk	40 min	Reinforcement Learning for Security Games and Beyond Fei Fang (Carnegie Mellon University)
13:00	16:00	22:00	Posters	60 min	Poster Session #2

Location

The workshop is taking place in Blue 1.

Invited Talks

Multiagent learning: from games to sports, and back
Karl Tuyls (DeepMind, University of Liverpool)

Many real-world scenarios can be modelled as multiagent systems, in which multiple autonomous decision makers interact in a single environment, often cast as a multi-player game. Recent progress in reinforcement learning, evolutionary and empirical game theory has made it possible to apply multiagent learning techniques in applications such as complex board games, multi-robot systems and sports. In this talk I'll provide an overview of how some of the theoretical progress on the game-theoretic front paves the way to apply some multiagent learning techniques in more challenging domains like sports and robotics, showing how these techniques allow for new insights and algorithms in this area. At the same time I will review some of the theoretical progress these multiagent learning techniques build on, based on evolutionary dynamics as a tool or means to capture the dynamics of multiple learning agents across different classes of multiagent learning.

Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning
Peter Stone (The University of Texas at Austin and Sony AI)

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world's best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world's best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.
Project webpage: https://www.gran-turismo.com/us/gran-turismo-sophy/

Reinforcement Learning for Security Games and Beyond
Fei Fang (Carnegie Mellon University)

Security game models have led to great success in modeling defender-attacker interaction or alike in security and environmental sustainability domains. Most of the earlier work in security games relies on mathematical programming-based algorithms to solve the game and compute the optimal strategy for the defender. However, for many problems that account for more practical aspects, the game model would be much more complex and mathematical programming-based methods are not applicable. In this talk, we introduce our work that leverages reinforcement learning to handle complex security games, including games with continuous action space, green security games with real-time information, and repeated games with unknown attacker type.

Accepted Posters

Poster Session #1

Poster Session #2

Cooperation Learning in Time-Varying Multi-Agent Networks
Vasanth Reddy Baddam, Almuatazbellah Boker and Hoda Eldardiry

Learning Generalizable Behavior via Visual Rewrite Rules
Mingxuan Li, Yiheng Xie, Shangqun Yu and Michael Littman

Graph augmented Deep Reinforcement Learning in the GameRLand3D environment
Edward Beeching, Maxim Peter, Philippe Marcotte, Jilles Debangoye, Olivier Simonin, Joshua Romoff and Christian Wolf

Anytime Optimal PSRO for Two-Player Zero-Sum Games
Stephen McAleer, Kevin Wang, Marc Lanctot, John Lanier, Pierre Baldi and Roy Fox

On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learning
Andrew Cohen, Ervin Teng, Vincent-Pierre Berges, Ruo-Ping Dong, Hunter Henry, Marwan Mattar, Alexander Zook and Sujoy Ganguly

The Partially Observable History Process
Dustin Morrill, Amy Greenwald and Michael Bowling

A Review for Deep Reinforcement Learning in Atari: Benchmarks, Challenges and Solutions
Jiajun Fan

Continual Depth-limited Responses for Computing Counter-strategies in Extensive-form Games
David Milec and Viliam Lisy

Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning
Yantian Zha, Lin Guan and Subbarao Kambhampati

Stackelberg MADDPG: Learning Emergent Behaviors via Information Asymmetry in Competitive Games
Boling Yang, Liyuan Zheng, Lillian Ratliff, Byron Boots and Joshua Smith

Faster No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
Ioannis Anagnostides, Gabriele Farina, Tuomas Sandholm and Christian Kroer

Connecting Optimal Ex-Ante Collusion in Teams to Extensive-Form Correlation: Faster Algorithms and Positive Complexity Results
Gabriele Farina, Andrea Celli, Nicola Gatti and Tuomas Sandholm

GDI: Rethinking What Makes Reinforcement Learning Different from Supervised Learning
Jiajun Fan, Changnan Xiao and Yue Huang

Where, When & Which Concepts does AlphaZero Learn? Lessons from the Game of Hex
Jessica Forde, Charles Lovering, Ellie Pavlick and Michael Littman

MDP Abstraction with Successor Features
Dongge Han, Michael Wooldridge and Sebastian Tschiatschek

Fast Algorithms for Poker Require Modelling it as a Sequential Bayesian Game
Vojtech Kovarik, David Milec, Michal Sustr, Dominik Seitz and Viliam Lisy

A Bayesian Policy Reuse Approach for Bilateral Negotiation Games
Xiaoyang Gao, Siqi Chen, Yan Zheng and Jianye Hao

An Adaptive State Aggregation Algorithm for Markov Decision Processes
Guanting Chen, Johann Gaebler, Matt Peng, Chunlin Sun and Yinyu Ye

Deep Catan
Brahim Driss and Tristan Cazenave

AAAI-22 Workshop on Reinforcement Learning in Games

Virtual Workshop, Feb 28th, 2022