English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multi-agent reinforcement learning (MARL) suffers from the non-stationarityproblem, which is the ever-changing targets at every iteration when multipleagents update their policies at the same time. Starting from first principle,in this paper, we manage to solve the non-stationarity problem by proposingbidirectional action-dependent Q-learning (ACE). Central to the development ofACE is the sequential decision-making process wherein only one agent is allowedto take action at one time. Within this process, each agent maximizes its valuefunction given the actions taken by the preceding agents at the inferencestage. In the learning phase, each agent minimizes the TD error that isdependent on how the subsequent agents have reacted to their chosen action.Given the design of bidirectional dependency, ACE effectively turns amultiagent MDP into a single-agent MDP. We implement the ACE framework byidentifying the proper network representation to formulate the actiondependency, so that the sequential decision process is computed implicitly inone forward pass. To validate ACE, we compare it with strong baselines on twoMARL benchmarks. Empirical experiments demonstrate that ACE outperforms thestate-of-the-art algorithms on Google Research Football and StarCraftMulti-Agent Challenge by a large margin. In particular, on SMAC tasks, ACEachieves 100% success rate on almost all the hard and super-hard maps. Wefurther study extensive research problems regarding ACE, including extension,generalization, and practicability. Code is made available to facilitatefurther research.

ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency