English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The ever-growing complexity of reinforcement learning (RL) tasks demands adistributed RL system to efficiently generate and process a massive amount ofdata to train intelligent agents. However, existing open-source librariessuffer from various limitations, which impede their practical use inchallenging scenarios where large-scale training is necessary. While industrialsystems from OpenAI and DeepMind have achieved successful large-scale RLtraining, their system architecture and implementation details remainundisclosed to the community. In this paper, we present a novel abstraction onthe dataflows of RL training, which unifies practical RL training acrossdiverse applications into a general framework and enables fine-grainedoptimizations. Following this abstraction, we develop a scalable, efficient,and extensible distributed RL system called ReaLly Scalable RL (SRL). Thesystem architecture of SRL separates major RL computation components and allowsmassively parallelized training. Moreover, SRL offers user-friendly andextensible interfaces for customized algorithms. Our evaluation shows that SRLoutperforms existing academic libraries in both a single machine and amedium-sized cluster. In a large-scale cluster, the novel architecture of SRLleads to up to 3.7x speedup compared to the design choices adopted by theexisting libraries. We also conduct a direct benchmark comparison to OpenAI'sindustrial system, Rapid, in the challenging hide-and-seek environment. SRLreproduces the same solution as reported by OpenAI with up to 5x speedup inwall-clock time. Furthermore, we also examine the performance of SRL in a muchharder variant of the hide-and-seek environment and achieve substantiallearning speedup by scaling SRL to over 15k CPU cores and 32 A100 GPUs.Notably, SRL is the first in the academic community to perform RL experimentsat such a large scale.

SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores