English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The reproducibility of many experimental results in Deep ReinforcementLearning (RL) is under question. To solve this reproducibility crisis, wepropose a theoretically sound methodology to compare multiple Deep RLalgorithms. The performance of one execution of a Deep RL algorithm is randomso that independent executions are needed to assess it precisely. Whencomparing several RL algorithms, a major question is how many executions mustbe made and how can we assure that the results of such a comparison istheoretically sound. Researchers in Deep RL often use less than 5 independentexecutions to compare algorithms: we claim that this is not enough in general.Moreover, when comparing several algorithms at once, the error of eachcomparison accumulates and must be taken into account with a multiple testsprocedure to preserve low error guarantees. To address this problem in astatistically sound way, we introduce AdaStop, a new statistical test based onmultiple group sequential tests. When comparing algorithms, AdaStop adapts thenumber of executions to stop as early as possible while ensuring that we haveenough information to distinguish algorithms that perform better than theothers in a statistical significant way. We prove both theoretically andempirically that AdaStop has a low probability of making an error (Family-WiseError). Finally, we illustrate the effectiveness of AdaStop in multipleuse-cases, including toy examples and difficult cases such as Mujocoenvironments.

AdaStop: sequential testing for efficient and reliable comparisons of Deep RL Agents