English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Real-time learning is crucial for robotic agents adapting to ever-changing,non-stationary environments. A common setup for a robotic agent is to have twodifferent computers simultaneously: a resource-limited local computer tetheredto the robot and a powerful remote computer connected wirelessly. Given such asetup, it is unclear to what extent the performance of a learning system can beaffected by resource limitations and how to efficiently use the wirelesslyconnected powerful computer to compensate for any performance loss. In thispaper, we implement a real-time learning system called the Remote-LocalDistributed (ReLoD) system to distribute computations of two deep reinforcementlearning (RL) algorithms, Soft Actor-Critic (SAC) and Proximal PolicyOptimization (PPO), between a local and a remote computer. The performance ofthe system is evaluated on two vision-based control tasks developed using arobotic arm and a mobile robot. Our results show that SAC's performancedegrades heavily on a resource-limited local computer. Strikingly, when allcomputations of the learning system are deployed on a remote workstation, SACfails to compensate for the performance loss, indicating that, without carefulconsideration, using a powerful remote computer may not result in performanceimprovement. However, a carefully chosen distribution of computations of SACconsistently and substantially improves its performance on both tasks. On theother hand, the performance of PPO remains largely unaffected by thedistribution of computations. In addition, when all computations happen solelyon a powerful tethered computer, the performance of our system remains on parwith an existing system that is well-tuned for using a single machine. ReLoD isthe only publicly available system for real-time RL that applies to multiplerobots for vision-based tasks.

Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computers