English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We study the problem of zero-delay coding for the transmission a Markovsource over a noisy channel with feedback and present a rigorous reinforcementtheoretic solution which is guaranteed to achieve near-optimality. To this end,we formulate the problem as a Markov decision process (MDP) where the state isa probability-measure valued predictor/belief and the actions are quantizermaps. This MDP formulation has been used to show the optimality of certainclasses of encoder policies in prior work. Despite such an analytical approachin determining optimal policies, their computation is prohibitively complex dueto the uncountable nature of the constructed state space and the lack ofminorization or strong ergodicity results which are commonly assumed foraverage cost optimal stochastic control. These challenges invite rigorousreinforcement learning methods, which entail several open questions addressedin our paper. We present two complementary approaches for this problem. In thefirst approach, we approximate the set of all beliefs by a finite set and usenearest-neighbor quantization to obtain a finite state MDP, whose optimalpolicies become near-optimal for the original MDP as the quantization becomesarbitrarily fine. In the second approach, a sliding finite window of channeloutputs and quantizers together with a prior belief state serve as the state ofthe MDP. We then approximate this state by marginalizing over all possiblebeliefs, so that our coding policies only use the finite window term to encodethe source. Under an appropriate notion of predictor stability, we show thatsuch policies are near-optimal for the zero-delay coding problem as the windowlength increases. We give sufficient conditions for predictor stability tohold. Finally, we propose a reinforcement learning algorithm to computenear-optimal policies.

Reinforcement Learning for Optimal Transmission of Markov Sources over Noisy Channels: Belief Quantization vs Sliding Finite Window Codes