English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider the minimax query complexity of online planning with a generativemodel in fixed-horizon Markov decision processes (MDPs) with linear functionapproximation. Following recent works, we consider broad classes of problemswhere either (i) the optimal value function $v^\star$ or (ii) the optimalaction-value function $q^\star$ lie in the linear span of some features; or(iii) both $v^\star$ and $q^\star$ lie in the linear span when restricted tothe states reachable from the starting state. Recently, Weisz et al. (2021b)showed that under (ii) the minimax query complexity of any planning algorithmis at least exponential in the horizon $H$ or in the feature dimension $d$ whenthe size $A$ of the action set can be chosen to be exponential in $\min(d,H)$.On the other hand, for the setting (i), Weisz et al. (2021a) introducedTensorPlan, a planner whose query cost is polynomial in all relevant quantitieswhen the number of actions is fixed. Among other things, these two works leftopen the question whether polynomial query complexity is possible when $A$ issubexponential in $\min(d,H)$. In this paper we answer this question in thenegative: we show that an exponentially large lower bound holds when$A=\Omega(\min(d^{1/4},H^{1/2}))$, under either (i), (ii) or (iii). Inparticular, this implies a perhaps surprising exponential separation of querycomplexity compared to the work of Du et al. (2021) who prove a polynomialupper bound when (iii) holds for all states. Furthermore, we show that theupper bound of TensorPlan can be extended to hold under (iii) and, for MDPswith deterministic transitions and stochastic rewards, also under (ii).

TensorPlan and the Few Actions Lower Bound for Planning in MDPs under Linear Realizability of Optimal Value Functions