English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this paper, we consider an intrusion detection application for WirelessSensor Networks (WSNs). We study the problem of scheduling the sleep times ofthe individual sensors to maximize the network lifetime while keeping thetracking error to a minimum. We formulate this problem as apartially-observable Markov decision process (POMDP) with continuousstate-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]).However, unlike their formulation, we consider infinite horizon discounted andaverage cost objectives as performance criteria. For each criterion, we proposea convergent on-policy Q-learning algorithm that operates on two timescales,while employing function approximation to handle the curse of dimensionalityassociated with the underlying POMDP. Our proposed algorithm incorporates apolicy gradient update using a one-simulation simultaneous perturbationstochastic approximation (SPSA) estimate on the faster timescale, while theQ-value parameter (arising from a linear function approximation for theQ-values) is updated in an on-policy temporal difference (TD) algorithm-likefashion on the slower timescale. The feature selection scheme employed in eachof our algorithms manages the energy and tracking components in a manner thatassists the search for the optimal sleep-scheduling policy. For the sake ofcomparison, in both discounted and average settings, we also develop a functionapproximation analogue of the Q-learning algorithm. This algorithm, unlike thetwo-timescale variant, does not possess theoretical convergence guarantees.Finally, we also adapt our algorithms to include a stochastic iterativeestimation scheme for the intruder's mobility model. Our simulation results ona 2-dimensional network setting suggest that our algorithms result in bettertracking accuracy at the cost of only a few additional sensors, in comparisonto a recent prior work.

Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks