English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Computational notebooks (e.g., Jupyter, Google Colab) are widely used forinteractive data science and machine learning. In those frameworks, users canstart a session, then execute cells (i.e., a set of statements) to createvariables, train models, visualize results, etc. Unfortunately, existingnotebook systems do not offer live migration: when a notebook launches on a newmachine, it loses its state, preventing users from continuing their tasks fromwhere they had left off. This is because, unlike DBMS, the sessions directlyrely on underlying kernels (e.g., Python/R interpreters) without an additionaldata management layer. Existing techniques for preserving states, such ascopying all variables or OS-level checkpointing, are unreliable (often fail),inefficient, and platform-dependent. Also, re-running code from scratch can behighly time-consuming. In this paper, we introduce a new notebook system,ElasticNotebook, that offers live migration via checkpointing/restoration usinga novel mechanism that is reliable, efficient, and platform-independent.Specifically, by observing all cell executions via transparent, lightweightmonitoring, ElasticNotebook can find a reliable and efficient way (i.e.,replication plan) for reconstructing the original session state, consideringvariable-cell dependencies, observed runtime, variable sizes, etc. To this end,our new graph-based optimization problem finds how to reconstruct all variables(efficiently) from a subset of variables that can be transferred acrossmachines. We show that ElasticNotebook reduces end-to-end migration andrestoration times by 85%-98% and 94%-99%, respectively, on a variety (i.e.,Kaggle, JWST, and Tutorial) of notebooks with negligible runtime and memoryoverheads of <2.5% and <10%.

ElasticNotebook: Enabling Live Migration for Computational Notebooks (Technical Report)