English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Quantifying the importance of each training point to a learning task is afundamental problem in machine learning and the estimated importance scoreshave been leveraged to guide a range of data workflows such as datasummarization and domain adaption. One simple idea is to use the leave-one-outerror of each training point to indicate its importance. Recent work has alsoproposed to use the Shapley value, as it defines a unique value distributionscheme that satisfies a set of appealing properties. However, calculatingShapley values is often expensive, which limits its applicability in real-worldapplications at scale. Multiple heuristics to improve the scalability ofcalculating Shapley values have been proposed recently, with the potential riskof compromising their utility in real-world applications.  \textit{How well do existing data quantification methods perform on existingworkflows? How do these methods compare with each other, empirically andtheoretically? Must we sacrifice scalability for the utility in these workflowswhen using these methods?} In this paper, we conduct a novel theoreticalanalysis comparing the utility of different importance quantification methods,and report extensive experimental studies on existing and proposed workflowssuch as noisy label detection, watermark removal, data summarization, dataacquisition, and domain adaptation. We show that Shapley value approximationbased on a $K$NN surrogate over pre-trained feature embeddings obtainscomparable utility with existing algorithms while achieving significantscalability improvement, often by orders of magnitude. Our theoretical analysisalso justifies its advantage over the leave-one-out error.  The code is available at \url{https://github.com/AI-secure/Shapley-Study}.

Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?