English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

This paper studies the estimation of high-dimensional, discrete, possiblysparse, mixture models in topic models. The data consists of observedmultinomial counts of $p$ words across $n$ independent documents. In topicmodels, the $p\times n$ expected word frequency matrix is assumed to befactorized as a $p\times K$ word-topic matrix $A$ and a $K\times n$topic-document matrix $T$. Since columns of both matrices represent conditionalprobabilities belonging to probability simplices, columns of $A$ are viewed as$p$-dimensional mixture components that are common to all documents whilecolumns of $T$ are viewed as the $K$-dimensional mixture weights that aredocument specific and are allowed to be sparse. The main interest is to providesharp, finite sample, $\ell_1$-norm convergence rates for estimators of themixture weights $T$ when $A$ is either known or unknown. For known $A$, wesuggest MLE estimation of $T$. Our non-standard analysis of the MLE not onlyestablishes its $\ell_1$ convergence rate, but reveals a remarkable property:the MLE, with no extra regularization, can be exactly sparse and contain thetrue zero pattern of $T$. We further show that the MLE is both minimax optimaland adaptive to the unknown sparsity in a large class of sparse topicdistributions. When $A$ is unknown, we estimate $T$ by optimizing thelikelihood function corresponding to a plug in, generic, estimator $\hat{A}$ of$A$. For any estimator $\hat{A}$ that satisfies carefully detailed conditionsfor proximity to $A$, the resulting estimator of $T$ is shown to retain theproperties established for the MLE. The ambient dimensions $K$ and $p$ areallowed to grow with the sample sizes. Our application is to the estimation of1-Wasserstein distances between document generating distributions. We propose,estimate and analyze new 1-Wasserstein distances between two probabilisticdocument representations.

Likelihood estimation of sparse topic distributions in topic models and its applications to Wasserstein document distance calculations