English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

For organizing large text corpora topic modeling provides useful tools. Awidely used method is Latent Dirichlet Allocation (LDA), a generativeprobabilistic model which models single texts in a collection of texts asmixtures of latent topics. The assignments of words to topics rely on initialvalues such that generally the outcome of LDA is not fully reproducible. Inaddition, the reassignment via Gibbs Sampling is based on conditionaldistributions, leading to different results in replicated runs on the same textdata. This fact is often neglected in everyday practice. We aim to improve thereliability of LDA results. Therefore, we study the stability of LDA bycomparing assignments from replicated runs. We propose to quantify thesimilarity of two generated topics by a modified Jaccard coefficient. Usingsuch similarities, topics can be clustered. A new pruning algorithm forhierarchical clustering results based on the idea that two LDA runs createpairs of similar topics is proposed. This approach leads to the new measureS-CLOP ({\bf S}imilarity of multiple sets by {\bf C}lustering with {\bf LO}cal{\bf P}runing) for quantifying the stability of LDA models. We discuss somecharacteristics of this measure and illustrate it with an application to realdata consisting of newspaper articles from \textit{USA Today}. Our results showthat the measure S-CLOP is useful for assessing the stability of LDA models orany other topic modeling procedure that characterize its topics by worddistributions. Based on the newly proposed measure for LDA stability, wepropose a method to increase the reliability and hence to improve thereproducibility of empirical findings based on topic modeling. This increase inreliability is obtained by running the LDA several times and taking asprototype the most representative run, that is the LDA run with highest averagesimilarity to all other runs.

Improving Reliability of Latent Dirichlet Allocation by Assessing Its Stability Using Clustering Techniques on Replicated Runs