English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Cultural-scale models of full text documents are prone to over-interpretationby researchers making unintentionally strong socio-linguistic claims (Pechenicket al., 2015) without recognizing that even large digital libraries are merelysamples of all the books ever produced. In this study, we test the sensitivityof the topic models to the sampling process by taking random samples of booksin the Hathi Trust Digital Library from different areas of the Library ofCongress Classification Outline. For each classification area, we train severaltopic models over the entire class with different random seeds, generating aset of spanning models. Then, we train topic models on random samples of booksfrom the classification area, generating a set of sample models. Finally, weperform a topic alignment between each pair of models by computing theJensen-Shannon distance (JSD) between the word probability distributions foreach topic. We take two measures on each model alignment: alignment distanceand topic overlap. We find that sample models with a large sample sizetypically have an alignment distance that falls in the range of the alignmentdistance between spanning models. Unsurprisingly, as sample size increases,alignment distance decreases. We also find that the topic overlap increases assample size increases. However, the decomposition of these measures by samplesize differs by number of topics and by classification area. We speculate thatthese measures could be used to find classes which have a common "canon"discussed among all books in the area, as shown by high topic overlap and lowalignment distance even in small sample sizes.

Towards Evaluation of Cultural-scale Claims in Light of Topic Model Sampling Effects