Premium
Estimating word frequencies from lexical dispersion data 1
Author(s) -
Indefrey P.,
Baayeny H.
Publication year - 1994
Publication title -
statistica neerlandica
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.52
H-Index - 39
eISSN - 1467-9574
pISSN - 0039-0402
DOI - 10.1111/j.1467-9574.1994.tb01448.x
Subject(s) - cohesion (chemistry) , word (group theory) , word lists by frequency , probabilistic logic , computer science , linguistics , natural language processing , artificial intelligence , mathematics , physics , philosophy , quantum mechanics , sentence
A method for estimating word frequencies on the basic's of lexical dispersion data that makes use of some results in occupancy theory is outlined. The accuracy of the method, which assumes that words occur independently in texts, is tested on Lewis Carroll's “Alice in Wonderland”. Although intra‐textual and inter‐textual cohesion can be traced as sources of misfit, the probabilistic aspect of the relation between frequency and dispersion is strong enough to allow lower boundary estimates of word frequency to be obtained for roughly 75% of the words.