Document (#37331)

Author

Altmann, E.G.
Cristadoro, G.
Esposti, M.D.

Title

On the origin of long-range correlations in texts

Source

Proceedings of the National Academy of Sciences, 2. Juli 2012. DOI: 10.1073/pnas.1117723109

Year

2012

Abstract

The complexity of human interactions with social and natural phenomena is mirrored in the way we describe our experiences through natural language. In order to retain and convey such a high dimensional information, the statistical properties of our linguistic output has to be highly correlated in time. An example are the robust observations, still largely not understood, of correlations on arbitrary long scales in literary texts. In this paper we explain how long-range correlations flow from highly structured linguistic levels down to the building blocks of a text (words, letters, etc..). By combining calculations and data analysis we show that correlations take form of a bursty sequence of events once we approach the semantically relevant topics of the text. The mechanisms we identify are fairly general and can be equally applied to other hierarchical settings.

Content

Vgl. die Pressemitteilung zum Artikel: Auf der Suche nach dem entscheidenden Wort: die Häufung bestimmter Wörter innerhalb eines Textes macht diese zu Schlüsselwörtern [11. Juli 2012]. Unter: http://www.mpg.de/5894319/statistische_Textanalyse?filter_order=L. Vgl. auch: http://arxiv.org/list/cs.CL/current.

Footnote

Volltext unter: Altmann_etal_Correlations_texts.pdf.

Theme

Computerlinguistik

Similar documents (author)

1. Altmann, E.: Assessment of reference services (1982) 5.87

   5.874302 = sum of:
     5.874302 = weight(author_txt:altmann in 4607) [ClassicSimilarity], result of:
       5.874302 = fieldWeight in 4607, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.625 = fieldNorm(doc=4607)
   

2. Altmann, O.: Internet in Öffentlichen Bibliotheken : Nutzungsmöglichkeiten und Probleme (1996) 5.87

   5.874302 = sum of:
     5.874302 = weight(author_txt:altmann in 6018) [ClassicSimilarity], result of:
       5.874302 = fieldWeight in 6018, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.625 = fieldNorm(doc=6018)
   

3. Altmann, O.: Internet in Öffentlichen Bibliotheken : Nutzungsmöglichkeiten und Probleme (1997) 5.87

   5.874302 = sum of:
     5.874302 = weight(author_txt:altmann in 833) [ClassicSimilarity], result of:
       5.874302 = fieldWeight in 833, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.625 = fieldNorm(doc=833)
   

4. Altmann, R.: Digitale Dia-Archive : Bilddatenbanken bringen Ordnung in die digitale Foto- und Grafiksammlung (1998) 5.87

   5.874302 = sum of:
     5.874302 = weight(author_txt:altmann in 860) [ClassicSimilarity], result of:
       5.874302 = fieldWeight in 860, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.625 = fieldNorm(doc=860)
   

5. Altmann, M.: Metadaten (2005) 5.87

   5.874302 = sum of:
     5.874302 = weight(author_txt:altmann in 5561) [ClassicSimilarity], result of:
       5.874302 = fieldWeight in 5561, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.625 = fieldNorm(doc=5561)
   

Similar documents (content)

1. Kokol, P.; Podgorelec, V.; Zorman, M.; Kokol, T.; Njivar, T.: Computer and natural language texts : a comparison based on long-range correlations (1999) 0.18

   0.17963858 = sum of:
     0.17963858 = product of:
       0.8981929 = sum of:
         0.094615884 = weight(abstract_txt:range in 5299) [ClassicSimilarity], result of:
           0.094615884 = score(doc=5299,freq=2.0), product of:
             0.14113618 = queryWeight, product of:
               1.4880027 = boost
               5.0563765 = idf(docFreq=768, maxDocs=44421)
               0.018758377 = queryNorm
             0.67038715 = fieldWeight in 5299, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.0563765 = idf(docFreq=768, maxDocs=44421)
               0.09375 = fieldNorm(doc=5299)
         0.09535256 = weight(abstract_txt:natural in 5299) [ClassicSimilarity], result of:
           0.09535256 = score(doc=5299,freq=2.0), product of:
             0.14186783 = queryWeight, product of:
               1.4918545 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.018758377 = queryNorm
             0.6721225 = fieldWeight in 5299, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.09375 = fieldNorm(doc=5299)
         0.13114376 = weight(abstract_txt:texts in 5299) [ClassicSimilarity], result of:
           0.13114376 = score(doc=5299,freq=2.0), product of:
             0.17545299 = queryWeight, product of:
               1.6590708 = boost
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.018758377 = queryNorm
             0.74745816 = fieldWeight in 5299, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.09375 = fieldNorm(doc=5299)
         0.15809554 = weight(abstract_txt:long in 5299) [ClassicSimilarity], result of:
           0.15809554 = score(doc=5299,freq=2.0), product of:
             0.22749534 = queryWeight, product of:
               2.3137498 = boost
               5.2415633 = idf(docFreq=638, maxDocs=44421)
               0.018758377 = queryNorm
             0.6949397 = fieldWeight in 5299, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2415633 = idf(docFreq=638, maxDocs=44421)
               0.09375 = fieldNorm(doc=5299)
         0.41898513 = weight(abstract_txt:correlations in 5299) [ClassicSimilarity], result of:
           0.41898513 = score(doc=5299,freq=1.0), product of:
             0.6041546 = queryWeight, product of:
               4.353851 = boost
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.018758377 = queryNorm
             0.6935065 = fieldWeight in 5299, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.09375 = fieldNorm(doc=5299)
       0.2 = coord(5/25)
   

2. Schwarz, C.: THESYS: Thesaurus Syntax System : a fully automatic thesaurus building aid (1988) 0.09

   0.0862925 = sum of:
     0.0862925 = product of:
       0.7191042 = sum of:
         0.059145276 = weight(abstract_txt:text in 1429) [ClassicSimilarity], result of:
           0.059145276 = score(doc=1429,freq=3.0), product of:
             0.09013884 = queryWeight, product of:
               1.1891606 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.018758377 = queryNorm
             0.6561575 = fieldWeight in 1429, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
         0.06742445 = weight(abstract_txt:natural in 1429) [ClassicSimilarity], result of:
           0.06742445 = score(doc=1429,freq=1.0), product of:
             0.14186783 = queryWeight, product of:
               1.4918545 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.018758377 = queryNorm
             0.4752624 = fieldWeight in 1429, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
         0.5925345 = weight(abstract_txt:correlations in 1429) [ClassicSimilarity], result of:
           0.5925345 = score(doc=1429,freq=2.0), product of:
             0.6041546 = queryWeight, product of:
               4.353851 = boost
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.018758377 = queryNorm
             0.9807663 = fieldWeight in 1429, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
       0.12 = coord(3/25)
   

3. Stamatatos, E.: ¬A survey of modern authorship attribution methods (2009) 0.08

   0.082232565 = sum of:
     0.082232565 = product of:
       0.3426357 = sum of:
         0.05415102 = weight(abstract_txt:literary in 3741) [ClassicSimilarity], result of:
           0.05415102 = score(doc=3741,freq=1.0), product of:
             0.12748554 = queryWeight, product of:
               6.7961926 = idf(docFreq=134, maxDocs=44421)
               0.018758377 = queryNorm
             0.42476204 = fieldWeight in 3741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7961926 = idf(docFreq=134, maxDocs=44421)
               0.0625 = fieldNorm(doc=3741)
         0.039430182 = weight(abstract_txt:text in 3741) [ClassicSimilarity], result of:
           0.039430182 = score(doc=3741,freq=3.0), product of:
             0.09013884 = queryWeight, product of:
               1.1891606 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.018758377 = queryNorm
             0.4374383 = fieldWeight in 3741, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=3741)
         0.044949632 = weight(abstract_txt:natural in 3741) [ClassicSimilarity], result of:
           0.044949632 = score(doc=3741,freq=1.0), product of:
             0.14186783 = queryWeight, product of:
               1.4918545 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.018758377 = queryNorm
             0.3168416 = fieldWeight in 3741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.0625 = fieldNorm(doc=3741)
         0.061821766 = weight(abstract_txt:texts in 3741) [ClassicSimilarity], result of:
           0.061821766 = score(doc=3741,freq=1.0), product of:
             0.17545299 = queryWeight, product of:
               1.6590708 = boost
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.018758377 = queryNorm
             0.35235515 = fieldWeight in 3741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.0625 = fieldNorm(doc=3741)
         0.06775613 = weight(abstract_txt:linguistic in 3741) [ClassicSimilarity], result of:
           0.06775613 = score(doc=3741,freq=1.0), product of:
             0.18650863 = queryWeight, product of:
               1.7105429 = boost
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.018758377 = queryNorm
             0.36328685 = fieldWeight in 3741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.0625 = fieldNorm(doc=3741)
         0.07452695 = weight(abstract_txt:long in 3741) [ClassicSimilarity], result of:
           0.07452695 = score(doc=3741,freq=1.0), product of:
             0.22749534 = queryWeight, product of:
               2.3137498 = boost
               5.2415633 = idf(docFreq=638, maxDocs=44421)
               0.018758377 = queryNorm
             0.3275977 = fieldWeight in 3741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2415633 = idf(docFreq=638, maxDocs=44421)
               0.0625 = fieldNorm(doc=3741)
       0.24 = coord(6/25)
   

4. Kousha, K.; Thelwall, M.: Google Scholar citations and Google Web/URL citations : a multi-discipline exploratory analysis (2007) 0.08

   0.07948488 = sum of:
     0.07948488 = product of:
       0.4967805 = sum of:
         0.064136684 = weight(abstract_txt:correlated in 1337) [ClassicSimilarity], result of:
           0.064136684 = score(doc=1337,freq=1.0), product of:
             0.14271212 = queryWeight, product of:
               1.0580348 = boost
               7.190608 = idf(docFreq=90, maxDocs=44421)
               0.018758377 = queryNorm
             0.449413 = fieldWeight in 1337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.190608 = idf(docFreq=90, maxDocs=44421)
               0.0625 = fieldNorm(doc=1337)
         0.09748307 = weight(abstract_txt:calculations in 1337) [ClassicSimilarity], result of:
           0.09748307 = score(doc=1337,freq=1.0), product of:
             0.18865833 = queryWeight, product of:
               1.2164872 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.018758377 = queryNorm
             0.51671755 = fieldWeight in 1337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.0625 = fieldNorm(doc=1337)
         0.055837344 = weight(abstract_txt:highly in 1337) [ClassicSimilarity], result of:
           0.055837344 = score(doc=1337,freq=1.0), product of:
             0.16393928 = queryWeight, product of:
               1.6037108 = boost
               5.4495635 = idf(docFreq=518, maxDocs=44421)
               0.018758377 = queryNorm
             0.34059772 = fieldWeight in 1337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4495635 = idf(docFreq=518, maxDocs=44421)
               0.0625 = fieldNorm(doc=1337)
         0.2793234 = weight(abstract_txt:correlations in 1337) [ClassicSimilarity], result of:
           0.2793234 = score(doc=1337,freq=1.0), product of:
             0.6041546 = queryWeight, product of:
               4.353851 = boost
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.018758377 = queryNorm
             0.46233764 = fieldWeight in 1337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.0625 = fieldNorm(doc=1337)
       0.16 = coord(4/25)
   

5. Teich, E.; Degaetano-Ortlieb, S.; Fankhauser, P.; Kermes, H.; Lapshinova-Koltunski, E.: ¬The linguistic construal of disciplinarity : a data-mining approach using register features (2016) 0.07

   0.07446515 = sum of:
     0.07446515 = product of:
       0.37232572 = sum of:
         0.049287725 = weight(abstract_txt:text in 4015) [ClassicSimilarity], result of:
           0.049287725 = score(doc=4015,freq=3.0), product of:
             0.09013884 = queryWeight, product of:
               1.1891606 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.018758377 = queryNorm
             0.5467979 = fieldWeight in 4015, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.078125 = fieldNorm(doc=4015)
         0.056187037 = weight(abstract_txt:natural in 4015) [ClassicSimilarity], result of:
           0.056187037 = score(doc=4015,freq=1.0), product of:
             0.14186783 = queryWeight, product of:
               1.4918545 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.018758377 = queryNorm
             0.396052 = fieldWeight in 4015, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.078125 = fieldNorm(doc=4015)
         0.06979668 = weight(abstract_txt:highly in 4015) [ClassicSimilarity], result of:
           0.06979668 = score(doc=4015,freq=1.0), product of:
             0.16393928 = queryWeight, product of:
               1.6037108 = boost
               5.4495635 = idf(docFreq=518, maxDocs=44421)
               0.018758377 = queryNorm
             0.42574716 = fieldWeight in 4015, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4495635 = idf(docFreq=518, maxDocs=44421)
               0.078125 = fieldNorm(doc=4015)
         0.077277206 = weight(abstract_txt:texts in 4015) [ClassicSimilarity], result of:
           0.077277206 = score(doc=4015,freq=1.0), product of:
             0.17545299 = queryWeight, product of:
               1.6590708 = boost
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.018758377 = queryNorm
             0.44044393 = fieldWeight in 4015, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.078125 = fieldNorm(doc=4015)
         0.11977704 = weight(abstract_txt:linguistic in 4015) [ClassicSimilarity], result of:
           0.11977704 = score(doc=4015,freq=2.0), product of:
             0.18650863 = queryWeight, product of:
               1.7105429 = boost
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.018758377 = queryNorm
             0.64220643 = fieldWeight in 4015, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.078125 = fieldNorm(doc=4015)
       0.2 = coord(5/25)