Document (#34463)

Author

Dumais, S.T.

Title

Latent semantic analysis

Source

Annual review of information science and technology. 38(2004), S.189-230

Year

2003

Abstract

Latent Semantic Analysis (LSA) was first introduced in Dumais, Furnas, Landauer, and Deerwester (1988) and Deerwester, Dumais, Furnas, Landauer, and Harshman (1990) as a technique for improving information retrieval. The key insight in LSA was to reduce the dimensionality of the information retrieval problem. Most approaches to retrieving information depend an a lexical match between words in the user's query and those in documents. Indeed, this lexical matching is the way that the popular Web and enterprise search engines work. Such systems are, however, far from ideal. We are all aware of the tremendous amount of irrelevant information that is retrieved when searching. We also fail to find much of the existing relevant material. LSA was designed to address these retrieval problems, using dimension reduction techniques. Fundamental characteristics of human word usage underlie these retrieval failures. People use a wide variety of words to describe the same object or concept (synonymy). Furnas, Landauer, Gomez, and Dumais (1987) showed that people generate the same keyword to describe well-known objects only 20 percent of the time. Poor agreement was also observed in studies of inter-indexer consistency (e.g., Chan, 1989; Tarr & Borko, 1974) in the generation of search terms (e.g., Fidel, 1985; Bates, 1986), and in the generation of hypertext links (Furner, Ellis, & Willett, 1999). Because searchers and authors often use different words, relevant materials are missed. Someone looking for documents an "human-computer interaction" will not find articles that use only the phrase "man-machine studies" or "human factors." People also use the same word to refer to different things (polysemy). Words like "saturn," "jaguar," or "chip" have several different meanings. A short query like "saturn" will thus return many irrelevant documents. The query "Saturn Gar" will return fewer irrelevant items, but it will miss some documents that use only the terms "Saturn automobile." In searching, there is a constant tension between being overly specific and missing relevant information, and being more general and returning irrelevant information.
A number of approaches have been developed in information retrieval to address the problems caused by the variability in word usage. Stemming is a popular technique used to normalize some kinds of surface-level variability by converting words to their morphological root. For example, the words "retrieve," "retrieval," "retrieved," and "retrieving" would all be converted to their root form, "retrieve." The root form is used for both document and query processing. Stemming sometimes helps retrieval, although not much (Harman, 1991; Hull, 1996). And, it does not address Gases where related words are not morphologically related (e.g., physician and doctor). Controlled vocabularies have also been used to limit variability by requiring that query and index terms belong to a pre-defined set of terms. Documents are indexed by a specified or authorized list of subject headings or index terms, called the controlled vocabulary. Library of Congress Subject Headings, Medical Subject Headings, Association for Computing Machinery (ACM) keywords, and Yellow Pages headings are examples of controlled vocabularies. If searchers can find the right controlled vocabulary terms, they do not have to think of all the morphologically related or synonymous terms that authors might have used. However, assigning controlled vocabulary terms in a consistent and thorough manner is a time-consuming and usually manual process. A good deal of research has been published about the effectiveness of controlled vocabulary indexing compared to full text indexing (e.g., Bates, 1998; Lancaster, 1986; Svenonius, 1986). The combination of both full text and controlled vocabularies is often better than either alone, although the size of the advantage is variable (Lancaster, 1986; Markey, Atherton, & Newton, 1982; Srinivasan, 1996). Richer thesauri have also been used to provide synonyms, generalizations, and specializations of users' search terms (see Srinivasan, 1992, for a review). Controlled vocabularies and thesaurus entries can be generated either manually or by the automatic analysis of large collections of texts.
With the advent of large-scale collections of full text, statistical approaches are being used more and more to analyze the relationships among terms and documents. LSA takes this approach. LSA induces knowledge about the meanings of documents and words by analyzing large collections of texts. The approach simultaneously models the relationships among documents based an their constituent words, and the relationships between words based an their occurrence in documents. By using fewer dimensions for representation than there are unique words, LSA induces similarities among terms that are useful in solving the information retrieval problems described earlier. LSA is a fully automatic statistical approach to extracting relations among words by means of their contexts of use in documents, passages, or sentences. It makes no use of natural language processing techniques for analyzing morphological, syntactic, or semantic relations. Nor does it use humanly constructed resources like dictionaries, thesauri, lexical reference systems (e.g., WordNet), semantic networks, or other knowledge representations. Its only input is large amounts of texts. LSA is an unsupervised learning technique. It starts with a large collection of texts, builds a term-document matrix, and tries to uncover some similarity structures that are useful for information retrieval and related text-analysis problems. Several recent ARIST chapters have focused an text mining and discovery (Benoit, 2002; Solomon, 2002; Trybula, 2000). These chapters provide complementary coverage of the field of text analysis.

Theme

Literaturübersicht

Object

Latent Semantic Indexing

Similar documents (author)

1. Gordon, M.D.; Dumais, S.: Using latent semantic indexing for literature based discovery (1998) 4.70

   4.6994414 = sum of:
     4.6994414 = weight(author_txt:dumais in 5892) [ClassicSimilarity], result of:
       4.6994414 = fieldWeight in 5892, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.5 = fieldNorm(doc=5892)
   

2. Dumais, S.; Chen, H.: Hierarchical classification of Web content (2000) 4.70

   4.6994414 = sum of:
     4.6994414 = weight(author_txt:dumais in 1492) [ClassicSimilarity], result of:
       4.6994414 = fieldWeight in 1492, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.5 = fieldNorm(doc=1492)
   

3. Dumais, S.T.; Belkin, N.J.: ¬The TREC interactive tracks : putting the user into search (2005) 4.70

   4.6994414 = sum of:
     4.6994414 = weight(author_txt:dumais in 81) [ClassicSimilarity], result of:
       4.6994414 = fieldWeight in 81, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.5 = fieldNorm(doc=81)
   

4. Teevan, J.; Dumais, S.: Web retrieval, ranking and personalization (2011) 4.70

   4.6994414 = sum of:
     4.6994414 = weight(author_txt:dumais in 1551) [ClassicSimilarity], result of:
       4.6994414 = fieldWeight in 1551, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.5 = fieldNorm(doc=1551)
   

5. Berry, M.W.; Dumais, S.T.; O'Brien, G.W.: Using linear algebra for intelligent information retrieval (1995) 3.52

   3.524581 = sum of:
     3.524581 = weight(author_txt:dumais in 3206) [ClassicSimilarity], result of:
       3.524581 = fieldWeight in 3206, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.398883 = idf(docFreq=9, maxDocs=44421)
         0.375 = fieldNorm(doc=3206)
   

Similar documents (content)

1. Krovetz, R.; Croft, W.B.: Lexical ambiguity and information retrieval (1992) 0.33

   0.33313876 = sum of:
     0.33313876 = product of:
       0.8328469 = sum of:
         0.029924134 = weight(abstract_txt:analysis in 4027) [ClassicSimilarity], result of:
           0.029924134 = score(doc=4027,freq=1.0), product of:
             0.08755327 = queryWeight, product of:
               1.0184517 = boost
               3.6456752 = idf(docFreq=3151, maxDocs=44421)
               0.023580551 = queryNorm
             0.34178203 = fieldWeight in 4027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6456752 = idf(docFreq=3151, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.14597875 = weight(abstract_txt:lexical in 4027) [ClassicSimilarity], result of:
           0.14597875 = score(doc=4027,freq=2.0), product of:
             0.16858728 = queryWeight, product of:
               1.0946928 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.023580551 = queryNorm
             0.8658942 = fieldWeight in 4027, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.020793753 = weight(abstract_txt:that in 4027) [ClassicSimilarity], result of:
           0.020793753 = score(doc=4027,freq=2.0), product of:
             0.06631738 = queryWeight, product of:
               1.189198 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.023580551 = queryNorm
             0.31354907 = fieldWeight in 4027, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.022249931 = weight(abstract_txt:information in 4027) [ClassicSimilarity], result of:
           0.022249931 = score(doc=4027,freq=2.0), product of:
             0.06937844 = queryWeight, product of:
               1.2163337 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023580551 = queryNorm
             0.3207038 = fieldWeight in 4027, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.040043443 = weight(abstract_txt:have in 4027) [ClassicSimilarity], result of:
           0.040043443 = score(doc=4027,freq=2.0), product of:
             0.09440136 = queryWeight, product of:
               1.2512883 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.023580551 = queryNorm
             0.4241829 = fieldWeight in 4027, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.0663743 = weight(abstract_txt:query in 4027) [ClassicSimilarity], result of:
           0.0663743 = score(doc=4027,freq=1.0), product of:
             0.14891025 = queryWeight, product of:
               1.3282094 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023580551 = queryNorm
             0.4457336 = fieldWeight in 4027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.04889871 = weight(abstract_txt:text in 4027) [ClassicSimilarity], result of:
           0.04889871 = score(doc=4027,freq=1.0), product of:
             0.12907733 = queryWeight, product of:
               1.3546274 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.023580551 = queryNorm
             0.3788327 = fieldWeight in 4027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.080899805 = weight(abstract_txt:retrieval in 4027) [ClassicSimilarity], result of:
           0.080899805 = score(doc=4027,freq=3.0), product of:
             0.14330897 = queryWeight, product of:
               1.7481438 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.023580551 = queryNorm
             0.5645132 = fieldWeight in 4027, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.1499754 = weight(abstract_txt:documents in 4027) [ClassicSimilarity], result of:
           0.1499754 = score(doc=4027,freq=3.0), product of:
             0.22399633 = queryWeight, product of:
               2.303773 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.023580551 = queryNorm
             0.66954404 = fieldWeight in 4027, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
         0.22770865 = weight(abstract_txt:words in 4027) [ClassicSimilarity], result of:
           0.22770865 = score(doc=4027,freq=1.0), product of:
             0.45350426 = queryWeight, product of:
               3.590877 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.023580551 = queryNorm
             0.50210917 = fieldWeight in 4027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.09375 = fieldNorm(doc=4027)
       0.4 = coord(10/25)
   

2. Vechtomova, O.; Karamuftuoglum, M.; Robertson, S.E.: On document relevance and lexical cohesion between query terms (2006) 0.33

   0.32988706 = sum of:
     0.32988706 = product of:
       0.9163529 = sum of:
         0.05216199 = weight(abstract_txt:semantic in 1987) [ClassicSimilarity], result of:
           0.05216199 = score(doc=1987,freq=2.0), product of:
             0.105511926 = queryWeight, product of:
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.023580551 = queryNorm
             0.49437058 = fieldWeight in 1987, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.24329793 = weight(abstract_txt:lexical in 1987) [ClassicSimilarity], result of:
           0.24329793 = score(doc=1987,freq=8.0), product of:
             0.16858728 = queryWeight, product of:
               1.0946928 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.023580551 = queryNorm
             1.4431571 = fieldWeight in 1987, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.017328128 = weight(abstract_txt:that in 1987) [ClassicSimilarity], result of:
           0.017328128 = score(doc=1987,freq=2.0), product of:
             0.06631738 = queryWeight, product of:
               1.189198 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.023580551 = queryNorm
             0.2612909 = fieldWeight in 1987, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.0131108975 = weight(abstract_txt:information in 1987) [ClassicSimilarity], result of:
           0.0131108975 = score(doc=1987,freq=1.0), product of:
             0.06937844 = queryWeight, product of:
               1.2163337 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023580551 = queryNorm
             0.18897653 = fieldWeight in 1987, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.09580307 = weight(abstract_txt:query in 1987) [ClassicSimilarity], result of:
           0.09580307 = score(doc=1987,freq=3.0), product of:
             0.14891025 = queryWeight, product of:
               1.3282094 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023580551 = queryNorm
             0.6433611 = fieldWeight in 1987, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.081497855 = weight(abstract_txt:text in 1987) [ClassicSimilarity], result of:
           0.081497855 = score(doc=1987,freq=4.0), product of:
             0.12907733 = queryWeight, product of:
               1.3546274 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.023580551 = queryNorm
             0.6313878 = fieldWeight in 1987, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.03892294 = weight(abstract_txt:retrieval in 1987) [ClassicSimilarity], result of:
           0.03892294 = score(doc=1987,freq=1.0), product of:
             0.14330897 = queryWeight, product of:
               1.7481438 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.023580551 = queryNorm
             0.27160156 = fieldWeight in 1987, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.105872884 = weight(abstract_txt:terms in 1987) [ClassicSimilarity], result of:
           0.105872884 = score(doc=1987,freq=2.0), product of:
             0.23697336 = queryWeight, product of:
               2.485223 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.023580551 = queryNorm
             0.44677126 = fieldWeight in 1987, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
         0.26835722 = weight(abstract_txt:words in 1987) [ClassicSimilarity], result of:
           0.26835722 = score(doc=1987,freq=2.0), product of:
             0.45350426 = queryWeight, product of:
               3.590877 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.023580551 = queryNorm
             0.5917413 = fieldWeight in 1987, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.078125 = fieldNorm(doc=1987)
       0.36 = coord(9/25)
   

3. Aldana, J.F.; Gómez, A.C.; Moreno, N.; Nebro, A.J.; Roldán, M.M.: Metadata functionality for semantic Web integration (2003) 0.32

   0.32396242 = sum of:
     0.32396242 = product of:
       0.6230047 = sum of:
         0.04948521 = weight(abstract_txt:semantic in 3731) [ClassicSimilarity], result of:
           0.04948521 = score(doc=3731,freq=5.0), product of:
             0.105511926 = queryWeight, product of:
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.023580551 = queryNorm
             0.4690011 = fieldWeight in 3731, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.03222582 = weight(abstract_txt:among in 3731) [ClassicSimilarity], result of:
           0.03222582 = score(doc=3731,freq=2.0), product of:
             0.10758884 = queryWeight, product of:
               1.0097941 = boost
               4.518356 = idf(docFreq=1316, maxDocs=44421)
               0.023580551 = queryNorm
             0.29952753 = fieldWeight in 3731, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.518356 = idf(docFreq=1316, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.014020728 = weight(abstract_txt:used in 3731) [ClassicSimilarity], result of:
           0.014020728 = score(doc=3731,freq=1.0), product of:
             0.08909474 = queryWeight, product of:
               1.1254362 = boost
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.023580551 = queryNorm
             0.15736875 = fieldWeight in 3731, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.016438905 = weight(abstract_txt:that in 3731) [ClassicSimilarity], result of:
           0.016438905 = score(doc=3731,freq=5.0), product of:
             0.06631738 = queryWeight, product of:
               1.189198 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.023580551 = queryNorm
             0.2478823 = fieldWeight in 3731, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.03806374 = weight(abstract_txt:vocabulary in 3731) [ClassicSimilarity], result of:
           0.03806374 = score(doc=3731,freq=1.0), product of:
             0.15146615 = queryWeight, product of:
               1.1981385 = boost
               5.3611083 = idf(docFreq=566, maxDocs=44421)
               0.023580551 = queryNorm
             0.25130194 = fieldWeight in 3731, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3611083 = idf(docFreq=566, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.017590115 = weight(abstract_txt:information in 3731) [ClassicSimilarity], result of:
           0.017590115 = score(doc=3731,freq=5.0), product of:
             0.06937844 = queryWeight, product of:
               1.2163337 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023580551 = queryNorm
             0.2535386 = fieldWeight in 3731, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.020021722 = weight(abstract_txt:have in 3731) [ClassicSimilarity], result of:
           0.020021722 = score(doc=3731,freq=2.0), product of:
             0.09440136 = queryWeight, product of:
               1.2512883 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.023580551 = queryNorm
             0.21209145 = fieldWeight in 3731, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.03318715 = weight(abstract_txt:query in 3731) [ClassicSimilarity], result of:
           0.03318715 = score(doc=3731,freq=1.0), product of:
             0.14891025 = queryWeight, product of:
               1.3282094 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023580551 = queryNorm
             0.2228668 = fieldWeight in 3731, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.10111188 = weight(abstract_txt:irrelevant in 3731) [ClassicSimilarity], result of:
           0.10111188 = score(doc=3731,freq=1.0), product of:
             0.29052 = queryWeight, product of:
               1.6593472 = boost
               7.4248013 = idf(docFreq=71, maxDocs=44421)
               0.023580551 = queryNorm
             0.34803757 = fieldWeight in 3731, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4248013 = idf(docFreq=71, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.03302721 = weight(abstract_txt:retrieval in 3731) [ClassicSimilarity], result of:
           0.03302721 = score(doc=3731,freq=2.0), product of:
             0.14330897 = queryWeight, product of:
               1.7481438 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.023580551 = queryNorm
             0.23046157 = fieldWeight in 3731, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.043294173 = weight(abstract_txt:documents in 3731) [ClassicSimilarity], result of:
           0.043294173 = score(doc=3731,freq=1.0), product of:
             0.22399633 = queryWeight, product of:
               2.303773 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.023580551 = queryNorm
             0.19328073 = fieldWeight in 3731, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.06352373 = weight(abstract_txt:terms in 3731) [ClassicSimilarity], result of:
           0.06352373 = score(doc=3731,freq=2.0), product of:
             0.23697336 = queryWeight, product of:
               2.485223 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.023580551 = queryNorm
             0.26806277 = fieldWeight in 3731, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
         0.16101432 = weight(abstract_txt:words in 3731) [ClassicSimilarity], result of:
           0.16101432 = score(doc=3731,freq=2.0), product of:
             0.45350426 = queryWeight, product of:
               3.590877 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.023580551 = queryNorm
             0.35504478 = fieldWeight in 3731, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.046875 = fieldNorm(doc=3731)
       0.52 = coord(13/25)
   

4. Jacquemin, C.: Spotting and discovering terms through natural language processing (2001) 0.32

   0.3222865 = sum of:
     0.3222865 = product of:
       0.7324693 = sum of:
         0.029507278 = weight(abstract_txt:semantic in 1119) [ClassicSimilarity], result of:
           0.029507278 = score(doc=1119,freq=1.0), product of:
             0.105511926 = queryWeight, product of:
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.023580551 = queryNorm
             0.27965823 = fieldWeight in 1119, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.02643774 = weight(abstract_txt:used in 1119) [ClassicSimilarity], result of:
           0.02643774 = score(doc=1119,freq=2.0), product of:
             0.08909474 = queryWeight, product of:
               1.1254362 = boost
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.023580551 = queryNorm
             0.29673737 = fieldWeight in 1119, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.013862502 = weight(abstract_txt:that in 1119) [ClassicSimilarity], result of:
           0.013862502 = score(doc=1119,freq=2.0), product of:
             0.06631738 = queryWeight, product of:
               1.189198 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.023580551 = queryNorm
             0.20903271 = fieldWeight in 1119, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.018166991 = weight(abstract_txt:information in 1119) [ClassicSimilarity], result of:
           0.018166991 = score(doc=1119,freq=3.0), product of:
             0.06937844 = queryWeight, product of:
               1.2163337 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023580551 = queryNorm
             0.26185355 = fieldWeight in 1119, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.059018508 = weight(abstract_txt:texts in 1119) [ClassicSimilarity], result of:
           0.059018508 = score(doc=1119,freq=1.0), product of:
             0.16749722 = queryWeight, product of:
               1.2599492 = boost
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.023580551 = queryNorm
             0.35235515 = fieldWeight in 1119, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.04424954 = weight(abstract_txt:query in 1119) [ClassicSimilarity], result of:
           0.04424954 = score(doc=1119,freq=1.0), product of:
             0.14891025 = queryWeight, product of:
               1.3282094 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023580551 = queryNorm
             0.29715574 = fieldWeight in 1119, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.046102148 = weight(abstract_txt:text in 1119) [ClassicSimilarity], result of:
           0.046102148 = score(doc=1119,freq=2.0), product of:
             0.12907733 = queryWeight, product of:
               1.3546274 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.023580551 = queryNorm
             0.3571669 = fieldWeight in 1119, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.053933203 = weight(abstract_txt:retrieval in 1119) [ClassicSimilarity], result of:
           0.053933203 = score(doc=1119,freq=3.0), product of:
             0.14330897 = queryWeight, product of:
               1.7481438 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.023580551 = queryNorm
             0.37634215 = fieldWeight in 1119, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.10672412 = weight(abstract_txt:controlled in 1119) [ClassicSimilarity], result of:
           0.10672412 = score(doc=1119,freq=1.0), product of:
             0.31323275 = queryWeight, product of:
               2.4366763 = boost
               5.4514923 = idf(docFreq=517, maxDocs=44421)
               0.023580551 = queryNorm
             0.34071827 = fieldWeight in 1119, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4514923 = idf(docFreq=517, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.11978151 = weight(abstract_txt:terms in 1119) [ClassicSimilarity], result of:
           0.11978151 = score(doc=1119,freq=4.0), product of:
             0.23697336 = queryWeight, product of:
               2.485223 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.023580551 = queryNorm
             0.505464 = fieldWeight in 1119, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
         0.21468577 = weight(abstract_txt:words in 1119) [ClassicSimilarity], result of:
           0.21468577 = score(doc=1119,freq=2.0), product of:
             0.45350426 = queryWeight, product of:
               3.590877 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.023580551 = queryNorm
             0.47339305 = fieldWeight in 1119, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=1119)
       0.44 = coord(11/25)
   

5. Wang, F.L.; Yang, C.C.: Mining Web data for Chinese segmentation (2007) 0.31

   0.30907264 = sum of:
     0.30907264 = product of:
       0.7726816 = sum of:
         0.013862502 = weight(abstract_txt:that in 1604) [ClassicSimilarity], result of:
           0.013862502 = score(doc=1604,freq=2.0), product of:
             0.06631738 = queryWeight, product of:
               1.189198 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.023580551 = queryNorm
             0.20903271 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.010488718 = weight(abstract_txt:information in 1604) [ClassicSimilarity], result of:
           0.010488718 = score(doc=1604,freq=1.0), product of:
             0.06937844 = queryWeight, product of:
               1.2163337 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023580551 = queryNorm
             0.15118122 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.051044635 = weight(abstract_txt:large in 1604) [ClassicSimilarity], result of:
           0.051044635 = score(doc=1604,freq=2.0), product of:
             0.12999965 = queryWeight, product of:
               1.2410102 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.023580551 = queryNorm
             0.3926521 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.032695334 = weight(abstract_txt:have in 1604) [ClassicSimilarity], result of:
           0.032695334 = score(doc=1604,freq=3.0), product of:
             0.09440136 = queryWeight, product of:
               1.2512883 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.023580551 = queryNorm
             0.3463439 = fieldWeight in 1604, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.08346477 = weight(abstract_txt:texts in 1604) [ClassicSimilarity], result of:
           0.08346477 = score(doc=1604,freq=2.0), product of:
             0.16749722 = queryWeight, product of:
               1.2599492 = boost
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.023580551 = queryNorm
             0.49830544 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.6376824 = idf(docFreq=429, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.032599144 = weight(abstract_txt:text in 1604) [ClassicSimilarity], result of:
           0.032599144 = score(doc=1604,freq=1.0), product of:
             0.12907733 = queryWeight, product of:
               1.3546274 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.023580551 = queryNorm
             0.25255513 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.031138351 = weight(abstract_txt:retrieval in 1604) [ClassicSimilarity], result of:
           0.031138351 = score(doc=1604,freq=1.0), product of:
             0.14330897 = queryWeight, product of:
               1.7481438 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.023580551 = queryNorm
             0.21728125 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.12907828 = weight(abstract_txt:documents in 1604) [ClassicSimilarity], result of:
           0.12907828 = score(doc=1604,freq=5.0), product of:
             0.22399633 = queryWeight, product of:
               2.303773 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.023580551 = queryNorm
             0.5762518 = fieldWeight in 1604, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.08469831 = weight(abstract_txt:terms in 1604) [ClassicSimilarity], result of:
           0.08469831 = score(doc=1604,freq=2.0), product of:
             0.23697336 = queryWeight, product of:
               2.485223 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.023580551 = queryNorm
             0.35741702 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.30361155 = weight(abstract_txt:words in 1604) [ClassicSimilarity], result of:
           0.30361155 = score(doc=1604,freq=4.0), product of:
             0.45350426 = queryWeight, product of:
               3.590877 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.023580551 = queryNorm
             0.6694789 = fieldWeight in 1604, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
       0.4 = coord(10/25)