Document (#29581)

Author

Yang, C.C.
Li, K.W.

Title

¬A heuristic method based on a statistical approach for chinese text segmentation

Source

Journal of the American Society for Information Science and Technology. 56(2005) no.13, S.1438-1447

Year

2005

Abstract

The authors propose a heuristic method for Chinese automatic text segmentation based an a statistical approach. This method is developed based an statistical information about the association among adjacent characters in Chinese text. Mutual information of bi-grams and significant estimation of tri-grams are utilized. A heuristic method with six rules is then proposed to determine the segmentation points in a Chinese sentence. No dictionary is required in this method. Chinese text segmentation is important in Chinese text indexing and thus greatly affects the performance of Chinese information retrieval. Due to the lack of delimiters of words in Chinese text, Chinese text segmentation is more difficult than English text segmentation. Besides, segmentation ambiguities and occurrences of out-of-vocabulary words (i.e., unknown words) are the major challenges in Chinese segmentation. Many research studies dealing with the problem of word segmentation have focused an the resolution of segmentation ambiguities. The problem of unknown word identification has not drawn much attention. The experimental result Shows that the proposed heuristic method is promising to segment the unknown words as weIl as the known words. The authors further investigated the distribution of the errors of commission and the errors of omission caused by the proposed heuristic method and benchmarked the proposed heuristic method with a previous proposed technique, boundary detection. It is found that the heuristic method outperformed the boundary detection method.

Similar documents (author)

1. Yang, S.C.: ¬An interpretive and situated approach to an evaluation of Perseus digital libraries (2001) 4.48

   4.4805427 = sum of:
     4.4805427 = weight(author_txt:yang in 933) [ClassicSimilarity], result of:
       4.4805427 = fieldWeight in 933, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.168868 = idf(docFreq=92, maxDocs=44421)
         0.625 = fieldNorm(doc=933)
   

2. Yang, K.: Information retrieval on the Web (2004) 4.48

   4.4805427 = sum of:
     4.4805427 = weight(author_txt:yang in 5278) [ClassicSimilarity], result of:
       4.4805427 = fieldWeight in 5278, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.168868 = idf(docFreq=92, maxDocs=44421)
         0.625 = fieldNorm(doc=5278)
   

3. Yang, C.C.: Content-based image retrievaI : a comparison between query by example and image browsing map approaches (2005) 4.48

   4.4805427 = sum of:
     4.4805427 = weight(author_txt:yang in 5649) [ClassicSimilarity], result of:
       4.4805427 = fieldWeight in 5649, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.168868 = idf(docFreq=92, maxDocs=44421)
         0.625 = fieldNorm(doc=5649)
   

4. Salton, G.; Yang, C.S.: On the specification of term values in automatic indexing (1973) 3.58

   3.584434 = sum of:
     3.584434 = weight(author_txt:yang in 5475) [ClassicSimilarity], result of:
       3.584434 = fieldWeight in 5475, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.168868 = idf(docFreq=92, maxDocs=44421)
         0.5 = fieldNorm(doc=5475)
   

5. Yang, Y.; Chute, C.G.A.: ¬A schematic analysis of the Unified Medical Language System (1992) 3.58

   3.584434 = sum of:
     3.584434 = weight(author_txt:yang in 6444) [ClassicSimilarity], result of:
       3.584434 = fieldWeight in 6444, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.168868 = idf(docFreq=92, maxDocs=44421)
         0.5 = fieldNorm(doc=6444)
   

Similar documents (content)

1. Lee, K.H.; Ng, M.K.M.; Lu, Q.: Text segmentation for Chinese spell checking (1999) 0.75

   0.74656284 = sum of:
     0.74656284 = product of:
       1.5553393 = sum of:
         0.004274765 = weight(abstract_txt:with in 4913) [ClassicSimilarity], result of:
           0.004274765 = score(doc=4913,freq=2.0), product of:
             0.019375296 = queryWeight, product of:
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0077621043 = queryNorm
             0.22062966 = fieldWeight in 4913, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.008864337 = weight(abstract_txt:based in 4913) [ClassicSimilarity], result of:
           0.008864337 = score(doc=4913,freq=2.0), product of:
             0.031506803 = queryWeight, product of:
               1.275199 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0077621043 = queryNorm
             0.28134674 = fieldWeight in 4913, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.041673843 = weight(abstract_txt:word in 4913) [ClassicSimilarity], result of:
           0.041673843 = score(doc=4913,freq=4.0), product of:
             0.06130682 = queryWeight, product of:
               1.4523944 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0077621043 = queryNorm
             0.67975867 = fieldWeight in 4913, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.036379803 = weight(abstract_txt:errors in 4913) [ClassicSimilarity], result of:
           0.036379803 = score(doc=4913,freq=1.0), product of:
             0.08889145 = queryWeight, product of:
               1.7488806 = boost
               6.548176 = idf(docFreq=172, maxDocs=44421)
               0.0077621043 = queryNorm
             0.409261 = fieldWeight in 4913, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.548176 = idf(docFreq=172, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.040278755 = weight(abstract_txt:detection in 4913) [ClassicSimilarity], result of:
           0.040278755 = score(doc=4913,freq=1.0), product of:
             0.095134296 = queryWeight, product of:
               1.8092506 = boost
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.0077621043 = queryNorm
             0.42338836 = fieldWeight in 4913, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.10478304 = weight(abstract_txt:ambiguities in 4913) [ClassicSimilarity], result of:
           0.10478304 = score(doc=4913,freq=2.0), product of:
             0.14282484 = queryWeight, product of:
               2.2168283 = boost
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.0077621043 = queryNorm
             0.73364717 = fieldWeight in 4913, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.031695284 = weight(abstract_txt:proposed in 4913) [ClassicSimilarity], result of:
           0.031695284 = score(doc=4913,freq=1.0), product of:
             0.11005156 = queryWeight, product of:
               3.076793 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0077621043 = queryNorm
             0.28800395 = fieldWeight in 4913, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.09952907 = weight(abstract_txt:words in 4913) [ClassicSimilarity], result of:
           0.09952907 = score(doc=4913,freq=4.0), product of:
             0.14866649 = queryWeight, product of:
               3.5760758 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0077621043 = queryNorm
             0.6694789 = fieldWeight in 4913, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.05923092 = weight(abstract_txt:text in 4913) [ClassicSimilarity], result of:
           0.05923092 = score(doc=4913,freq=3.0), product of:
             0.13540405 = queryWeight, product of:
               4.31694 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0077621043 = queryNorm
             0.4374383 = fieldWeight in 4913, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.08342116 = weight(abstract_txt:method in 4913) [ClassicSimilarity], result of:
           0.08342116 = score(doc=4913,freq=2.0), product of:
             0.20978957 = queryWeight, product of:
               6.0076847 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0077621043 = queryNorm
             0.39764208 = fieldWeight in 4913, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.39656687 = weight(abstract_txt:chinese in 4913) [ClassicSimilarity], result of:
           0.39656687 = score(doc=4913,freq=6.0), product of:
             0.41124782 = queryWeight, product of:
               8.411375 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0077621043 = queryNorm
             0.96430147 = fieldWeight in 4913, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
         0.6486415 = weight(abstract_txt:segmentation in 4913) [ClassicSimilarity], result of:
           0.6486415 = score(doc=4913,freq=4.0), product of:
             0.65352106 = queryWeight, product of:
               10.603396 = boost
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0077621043 = queryNorm
             0.99253345 = fieldWeight in 4913, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0625 = fieldNorm(doc=4913)
       0.48 = coord(12/25)
   

2. Wang, F.L.; Yang, C.C.: Mining Web data for Chinese segmentation (2007) 0.71

   0.70889443 = sum of:
     0.70889443 = product of:
       1.7722361 = sum of:
         0.004274765 = weight(abstract_txt:with in 1604) [ClassicSimilarity], result of:
           0.004274765 = score(doc=1604,freq=2.0), product of:
             0.019375296 = queryWeight, product of:
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0077621043 = queryNorm
             0.22062966 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.006268033 = weight(abstract_txt:based in 1604) [ClassicSimilarity], result of:
           0.006268033 = score(doc=1604,freq=1.0), product of:
             0.031506803 = queryWeight, product of:
               1.275199 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0077621043 = queryNorm
             0.1989422 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.029467857 = weight(abstract_txt:word in 1604) [ClassicSimilarity], result of:
           0.029467857 = score(doc=1604,freq=2.0), product of:
             0.06130682 = queryWeight, product of:
               1.4523944 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0077621043 = queryNorm
             0.48066196 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.040278755 = weight(abstract_txt:detection in 1604) [ClassicSimilarity], result of:
           0.040278755 = score(doc=1604,freq=1.0), product of:
             0.095134296 = queryWeight, product of:
               1.8092506 = boost
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.0077621043 = queryNorm
             0.42338836 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.07259187 = weight(abstract_txt:unknown in 1604) [ClassicSimilarity], result of:
           0.07259187 = score(doc=1604,freq=1.0), product of:
             0.16127814 = queryWeight, product of:
               2.8851178 = boost
               7.201658 = idf(docFreq=89, maxDocs=44421)
               0.0077621043 = queryNorm
             0.4501036 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.201658 = idf(docFreq=89, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.031695284 = weight(abstract_txt:proposed in 1604) [ClassicSimilarity], result of:
           0.031695284 = score(doc=1604,freq=1.0), product of:
             0.11005156 = queryWeight, product of:
               3.076793 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0077621043 = queryNorm
             0.28800395 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.09952907 = weight(abstract_txt:words in 1604) [ClassicSimilarity], result of:
           0.09952907 = score(doc=1604,freq=4.0), product of:
             0.14866649 = queryWeight, product of:
               3.5760758 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0077621043 = queryNorm
             0.6694789 = fieldWeight in 1604, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.034196988 = weight(abstract_txt:text in 1604) [ClassicSimilarity], result of:
           0.034196988 = score(doc=1604,freq=1.0), product of:
             0.13540405 = queryWeight, product of:
               4.31694 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0077621043 = queryNorm
             0.25255513 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.42834115 = weight(abstract_txt:chinese in 1604) [ClassicSimilarity], result of:
           0.42834115 = score(doc=1604,freq=7.0), product of:
             0.41124782 = queryWeight, product of:
               8.411375 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0077621043 = queryNorm
             1.0415646 = fieldWeight in 1604, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         1.0255923 = weight(abstract_txt:segmentation in 1604) [ClassicSimilarity], result of:
           1.0255923 = score(doc=1604,freq=10.0), product of:
             0.65352106 = queryWeight, product of:
               10.603396 = boost
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0077621043 = queryNorm
             1.5693332 = fieldWeight in 1604, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
       0.4 = coord(10/25)
   

3. Khoo, C.S.G.; Dai, D.; Loh, T.E.: Using statistical and contextual information to identify two- and three-character words in Chinese text (2002) 0.54

   0.54177314 = sum of:
     0.54177314 = product of:
       1.5049254 = sum of:
         0.004274765 = weight(abstract_txt:with in 206) [ClassicSimilarity], result of:
           0.004274765 = score(doc=206,freq=2.0), product of:
             0.019375296 = queryWeight, product of:
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0077621043 = queryNorm
             0.22062966 = fieldWeight in 206, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.045183904 = weight(abstract_txt:adjacent in 206) [ClassicSimilarity], result of:
           0.045183904 = score(doc=206,freq=1.0), product of:
             0.08152027 = queryWeight, product of:
               1.1842625 = boost
               8.868255 = idf(docFreq=16, maxDocs=44421)
               0.0077621043 = queryNorm
             0.5542659 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.868255 = idf(docFreq=16, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.020836921 = weight(abstract_txt:word in 206) [ClassicSimilarity], result of:
           0.020836921 = score(doc=206,freq=1.0), product of:
             0.06130682 = queryWeight, product of:
               1.4523944 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0077621043 = queryNorm
             0.33987933 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.14313924 = weight(abstract_txt:grams in 206) [ClassicSimilarity], result of:
           0.14313924 = score(doc=206,freq=4.0), product of:
             0.13956362 = queryWeight, product of:
               2.1913729 = boost
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.0077621043 = queryNorm
             1.02562 = fieldWeight in 206, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.03316449 = weight(abstract_txt:statistical in 206) [ClassicSimilarity], result of:
           0.03316449 = score(doc=206,freq=1.0), product of:
             0.09566779 = queryWeight, product of:
               2.2220747 = boost
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.0077621043 = queryNorm
             0.3466631 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.14929362 = weight(abstract_txt:words in 206) [ClassicSimilarity], result of:
           0.14929362 = score(doc=206,freq=9.0), product of:
             0.14866649 = queryWeight, product of:
               3.5760758 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0077621043 = queryNorm
             1.0042183 = fieldWeight in 206, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.034196988 = weight(abstract_txt:text in 206) [ClassicSimilarity], result of:
           0.034196988 = score(doc=206,freq=1.0), product of:
             0.13540405 = queryWeight, product of:
               4.31694 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0077621043 = queryNorm
             0.25255513 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.28041512 = weight(abstract_txt:chinese in 206) [ClassicSimilarity], result of:
           0.28041512 = score(doc=206,freq=3.0), product of:
             0.41124782 = queryWeight, product of:
               8.411375 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0077621043 = queryNorm
             0.6818641 = fieldWeight in 206, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.79442036 = weight(abstract_txt:segmentation in 206) [ClassicSimilarity], result of:
           0.79442036 = score(doc=206,freq=6.0), product of:
             0.65352106 = queryWeight, product of:
               10.603396 = boost
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0077621043 = queryNorm
             1.2156003 = fieldWeight in 206, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
       0.36 = coord(9/25)
   

4. Peng, F.; Huang, X.: Machine learning for Asian language text classification (2007) 0.45

   0.45051783 = sum of:
     0.45051783 = product of:
       1.4078683 = sum of:
         0.005235497 = weight(abstract_txt:with in 1831) [ClassicSimilarity], result of:
           0.005235497 = score(doc=1831,freq=3.0), product of:
             0.019375296 = queryWeight, product of:
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0077621043 = queryNorm
             0.27021506 = fieldWeight in 1831, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.010856551 = weight(abstract_txt:based in 1831) [ClassicSimilarity], result of:
           0.010856551 = score(doc=1831,freq=3.0), product of:
             0.031506803 = queryWeight, product of:
               1.275199 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0077621043 = queryNorm
             0.344578 = fieldWeight in 1831, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.041673843 = weight(abstract_txt:word in 1831) [ClassicSimilarity], result of:
           0.041673843 = score(doc=1831,freq=4.0), product of:
             0.06130682 = queryWeight, product of:
               1.4523944 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0077621043 = queryNorm
             0.67975867 = fieldWeight in 1831, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.05860355 = weight(abstract_txt:boundary in 1831) [ClassicSimilarity], result of:
           0.05860355 = score(doc=1831,freq=1.0), product of:
             0.12215251 = queryWeight, product of:
               2.05013 = boost
               7.676116 = idf(docFreq=55, maxDocs=44421)
               0.0077621043 = queryNorm
             0.47975725 = fieldWeight in 1831, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.676116 = idf(docFreq=55, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.03316449 = weight(abstract_txt:statistical in 1831) [ClassicSimilarity], result of:
           0.03316449 = score(doc=1831,freq=1.0), product of:
             0.09566779 = queryWeight, product of:
               2.2220747 = boost
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.0077621043 = queryNorm
             0.3466631 = fieldWeight in 1831, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.07646679 = weight(abstract_txt:text in 1831) [ClassicSimilarity], result of:
           0.07646679 = score(doc=1831,freq=5.0), product of:
             0.13540405 = queryWeight, product of:
               4.31694 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0077621043 = queryNorm
             0.56473047 = fieldWeight in 1831, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.32379547 = weight(abstract_txt:chinese in 1831) [ClassicSimilarity], result of:
           0.32379547 = score(doc=1831,freq=4.0), product of:
             0.41124782 = queryWeight, product of:
               8.411375 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0077621043 = queryNorm
             0.7873488 = fieldWeight in 1831, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
         0.85807204 = weight(abstract_txt:segmentation in 1831) [ClassicSimilarity], result of:
           0.85807204 = score(doc=1831,freq=7.0), product of:
             0.65352106 = queryWeight, product of:
               10.603396 = boost
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0077621043 = queryNorm
             1.3129983 = fieldWeight in 1831, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0625 = fieldNorm(doc=1831)
       0.32 = coord(8/25)
   

5. Xinglin, L.: Automatic summarization method based on compound word recognition (2015) 0.39

   0.39477706 = sum of:
     0.39477706 = product of:
       0.8972206 = sum of:
         0.0030227154 = weight(abstract_txt:with in 2841) [ClassicSimilarity], result of:
           0.0030227154 = score(doc=2841,freq=1.0), product of:
             0.019375296 = queryWeight, product of:
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0077621043 = queryNorm
             0.15600874 = fieldWeight in 2841, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.011490115 = weight(abstract_txt:problem in 2841) [ClassicSimilarity], result of:
           0.011490115 = score(doc=2841,freq=1.0), product of:
             0.041225825 = queryWeight, product of:
               1.1910084 = boost
               4.4593854 = idf(docFreq=1396, maxDocs=44421)
               0.0077621043 = queryNorm
             0.2787116 = fieldWeight in 2841, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4593854 = idf(docFreq=1396, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.006268033 = weight(abstract_txt:based in 2841) [ClassicSimilarity], result of:
           0.006268033 = score(doc=2841,freq=1.0), product of:
             0.031506803 = queryWeight, product of:
               1.275199 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0077621043 = queryNorm
             0.1989422 = fieldWeight in 2841, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.036090605 = weight(abstract_txt:word in 2841) [ClassicSimilarity], result of:
           0.036090605 = score(doc=2841,freq=3.0), product of:
             0.06130682 = queryWeight, product of:
               1.4523944 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0077621043 = queryNorm
             0.58868825 = fieldWeight in 2841, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.07259187 = weight(abstract_txt:unknown in 2841) [ClassicSimilarity], result of:
           0.07259187 = score(doc=2841,freq=1.0), product of:
             0.16127814 = queryWeight, product of:
               2.8851178 = boost
               7.201658 = idf(docFreq=89, maxDocs=44421)
               0.0077621043 = queryNorm
             0.4501036 = fieldWeight in 2841, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.201658 = idf(docFreq=89, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.0448239 = weight(abstract_txt:proposed in 2841) [ClassicSimilarity], result of:
           0.0448239 = score(doc=2841,freq=2.0), product of:
             0.11005156 = queryWeight, product of:
               3.076793 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0077621043 = queryNorm
             0.4072991 = fieldWeight in 2841, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.070377685 = weight(abstract_txt:words in 2841) [ClassicSimilarity], result of:
           0.070377685 = score(doc=2841,freq=2.0), product of:
             0.14866649 = queryWeight, product of:
               3.5760758 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0077621043 = queryNorm
             0.47339305 = fieldWeight in 2841, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.04836184 = weight(abstract_txt:text in 2841) [ClassicSimilarity], result of:
           0.04836184 = score(doc=2841,freq=2.0), product of:
             0.13540405 = queryWeight, product of:
               4.31694 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0077621043 = queryNorm
             0.3571669 = fieldWeight in 2841, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.11797534 = weight(abstract_txt:method in 2841) [ClassicSimilarity], result of:
           0.11797534 = score(doc=2841,freq=4.0), product of:
             0.20978957 = queryWeight, product of:
               6.0076847 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0077621043 = queryNorm
             0.5623508 = fieldWeight in 2841, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.16189773 = weight(abstract_txt:chinese in 2841) [ClassicSimilarity], result of:
           0.16189773 = score(doc=2841,freq=1.0), product of:
             0.41124782 = queryWeight, product of:
               8.411375 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0077621043 = queryNorm
             0.3936744 = fieldWeight in 2841, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
         0.32432076 = weight(abstract_txt:segmentation in 2841) [ClassicSimilarity], result of:
           0.32432076 = score(doc=2841,freq=1.0), product of:
             0.65352106 = queryWeight, product of:
               10.603396 = boost
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0077621043 = queryNorm
             0.49626672 = fieldWeight in 2841, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.9402676 = idf(docFreq=42, maxDocs=44421)
               0.0625 = fieldNorm(doc=2841)
       0.44 = coord(11/25)