Document (#35332)

Author

Liu, R.-L.

Title

Context-based term frequency assessment for text classification

Source

Journal of the American Society for Information Science and Technology. 61(2010) no.2, S.300-309

Year

2010

Abstract

Automatic text classification (TC) is essential for the management of information. To properly classify a document d, it is essential to identify the semantics of each term t in d, while the semantics heavily depend on context (neighboring terms) of t in d. Therefore, we present a technique CTFA (Context-based Term Frequency Assessment) that improves text classifiers by considering term contexts in test documents. The results of the term context recognition are used to assess term frequencies of terms, and hence CTFA may easily work with various kinds of text classifiers that base their TC decisions on term frequencies, without needing to modify the classifiers. Moreover, CTFA is efficient, and neither huge memory nor domain-specific knowledge is required. Empirical results show that CTFA successfully enhances performance of several kinds of text classifiers on different experimental data.

Theme

Automatisches Klassifizieren

Object

Context-based Term Frequency Assessment

Similar documents (content)

1. Liu, R.-L.: ¬A passage extractor for classification of disease aspect information (2013) 0.30

   0.3000548 = sum of:
     0.3000548 = product of:
       1.0716243 = sum of:
         0.010525952 = weight(abstract_txt:that in 2107) [ClassicSimilarity], result of:
           0.010525952 = score(doc=2107,freq=3.0), product of:
             0.041115116 = queryWeight, product of:
               1.0614312 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.016379114 = queryNorm
             0.25601172 = fieldWeight in 2107, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
         0.07294844 = weight(abstract_txt:enhances in 2107) [ClassicSimilarity], result of:
           0.07294844 = score(doc=2107,freq=1.0), product of:
             0.14945298 = queryWeight, product of:
               1.1683763 = boost
               7.809647 = idf(docFreq=48, maxDocs=44421)
               0.016379114 = queryNorm
             0.48810294 = fieldWeight in 2107, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.809647 = idf(docFreq=48, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
         0.033754837 = weight(abstract_txt:classification in 2107) [ClassicSimilarity], result of:
           0.033754837 = score(doc=2107,freq=3.0), product of:
             0.07810656 = queryWeight, product of:
               1.1945084 = boost
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.016379114 = queryNorm
             0.43216392 = fieldWeight in 2107, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
         0.05337717 = weight(abstract_txt:essential in 2107) [ClassicSimilarity], result of:
           0.05337717 = score(doc=2107,freq=1.0), product of:
             0.15290004 = queryWeight, product of:
               1.6712803 = boost
               5.5855756 = idf(docFreq=452, maxDocs=44421)
               0.016379114 = queryNorm
             0.34909847 = fieldWeight in 2107, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5855756 = idf(docFreq=452, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
         0.14291088 = weight(abstract_txt:text in 2107) [ClassicSimilarity], result of:
           0.14291088 = score(doc=2107,freq=8.0), product of:
             0.20006178 = queryWeight, product of:
               3.0227172 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.016379114 = queryNorm
             0.7143338 = fieldWeight in 2107, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
         0.63993686 = weight(abstract_txt:classifiers in 2107) [ClassicSimilarity], result of:
           0.63993686 = score(doc=2107,freq=6.0), product of:
             0.5553348 = queryWeight, product of:
               4.50441 = boost
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.016379114 = queryNorm
             1.1523442 = fieldWeight in 2107, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
         0.11817012 = weight(abstract_txt:term in 2107) [ClassicSimilarity], result of:
           0.11817012 = score(doc=2107,freq=1.0), product of:
             0.39433473 = queryWeight, product of:
               5.021252 = boost
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.016379114 = queryNorm
             0.29966956 = fieldWeight in 2107, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.0625 = fieldNorm(doc=2107)
       0.28 = coord(7/25)
   

2. Altinel, B.; Ganiz, M.C.: Semantic text classification : a survey of past and recent advances (2018) 0.26

   0.25810215 = sum of:
     0.25810215 = product of:
       0.7169504 = sum of:
         0.0092102075 = weight(abstract_txt:that in 51) [ClassicSimilarity], result of:
           0.0092102075 = score(doc=51,freq=3.0), product of:
             0.041115116 = queryWeight, product of:
               1.0614312 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.016379114 = queryNorm
             0.22401026 = fieldWeight in 51, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.06148301 = weight(abstract_txt:classification in 51) [ClassicSimilarity], result of:
           0.06148301 = score(doc=51,freq=13.0), product of:
             0.07810656 = queryWeight, product of:
               1.1945084 = boost
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.016379114 = queryNorm
             0.7871683 = fieldWeight in 51, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.017721502 = weight(abstract_txt:terms in 51) [ClassicSimilarity], result of:
           0.017721502 = score(doc=51,freq=1.0), product of:
             0.08013684 = queryWeight, product of:
               1.2099338 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.016379114 = queryNorm
             0.2211405 = fieldWeight in 51, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.10791432 = weight(abstract_txt:neighboring in 51) [ClassicSimilarity], result of:
           0.10791432 = score(doc=51,freq=1.0), product of:
             0.2121003 = queryWeight, product of:
               1.3918778 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.016379114 = queryNorm
             0.5087891 = fieldWeight in 51, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.056424633 = weight(abstract_txt:frequency in 51) [ClassicSimilarity], result of:
           0.056424633 = score(doc=51,freq=1.0), product of:
             0.17343804 = queryWeight, product of:
               1.7799903 = boost
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.016379114 = queryNorm
             0.3253302 = fieldWeight in 51, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.08302822 = weight(abstract_txt:semantics in 51) [ClassicSimilarity], result of:
           0.08302822 = score(doc=51,freq=2.0), product of:
             0.1780898 = queryWeight, product of:
               1.8037028 = boost
               6.0281444 = idf(docFreq=290, maxDocs=44421)
               0.016379114 = queryNorm
             0.4662155 = fieldWeight in 51, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.0281444 = idf(docFreq=290, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.07553617 = weight(abstract_txt:context in 51) [ClassicSimilarity], result of:
           0.07553617 = score(doc=51,freq=3.0), product of:
             0.18403684 = queryWeight, product of:
               2.5930617 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.016379114 = queryNorm
             0.4104405 = fieldWeight in 51, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.15940431 = weight(abstract_txt:text in 51) [ClassicSimilarity], result of:
           0.15940431 = score(doc=51,freq=13.0), product of:
             0.20006178 = queryWeight, product of:
               3.0227172 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.016379114 = queryNorm
             0.7967754 = fieldWeight in 51, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
         0.14622806 = weight(abstract_txt:term in 51) [ClassicSimilarity], result of:
           0.14622806 = score(doc=51,freq=2.0), product of:
             0.39433473 = queryWeight, product of:
               5.021252 = boost
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.016379114 = queryNorm
             0.37082216 = fieldWeight in 51, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.0546875 = fieldNorm(doc=51)
       0.36 = coord(9/25)
   

3. Liu, R.-L.; Huang, Y.-C.: Ranker enhancement for proximity-based ranking of biomedical texts (2011) 0.20

   0.20354946 = sum of:
     0.20354946 = product of:
       0.63609207 = sum of:
         0.046431728 = weight(abstract_txt:improves in 947) [ClassicSimilarity], result of:
           0.046431728 = score(doc=947,freq=1.0), product of:
             0.110586934 = queryWeight, product of:
               1.0050378 = boost
               6.717861 = idf(docFreq=145, maxDocs=44421)
               0.016379114 = queryNorm
             0.41986632 = fieldWeight in 947, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.717861 = idf(docFreq=145, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.054047905 = weight(abstract_txt:huge in 947) [ClassicSimilarity], result of:
           0.054047905 = score(doc=947,freq=1.0), product of:
             0.12237137 = queryWeight, product of:
               1.0572321 = boost
               7.0667386 = idf(docFreq=102, maxDocs=44421)
               0.016379114 = queryNorm
             0.44167116 = fieldWeight in 947, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0667386 = idf(docFreq=102, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.013588947 = weight(abstract_txt:that in 947) [ClassicSimilarity], result of:
           0.013588947 = score(doc=947,freq=5.0), product of:
             0.041115116 = queryWeight, product of:
               1.0614312 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.016379114 = queryNorm
             0.33050975 = fieldWeight in 947, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.020253146 = weight(abstract_txt:terms in 947) [ClassicSimilarity], result of:
           0.020253146 = score(doc=947,freq=1.0), product of:
             0.08013684 = queryWeight, product of:
               1.2099338 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.016379114 = queryNorm
             0.252732 = fieldWeight in 947, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.060067646 = weight(abstract_txt:kinds in 947) [ClassicSimilarity], result of:
           0.060067646 = score(doc=947,freq=1.0), product of:
             0.16542368 = queryWeight, product of:
               1.7383783 = boost
               5.8098235 = idf(docFreq=361, maxDocs=44421)
               0.016379114 = queryNorm
             0.36311397 = fieldWeight in 947, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8098235 = idf(docFreq=361, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.0644853 = weight(abstract_txt:frequency in 947) [ClassicSimilarity], result of:
           0.0644853 = score(doc=947,freq=1.0), product of:
             0.17343804 = queryWeight, product of:
               1.7799903 = boost
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.016379114 = queryNorm
             0.37180594 = fieldWeight in 947, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.112980984 = weight(abstract_txt:text in 947) [ClassicSimilarity], result of:
           0.112980984 = score(doc=947,freq=5.0), product of:
             0.20006178 = queryWeight, product of:
               3.0227172 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.016379114 = queryNorm
             0.56473047 = fieldWeight in 947, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
         0.26423642 = weight(abstract_txt:term in 947) [ClassicSimilarity], result of:
           0.26423642 = score(doc=947,freq=5.0), product of:
             0.39433473 = queryWeight, product of:
               5.021252 = boost
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.016379114 = queryNorm
             0.6700815 = fieldWeight in 947, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.0625 = fieldNorm(doc=947)
       0.32 = coord(8/25)
   

4. Lee, D.L.; Ren, L.: Document ranking on weight-partitioned signature files (1996) 0.19

   0.18763825 = sum of:
     0.18763825 = product of:
       0.7818261 = sum of:
         0.019340763 = weight(abstract_txt:results in 3417) [ClassicSimilarity], result of:
           0.019340763 = score(doc=3417,freq=1.0), product of:
             0.059305113 = queryWeight, product of:
               1.0408581 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.016379114 = queryNorm
             0.32612303 = fieldWeight in 3417, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.09375 = fieldNorm(doc=3417)
         0.0091157425 = weight(abstract_txt:that in 3417) [ClassicSimilarity], result of:
           0.0091157425 = score(doc=3417,freq=1.0), product of:
             0.041115116 = queryWeight, product of:
               1.0614312 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.016379114 = queryNorm
             0.22171268 = fieldWeight in 3417, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.09375 = fieldNorm(doc=3417)
         0.030379716 = weight(abstract_txt:terms in 3417) [ClassicSimilarity], result of:
           0.030379716 = score(doc=3417,freq=1.0), product of:
             0.08013684 = queryWeight, product of:
               1.2099338 = boost
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.016379114 = queryNorm
             0.379098 = fieldWeight in 3417, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.043712 = idf(docFreq=2116, maxDocs=44421)
               0.09375 = fieldNorm(doc=3417)
         0.1675377 = weight(abstract_txt:frequency in 3417) [ClassicSimilarity], result of:
           0.1675377 = score(doc=3417,freq=3.0), product of:
             0.17343804 = queryWeight, product of:
               1.7799903 = boost
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.016379114 = queryNorm
             0.9659802 = fieldWeight in 3417, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.09375 = fieldNorm(doc=3417)
         0.20094179 = weight(abstract_txt:frequencies in 3417) [ClassicSimilarity], result of:
           0.20094179 = score(doc=3417,freq=1.0), product of:
             0.28237304 = queryWeight, product of:
               2.271209 = boost
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.016379114 = queryNorm
             0.7116182 = fieldWeight in 3417, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.09375 = fieldNorm(doc=3417)
         0.35451037 = weight(abstract_txt:term in 3417) [ClassicSimilarity], result of:
           0.35451037 = score(doc=3417,freq=4.0), product of:
             0.39433473 = queryWeight, product of:
               5.021252 = boost
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.016379114 = queryNorm
             0.8990087 = fieldWeight in 3417, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.09375 = fieldNorm(doc=3417)
       0.24 = coord(6/25)
   

5. Liu, R.-L.: Dynamic category profiling for text filtering and classification (2007) 0.18

   0.18375002 = sum of:
     0.18375002 = product of:
       0.91875005 = sum of:
         0.016117303 = weight(abstract_txt:results in 1900) [ClassicSimilarity], result of:
           0.016117303 = score(doc=1900,freq=1.0), product of:
             0.059305113 = queryWeight, product of:
               1.0408581 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.016379114 = queryNorm
             0.2717692 = fieldWeight in 1900, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.078125 = fieldNorm(doc=1900)
         0.015192904 = weight(abstract_txt:that in 1900) [ClassicSimilarity], result of:
           0.015192904 = score(doc=1900,freq=4.0), product of:
             0.041115116 = queryWeight, product of:
               1.0614312 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.016379114 = queryNorm
             0.3695211 = fieldWeight in 1900, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=1900)
         0.024360456 = weight(abstract_txt:classification in 1900) [ClassicSimilarity], result of:
           0.024360456 = score(doc=1900,freq=1.0), product of:
             0.07810656 = queryWeight, product of:
               1.1945084 = boost
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.016379114 = queryNorm
             0.31188744 = fieldWeight in 1900, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.078125 = fieldNorm(doc=1900)
         0.06315829 = weight(abstract_txt:text in 1900) [ClassicSimilarity], result of:
           0.06315829 = score(doc=1900,freq=1.0), product of:
             0.20006178 = queryWeight, product of:
               3.0227172 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.016379114 = queryNorm
             0.3156939 = fieldWeight in 1900, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.078125 = fieldNorm(doc=1900)
         0.7999211 = weight(abstract_txt:classifiers in 1900) [ClassicSimilarity], result of:
           0.7999211 = score(doc=1900,freq=6.0), product of:
             0.5553348 = queryWeight, product of:
               4.50441 = boost
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.016379114 = queryNorm
             1.4404303 = fieldWeight in 1900, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.078125 = fieldNorm(doc=1900)
       0.2 = coord(5/25)