Document (#29486)

Author

Lehtokangas, R.
Järvelin, K.

Title

Consistency of textual expression in newspaper articles : an argument for semantically based query expansion

Source

Journal of documentation. 57(2001) no.4, S.535-548

Year

2001

Abstract

This article investigates how consistent different newspapers are in their choice of words when writing about the same news events. News articles on the same news events were taken from three Finnish newspapers and compared in regard to their central concepts and words representing the concepts in the news texts. Consistency figures were calculated for each set of three articles (the total number of sets was sixty). Inconsistency in words and concepts was found between news articles from different newspapers. The mean value of consistency calculated on the basis of words was 65 per cent; this however depended on the article length. For short news wires consistency was 83 per cent while for long articles it was only 47 per cent. At the concept level, consistency was considerably higher, ranging from 92 per cent to 97 per cent between short and long articles. The articles also represented three categories of topic (event, process and opinion). Statistically significant differences in consistency were found in regard to length but not in regard to the categories of topic. We argue that the expression inconsistency is a clear sign of a retrieval problem and that query expansion based on semantic relationships can significantly improve retrieval performance on free-text sources.

Footnote

Vgl. auch: http://www.emeraldinsight.com/10.1108/EUM0000000007104.

Theme

Semantisches Umfeld in Indexierung u. Retrieval

Similar documents (author)

1. Järvelin, K.: ¬An analysis of two approaches in information retrieval : from frameworks to study designs (2007) 4.99

   4.9924345 = sum of:
     4.9924345 = weight(author_txt:järvelin in 1326) [ClassicSimilarity], result of:
       4.9924345 = fieldWeight in 1326, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.9878955 = idf(docFreq=40, maxDocs=44421)
         0.625 = fieldNorm(doc=1326)
   

2. Järvelin, K.: Evaluation (2011) 4.99

   4.9924345 = sum of:
     4.9924345 = weight(author_txt:järvelin in 1548) [ClassicSimilarity], result of:
       4.9924345 = fieldWeight in 1548, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.9878955 = idf(docFreq=40, maxDocs=44421)
         0.625 = fieldNorm(doc=1548)
   

3. Järvelin, K.; Vakkari, P.: ¬The evolution of library and information science 1965-1985 : a content analysis of journal titles (1993) 3.99

   3.9939477 = sum of:
     3.9939477 = weight(author_txt:järvelin in 4648) [ClassicSimilarity], result of:
       3.9939477 = fieldWeight in 4648, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.9878955 = idf(docFreq=40, maxDocs=44421)
         0.5 = fieldNorm(doc=4648)
   

4. Kristensen, J.; Järvelin, K.: ¬The effectiveness of a searching thesaurus in free-text searching in a full-text database (1990) 3.99

   3.9939477 = sum of:
     3.9939477 = weight(author_txt:järvelin in 2111) [ClassicSimilarity], result of:
       3.9939477 = fieldWeight in 2111, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.9878955 = idf(docFreq=40, maxDocs=44421)
         0.5 = fieldNorm(doc=2111)
   

5. Pirkola, A.; Järvelin, K.: Employing the resolution power of search keys (2001) 3.99

   3.9939477 = sum of:
     3.9939477 = weight(author_txt:järvelin in 6907) [ClassicSimilarity], result of:
       3.9939477 = fieldWeight in 6907, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.9878955 = idf(docFreq=40, maxDocs=44421)
         0.5 = fieldNorm(doc=6907)
   

Similar documents (content)

1. Khoo, C.S.G.; Dai, D.; Loh, T.E.: Using statistical and contextual information to identify two- and three-character words in Chinese text (2002) 0.16

   0.15789635 = sum of:
     0.15789635 = product of:
       0.56391555 = sum of:
         0.018135255 = weight(abstract_txt:found in 206) [ClassicSimilarity], result of:
           0.018135255 = score(doc=206,freq=1.0), product of:
             0.06489002 = queryWeight, product of:
               1.0149994 = boost
               4.4716287 = idf(docFreq=1379, maxDocs=44421)
               0.014297048 = queryNorm
             0.2794768 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4716287 = idf(docFreq=1379, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.03001599 = weight(abstract_txt:were in 206) [ClassicSimilarity], result of:
           0.03001599 = score(doc=206,freq=4.0), product of:
             0.06547482 = queryWeight, product of:
               1.2487044 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.014297048 = queryNorm
             0.4584356 = fieldWeight in 206, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.044981193 = weight(abstract_txt:three in 206) [ClassicSimilarity], result of:
           0.044981193 = score(doc=206,freq=3.0), product of:
             0.09437119 = queryWeight, product of:
               1.4991399 = boost
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.014297048 = queryNorm
             0.47664115 = fieldWeight in 206, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.07757536 = weight(abstract_txt:calculated in 206) [ClassicSimilarity], result of:
           0.07757536 = score(doc=206,freq=1.0), product of:
             0.17099287 = queryWeight, product of:
               1.6476533 = boost
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.014297048 = queryNorm
             0.45367602 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.18696414 = weight(abstract_txt:words in 206) [ClassicSimilarity], result of:
           0.18696414 = score(doc=206,freq=9.0), product of:
             0.18617877 = queryWeight, product of:
               2.431403 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.014297048 = queryNorm
             1.0042183 = fieldWeight in 206, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.07752223 = weight(abstract_txt:articles in 206) [ClassicSimilarity], result of:
           0.07752223 = score(doc=206,freq=1.0), product of:
             0.25949898 = queryWeight, product of:
               3.7973332 = boost
               4.7798095 = idf(docFreq=1013, maxDocs=44421)
               0.014297048 = queryNorm
             0.2987381 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7798095 = idf(docFreq=1013, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
         0.1287214 = weight(abstract_txt:news in 206) [ClassicSimilarity], result of:
           0.1287214 = score(doc=206,freq=1.0), product of:
             0.34564984 = queryWeight, product of:
               4.057473 = boost
               5.9584646 = idf(docFreq=311, maxDocs=44421)
               0.014297048 = queryNorm
             0.37240404 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9584646 = idf(docFreq=311, maxDocs=44421)
               0.0625 = fieldNorm(doc=206)
       0.28 = coord(7/25)
   

2. Iivonen, M.: ¬The impact of the indexing environment on interindexer consistency (1990) 0.14

   0.13985617 = sum of:
     0.13985617 = product of:
       0.8741011 = sum of:
         0.03001599 = weight(abstract_txt:were in 4778) [ClassicSimilarity], result of:
           0.03001599 = score(doc=4778,freq=1.0), product of:
             0.06547482 = queryWeight, product of:
               1.2487044 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.014297048 = queryNorm
             0.4584356 = fieldWeight in 4778, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.125 = fieldNorm(doc=4778)
         0.057316113 = weight(abstract_txt:concepts in 4778) [ClassicSimilarity], result of:
           0.057316113 = score(doc=4778,freq=1.0), product of:
             0.10077598 = queryWeight, product of:
               1.5491768 = boost
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.014297048 = queryNorm
             0.56874776 = fieldWeight in 4778, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.125 = fieldNorm(doc=4778)
         0.15515073 = weight(abstract_txt:calculated in 4778) [ClassicSimilarity], result of:
           0.15515073 = score(doc=4778,freq=1.0), product of:
             0.17099287 = queryWeight, product of:
               1.6476533 = boost
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.014297048 = queryNorm
             0.90735203 = fieldWeight in 4778, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.125 = fieldNorm(doc=4778)
         0.63161826 = weight(abstract_txt:consistency in 4778) [ClassicSimilarity], result of:
           0.63161826 = score(doc=4778,freq=4.0), product of:
             0.39609468 = queryWeight, product of:
               4.3434715 = boost
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.014297048 = queryNorm
             1.5946144 = fieldWeight in 4778, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.125 = fieldNorm(doc=4778)
       0.16 = coord(4/25)
   

3. Buccio, E. Di; Melucci, M.; Moro, F.: Detecting verbose queries and improving information retrieval (2014) 0.14

   0.13657014 = sum of:
     0.13657014 = product of:
       0.4267817 = sum of:
         0.068934835 = weight(abstract_txt:query in 3695) [ClassicSimilarity], result of:
           0.068934835 = score(doc=3695,freq=10.0), product of:
             0.07335921 = queryWeight, product of:
               1.0792055 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.014297048 = queryNorm
             0.939689 = fieldWeight in 3695, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.058541007 = weight(abstract_txt:topic in 3695) [ClassicSimilarity], result of:
           0.058541007 = score(doc=3695,freq=5.0), product of:
             0.08288557 = queryWeight, product of:
               1.1471397 = boost
               5.053779 = idf(docFreq=770, maxDocs=44421)
               0.014297048 = queryNorm
             0.7062871 = fieldWeight in 3695, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.053779 = idf(docFreq=770, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.029208478 = weight(abstract_txt:long in 3695) [ClassicSimilarity], result of:
           0.029208478 = score(doc=3695,freq=1.0), product of:
             0.08915959 = queryWeight, product of:
               1.1897641 = boost
               5.2415633 = idf(docFreq=638, maxDocs=44421)
               0.014297048 = queryNorm
             0.3275977 = fieldWeight in 3695, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2415633 = idf(docFreq=638, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.03955019 = weight(abstract_txt:short in 3695) [ClassicSimilarity], result of:
           0.03955019 = score(doc=3695,freq=1.0), product of:
             0.1091259 = queryWeight, product of:
               1.3162572 = boost
               5.7988343 = idf(docFreq=365, maxDocs=44421)
               0.014297048 = queryNorm
             0.36242715 = fieldWeight in 3695, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7988343 = idf(docFreq=365, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.113597855 = weight(abstract_txt:length in 3695) [ClassicSimilarity], result of:
           0.113597855 = score(doc=3695,freq=4.0), product of:
             0.13890639 = queryWeight, product of:
               1.4850394 = boost
               6.5424123 = idf(docFreq=173, maxDocs=44421)
               0.014297048 = queryNorm
             0.81780154 = fieldWeight in 3695, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.5424123 = idf(docFreq=173, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.025969904 = weight(abstract_txt:three in 3695) [ClassicSimilarity], result of:
           0.025969904 = score(doc=3695,freq=1.0), product of:
             0.09437119 = queryWeight, product of:
               1.4991399 = boost
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.014297048 = queryNorm
             0.2751889 = fieldWeight in 3695, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.028658057 = weight(abstract_txt:concepts in 3695) [ClassicSimilarity], result of:
           0.028658057 = score(doc=3695,freq=1.0), product of:
             0.10077598 = queryWeight, product of:
               1.5491768 = boost
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.014297048 = queryNorm
             0.28437388 = fieldWeight in 3695, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
         0.06232138 = weight(abstract_txt:words in 3695) [ClassicSimilarity], result of:
           0.06232138 = score(doc=3695,freq=1.0), product of:
             0.18617877 = queryWeight, product of:
               2.431403 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.014297048 = queryNorm
             0.33473945 = fieldWeight in 3695, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=3695)
       0.32 = coord(8/25)
   

4. Iivonen, M.: Interindexer consistency and the indexing environment (1990) 0.12

   0.121926494 = sum of:
     0.121926494 = product of:
       0.7620406 = sum of:
         0.022511993 = weight(abstract_txt:were in 3661) [ClassicSimilarity], result of:
           0.022511993 = score(doc=3661,freq=1.0), product of:
             0.06547482 = queryWeight, product of:
               1.2487044 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.014297048 = queryNorm
             0.3438267 = fieldWeight in 3661, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.09375 = fieldNorm(doc=3661)
         0.042987086 = weight(abstract_txt:concepts in 3661) [ClassicSimilarity], result of:
           0.042987086 = score(doc=3661,freq=1.0), product of:
             0.10077598 = queryWeight, product of:
               1.5491768 = boost
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.014297048 = queryNorm
             0.42656082 = fieldWeight in 3661, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.09375 = fieldNorm(doc=3661)
         0.11636305 = weight(abstract_txt:calculated in 3661) [ClassicSimilarity], result of:
           0.11636305 = score(doc=3661,freq=1.0), product of:
             0.17099287 = queryWeight, product of:
               1.6476533 = boost
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.014297048 = queryNorm
             0.68051404 = fieldWeight in 3661, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.09375 = fieldNorm(doc=3661)
         0.5801785 = weight(abstract_txt:consistency in 3661) [ClassicSimilarity], result of:
           0.5801785 = score(doc=3661,freq=6.0), product of:
             0.39609468 = queryWeight, product of:
               4.3434715 = boost
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.014297048 = queryNorm
             1.464747 = fieldWeight in 3661, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.09375 = fieldNorm(doc=3661)
       0.16 = coord(4/25)
   

5. Iivonen, M.: Factors lowering the consistency in online searching (1995) 0.12

   0.11648893 = sum of:
     0.11648893 = product of:
       0.58244467 = sum of:
         0.056192968 = weight(abstract_txt:same in 3937) [ClassicSimilarity], result of:
           0.056192968 = score(doc=3937,freq=3.0), product of:
             0.07297636 = queryWeight, product of:
               1.0763857 = boost
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.014297048 = queryNorm
             0.7700161 = fieldWeight in 3937, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.09375 = fieldNorm(doc=3937)
         0.032698665 = weight(abstract_txt:query in 3937) [ClassicSimilarity], result of:
           0.032698665 = score(doc=3937,freq=1.0), product of:
             0.07335921 = queryWeight, product of:
               1.0792055 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.014297048 = queryNorm
             0.4457336 = fieldWeight in 3937, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.09375 = fieldNorm(doc=3937)
         0.022511993 = weight(abstract_txt:were in 3937) [ClassicSimilarity], result of:
           0.022511993 = score(doc=3937,freq=1.0), product of:
             0.06547482 = queryWeight, product of:
               1.2487044 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.014297048 = queryNorm
             0.3438267 = fieldWeight in 3937, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.09375 = fieldNorm(doc=3937)
         0.060792916 = weight(abstract_txt:concepts in 3937) [ClassicSimilarity], result of:
           0.060792916 = score(doc=3937,freq=2.0), product of:
             0.10077598 = queryWeight, product of:
               1.5491768 = boost
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.014297048 = queryNorm
             0.60324806 = fieldWeight in 3937, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.549982 = idf(docFreq=1275, maxDocs=44421)
               0.09375 = fieldNorm(doc=3937)
         0.4102481 = weight(abstract_txt:consistency in 3937) [ClassicSimilarity], result of:
           0.4102481 = score(doc=3937,freq=3.0), product of:
             0.39609468 = queryWeight, product of:
               4.3434715 = boost
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.014297048 = queryNorm
             1.0357324 = fieldWeight in 3937, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.09375 = fieldNorm(doc=3937)
       0.2 = coord(5/25)