Document (#16329)

Author
Shaw, W.M.
Burgin, R.
Howell, P.
Title
Performance standards and evaluations in IR test collections : vector-space and other retrieval models
Source
Information processing and management. 33(1997) no.1, S.15-36
Year
1997
Abstract
Computes low performance standards for each query and for the group of queries in 13 traditional and 4 TREC test collections. Predicted by the hypergeometric distribution, the standards represent the highest level of retrieval effectiveness attributable to chance. Compares operational levels of performance for vector-space, ad-hoc-feature-based, probabilistic, and other retrieval models to the standards. The effectiveness of these techniques in small, traditional test collections, can be explained by retrieving a few more relevant documents for most queries than expected by chance. The effectiveness of retrieval techniques in the larger TREC test collections can only be explained by retrieving many more relevant documents for most queries than expected by chance. The discrepancy between deviations form chance in traditional and TREC test collections is due to a decrease in performance standards for large test collections, not to an increase in operational performance. The next generation of information retrieval systems would be enhanced by abandoning uninformative performance summaries and focusing on effectiveness and improvements in effectiveness of individual queries
Theme
Retrievalstudien
Object
TREC

Similar documents (author)

  1. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 4.68
    4.677319 = sum of:
      4.677319 = sum of:
        1.7122161 = weight(author_txt:shaw in 325) [ClassicSimilarity], result of:
          1.7122161 = score(doc=325,freq=1.0), product of:
            0.56984204 = queryWeight, product of:
              8.0125885 = idf(docFreq=39, maxDocs=44421)
              0.07111835 = queryNorm
            3.0047207 = fieldWeight in 325, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.0125885 = idf(docFreq=39, maxDocs=44421)
              0.375 = fieldNorm(doc=325)
        2.965103 = weight(author_txt:burgin in 325) [ClassicSimilarity], result of:
          2.965103 = score(doc=325,freq=1.0), product of:
            0.8217542 = queryWeight, product of:
              1.2008637 = boost
              9.622026 = idf(docFreq=7, maxDocs=44421)
              0.07111835 = queryNorm
            3.60826 = fieldWeight in 325, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.622026 = idf(docFreq=7, maxDocs=44421)
              0.375 = fieldNorm(doc=325)
    
  2. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 2.47
    2.470919 = sum of:
      2.470919 = product of:
        4.941838 = sum of:
          4.941838 = weight(author_txt:burgin in 3307) [ClassicSimilarity], result of:
            4.941838 = score(doc=3307,freq=1.0), product of:
              0.8217542 = queryWeight, product of:
                1.2008637 = boost
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.07111835 = queryNorm
              6.0137663 = fieldWeight in 3307, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.625 = fieldNorm(doc=3307)
        0.5 = coord(1/2)
    
  3. Burgin, R.: ¬The retrieval effectiveness of 5 clustering algorithms as a function of indexing exhaustivity (1995) 2.47
    2.470919 = sum of:
      2.470919 = product of:
        4.941838 = sum of:
          4.941838 = weight(author_txt:burgin in 3433) [ClassicSimilarity], result of:
            4.941838 = score(doc=3433,freq=1.0), product of:
              0.8217542 = queryWeight, product of:
                1.2008637 = boost
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.07111835 = queryNorm
              6.0137663 = fieldWeight in 3433, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.625 = fieldNorm(doc=3433)
        0.5 = coord(1/2)
    
  4. Burgin, M.: Terminological and teleological systems (1996) 2.47
    2.470919 = sum of:
      2.470919 = product of:
        4.941838 = sum of:
          4.941838 = weight(author_txt:burgin in 6391) [ClassicSimilarity], result of:
            4.941838 = score(doc=6391,freq=1.0), product of:
              0.8217542 = queryWeight, product of:
                1.2008637 = boost
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.07111835 = queryNorm
              6.0137663 = fieldWeight in 6391, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.625 = fieldNorm(doc=6391)
        0.5 = coord(1/2)
    
  5. Burgin, R.: ¬The Monte Carlo method and the evaluation of retrieval system performance (1999) 2.47
    2.470919 = sum of:
      2.470919 = product of:
        4.941838 = sum of:
          4.941838 = weight(author_txt:burgin in 3946) [ClassicSimilarity], result of:
            4.941838 = score(doc=3946,freq=1.0), product of:
              0.8217542 = queryWeight, product of:
                1.2008637 = boost
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.07111835 = queryNorm
              6.0137663 = fieldWeight in 3946, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.622026 = idf(docFreq=7, maxDocs=44421)
                0.625 = fieldNorm(doc=3946)
        0.5 = coord(1/2)
    

Similar documents (content)

  1. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 0.55
    0.545292 = sum of:
      0.545292 = product of:
        1.36323 = sum of:
          0.13767985 = weight(abstract_txt:computes in 325) [ClassicSimilarity], result of:
            0.13767985 = score(doc=325,freq=1.0), product of:
              0.16329554 = queryWeight, product of:
                1.0905548 = boost
                8.993418 = idf(docFreq=14, maxDocs=44421)
                0.016649531 = queryNorm
              0.8431329 = fieldWeight in 325, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.993418 = idf(docFreq=14, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.035239477 = weight(abstract_txt:techniques in 325) [ClassicSimilarity], result of:
            0.035239477 = score(doc=325,freq=1.0), product of:
              0.08293866 = queryWeight, product of:
                1.0991422 = boost
                4.5321174 = idf(docFreq=1298, maxDocs=44421)
                0.016649531 = queryNorm
              0.424886 = fieldWeight in 325, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.5321174 = idf(docFreq=1298, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.10628869 = weight(abstract_txt:explained in 325) [ClassicSimilarity], result of:
            0.10628869 = score(doc=325,freq=1.0), product of:
              0.17313921 = queryWeight, product of:
                1.5880826 = boost
                6.548176 = idf(docFreq=172, maxDocs=44421)
                0.016649531 = queryNorm
              0.6138915 = fieldWeight in 325, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.548176 = idf(docFreq=172, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.19130702 = weight(abstract_txt:operational in 325) [ClassicSimilarity], result of:
            0.19130702 = score(doc=325,freq=3.0), product of:
              0.17763053 = queryWeight, product of:
                1.6085485 = boost
                6.6325636 = idf(docFreq=158, maxDocs=44421)
                0.016649531 = queryNorm
              1.0769941 = fieldWeight in 325, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.6325636 = idf(docFreq=158, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.079528496 = weight(abstract_txt:retrieval in 325) [ClassicSimilarity], result of:
            0.079528496 = score(doc=325,freq=4.0), product of:
              0.122005455 = queryWeight, product of:
                2.1078272 = boost
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.016649531 = queryNorm
              0.6518438 = fieldWeight in 325, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.10052512 = weight(abstract_txt:queries in 325) [ClassicSimilarity], result of:
            0.10052512 = score(doc=325,freq=1.0), product of:
              0.21018277 = queryWeight, product of:
                2.4745083 = boost
                5.1015973 = idf(docFreq=734, maxDocs=44421)
                0.016649531 = queryNorm
              0.47827476 = fieldWeight in 325, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1015973 = idf(docFreq=734, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.1814779 = weight(abstract_txt:standards in 325) [ClassicSimilarity], result of:
            0.1814779 = score(doc=325,freq=3.0), product of:
              0.23275168 = queryWeight, product of:
                2.9113324 = boost
                4.8017445 = idf(docFreq=991, maxDocs=44421)
                0.016649531 = queryNorm
              0.7797061 = fieldWeight in 325, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.8017445 = idf(docFreq=991, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.15835081 = weight(abstract_txt:performance in 325) [ClassicSimilarity], result of:
            0.15835081 = score(doc=325,freq=2.0), product of:
              0.25853223 = queryWeight, product of:
                3.3611922 = boost
                4.619759 = idf(docFreq=1189, maxDocs=44421)
                0.016649531 = queryNorm
              0.6124993 = fieldWeight in 325, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.619759 = idf(docFreq=1189, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.16647954 = weight(abstract_txt:collections in 325) [ClassicSimilarity], result of:
            0.16647954 = score(doc=325,freq=2.0), product of:
              0.26730582 = queryWeight, product of:
                3.4177492 = boost
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.016649531 = queryNorm
              0.62280554 = fieldWeight in 325, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
          0.20635311 = weight(abstract_txt:test in 325) [ClassicSimilarity], result of:
            0.20635311 = score(doc=325,freq=2.0), product of:
              0.3084432 = queryWeight, product of:
                3.6713314 = boost
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.016649531 = queryNorm
              0.669015 = fieldWeight in 325, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.09375 = fieldNorm(doc=325)
        0.4 = coord(10/25)
    
  2. TREC: experiment and evaluation in information retrieval (2005) 0.36
    0.3599822 = sum of:
      0.3599822 = product of:
        0.9999505 = sum of:
          0.017691983 = weight(abstract_txt:documents in 761) [ClassicSimilarity], result of:
            0.017691983 = score(doc=761,freq=1.0), product of:
              0.06865138 = queryWeight, product of:
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.016649531 = queryNorm
              0.25770763 = fieldWeight in 761, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.023492984 = weight(abstract_txt:techniques in 761) [ClassicSimilarity], result of:
            0.023492984 = score(doc=761,freq=1.0), product of:
              0.08293866 = queryWeight, product of:
                1.0991422 = boost
                4.5321174 = idf(docFreq=1298, maxDocs=44421)
                0.016649531 = queryNorm
              0.28325734 = fieldWeight in 761, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.5321174 = idf(docFreq=1298, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.06549102 = weight(abstract_txt:retrieving in 761) [ClassicSimilarity], result of:
            0.06549102 = score(doc=761,freq=1.0), product of:
              0.16428052 = queryWeight, product of:
                1.546922 = boost
                6.3784575 = idf(docFreq=204, maxDocs=44421)
                0.016649531 = queryNorm
              0.3986536 = fieldWeight in 761, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.3784575 = idf(docFreq=204, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.09183159 = weight(abstract_txt:retrieval in 761) [ClassicSimilarity], result of:
            0.09183159 = score(doc=761,freq=12.0), product of:
              0.122005455 = queryWeight, product of:
                2.1078272 = boost
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.016649531 = queryNorm
              0.7526843 = fieldWeight in 761, product of:
                3.4641016 = tf(freq=12.0), with freq of:
                  12.0 = termFreq=12.0
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.32264328 = weight(abstract_txt:trec in 761) [ClassicSimilarity], result of:
            0.32264328 = score(doc=761,freq=8.0), product of:
              0.27223697 = queryWeight, product of:
                2.438903 = boost
                6.704255 = idf(docFreq=147, maxDocs=44421)
                0.016649531 = queryNorm
              1.185156 = fieldWeight in 761, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                6.704255 = idf(docFreq=147, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.069850884 = weight(abstract_txt:standards in 761) [ClassicSimilarity], result of:
            0.069850884 = score(doc=761,freq=1.0), product of:
              0.23275168 = queryWeight, product of:
                2.9113324 = boost
                4.8017445 = idf(docFreq=991, maxDocs=44421)
                0.016649531 = queryNorm
              0.30010903 = fieldWeight in 761, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8017445 = idf(docFreq=991, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.08350382 = weight(abstract_txt:effectiveness in 761) [ClassicSimilarity], result of:
            0.08350382 = score(doc=761,freq=1.0), product of:
              0.2621697 = queryWeight, product of:
                3.0898445 = boost
                5.0961695 = idf(docFreq=738, maxDocs=44421)
                0.016649531 = queryNorm
              0.3185106 = fieldWeight in 761, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0961695 = idf(docFreq=738, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.15695842 = weight(abstract_txt:collections in 761) [ClassicSimilarity], result of:
            0.15695842 = score(doc=761,freq=4.0), product of:
              0.26730582 = queryWeight, product of:
                3.4177492 = boost
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.016649531 = queryNorm
              0.5871867 = fieldWeight in 761, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
          0.16848661 = weight(abstract_txt:test in 761) [ClassicSimilarity], result of:
            0.16848661 = score(doc=761,freq=3.0), product of:
              0.3084432 = queryWeight, product of:
                3.6713314 = boost
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.016649531 = queryNorm
              0.54624844 = fieldWeight in 761, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.0625 = fieldNorm(doc=761)
        0.36 = coord(9/25)
    
  3. Kantor, P.; Kim, M.H.; Ibraev, U.; Atasoy, K.: Estimating the number of relevant documents in enormous collections (1999) 0.32
    0.3241995 = sum of:
      0.3241995 = product of:
        0.9005541 = sum of:
          0.039560482 = weight(abstract_txt:documents in 690) [ClassicSimilarity], result of:
            0.039560482 = score(doc=690,freq=5.0), product of:
              0.06865138 = queryWeight, product of:
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.016649531 = queryNorm
              0.5762518 = fieldWeight in 690, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.07513967 = weight(abstract_txt:relevant in 690) [ClassicSimilarity], result of:
            0.07513967 = score(doc=690,freq=9.0), product of:
              0.08655595 = queryWeight, product of:
                1.1228554 = boost
                4.6298943 = idf(docFreq=1177, maxDocs=44421)
                0.016649531 = queryNorm
              0.8681052 = fieldWeight in 690, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                4.6298943 = idf(docFreq=1177, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.09261829 = weight(abstract_txt:retrieving in 690) [ClassicSimilarity], result of:
            0.09261829 = score(doc=690,freq=2.0), product of:
              0.16428052 = queryWeight, product of:
                1.546922 = boost
                6.3784575 = idf(docFreq=204, maxDocs=44421)
                0.016649531 = queryNorm
              0.5637813 = fieldWeight in 690, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.3784575 = idf(docFreq=204, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.037490092 = weight(abstract_txt:retrieval in 690) [ClassicSimilarity], result of:
            0.037490092 = score(doc=690,freq=2.0), product of:
              0.122005455 = queryWeight, product of:
                2.1078272 = boost
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.016649531 = queryNorm
              0.3072821 = fieldWeight in 690, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.19757786 = weight(abstract_txt:trec in 690) [ClassicSimilarity], result of:
            0.19757786 = score(doc=690,freq=3.0), product of:
              0.27223697 = queryWeight, product of:
                2.438903 = boost
                6.704255 = idf(docFreq=147, maxDocs=44421)
                0.016649531 = queryNorm
              0.7257569 = fieldWeight in 690, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.704255 = idf(docFreq=147, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.074647285 = weight(abstract_txt:performance in 690) [ClassicSimilarity], result of:
            0.074647285 = score(doc=690,freq=1.0), product of:
              0.25853223 = queryWeight, product of:
                3.3611922 = boost
                4.619759 = idf(docFreq=1189, maxDocs=44421)
                0.016649531 = queryNorm
              0.28873494 = fieldWeight in 690, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.619759 = idf(docFreq=1189, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.17525825 = weight(abstract_txt:chance in 690) [ClassicSimilarity], result of:
            0.17525825 = score(doc=690,freq=1.0), product of:
              0.39895803 = queryWeight, product of:
                3.4092116 = boost
                7.028639 = idf(docFreq=106, maxDocs=44421)
                0.016649531 = queryNorm
              0.43928993 = fieldWeight in 690, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.028639 = idf(docFreq=106, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.11098637 = weight(abstract_txt:collections in 690) [ClassicSimilarity], result of:
            0.11098637 = score(doc=690,freq=2.0), product of:
              0.26730582 = queryWeight, product of:
                3.4177492 = boost
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.016649531 = queryNorm
              0.4152037 = fieldWeight in 690, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
          0.09727579 = weight(abstract_txt:test in 690) [ClassicSimilarity], result of:
            0.09727579 = score(doc=690,freq=1.0), product of:
              0.3084432 = queryWeight, product of:
                3.6713314 = boost
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.016649531 = queryNorm
              0.3153767 = fieldWeight in 690, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.0625 = fieldNorm(doc=690)
        0.36 = coord(9/25)
    
  4. Fujita, S.: Technology survey and invalidity search : a comparative study of different tasks for Japanese patent document retrieval (2007) 0.24
    0.24460343 = sum of:
      0.24460343 = product of:
        0.7643857 = sum of:
          0.037530366 = weight(abstract_txt:documents in 1918) [ClassicSimilarity], result of:
            0.037530366 = score(doc=1918,freq=2.0), product of:
              0.06865138 = queryWeight, product of:
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.016649531 = queryNorm
              0.54668045 = fieldWeight in 1918, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.05289909 = weight(abstract_txt:models in 1918) [ClassicSimilarity], result of:
            0.05289909 = score(doc=1918,freq=2.0), product of:
              0.08630306 = queryWeight, product of:
                1.1212139 = boost
                4.623126 = idf(docFreq=1185, maxDocs=44421)
                0.016649531 = queryNorm
              0.6129457 = fieldWeight in 1918, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.623126 = idf(docFreq=1185, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.053131767 = weight(abstract_txt:relevant in 1918) [ClassicSimilarity], result of:
            0.053131767 = score(doc=1918,freq=2.0), product of:
              0.08655595 = queryWeight, product of:
                1.1228554 = boost
                4.6298943 = idf(docFreq=1177, maxDocs=44421)
                0.016649531 = queryNorm
              0.613843 = fieldWeight in 1918, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6298943 = idf(docFreq=1177, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.079528496 = weight(abstract_txt:retrieval in 1918) [ClassicSimilarity], result of:
            0.079528496 = score(doc=1918,freq=4.0), product of:
              0.122005455 = queryWeight, product of:
                2.1078272 = boost
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.016649531 = queryNorm
              0.6518438 = fieldWeight in 1918, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.10052512 = weight(abstract_txt:queries in 1918) [ClassicSimilarity], result of:
            0.10052512 = score(doc=1918,freq=1.0), product of:
              0.21018277 = queryWeight, product of:
                2.4745083 = boost
                5.1015973 = idf(docFreq=734, maxDocs=44421)
                0.016649531 = queryNorm
              0.47827476 = fieldWeight in 1918, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1015973 = idf(docFreq=734, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.17713836 = weight(abstract_txt:effectiveness in 1918) [ClassicSimilarity], result of:
            0.17713836 = score(doc=1918,freq=2.0), product of:
              0.2621697 = queryWeight, product of:
                3.0898445 = boost
                5.0961695 = idf(docFreq=738, maxDocs=44421)
                0.016649531 = queryNorm
              0.675663 = fieldWeight in 1918, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.0961695 = idf(docFreq=738, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.117718816 = weight(abstract_txt:collections in 1918) [ClassicSimilarity], result of:
            0.117718816 = score(doc=1918,freq=1.0), product of:
              0.26730582 = queryWeight, product of:
                3.4177492 = boost
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.016649531 = queryNorm
              0.44039002 = fieldWeight in 1918, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
          0.14591369 = weight(abstract_txt:test in 1918) [ClassicSimilarity], result of:
            0.14591369 = score(doc=1918,freq=1.0), product of:
              0.3084432 = queryWeight, product of:
                3.6713314 = boost
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.016649531 = queryNorm
              0.47306505 = fieldWeight in 1918, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.09375 = fieldNorm(doc=1918)
        0.32 = coord(8/25)
    
  5. Voorhees, E.M.; Harman, D.K.: ¬The Text REtrieval Conference (2005) 0.24
    0.23680227 = sum of:
      0.23680227 = product of:
        0.65778404 = sum of:
          0.01105749 = weight(abstract_txt:documents in 82) [ClassicSimilarity], result of:
            0.01105749 = score(doc=82,freq=1.0), product of:
              0.06865138 = queryWeight, product of:
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.016649531 = queryNorm
              0.16106726 = fieldWeight in 82, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.123322 = idf(docFreq=1954, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.015654098 = weight(abstract_txt:relevant in 82) [ClassicSimilarity], result of:
            0.015654098 = score(doc=82,freq=1.0), product of:
              0.08655595 = queryWeight, product of:
                1.1228554 = boost
                4.6298943 = idf(docFreq=1177, maxDocs=44421)
                0.016649531 = queryNorm
              0.18085524 = fieldWeight in 82, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6298943 = idf(docFreq=1177, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.023929574 = weight(abstract_txt:traditional in 82) [ClassicSimilarity], result of:
            0.023929574 = score(doc=82,freq=1.0), product of:
              0.13148169 = queryWeight, product of:
                1.6949373 = boost
                4.6591816 = idf(docFreq=1143, maxDocs=44421)
                0.016649531 = queryNorm
              0.18199928 = fieldWeight in 82, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6591816 = idf(docFreq=1143, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.061993413 = weight(abstract_txt:retrieval in 82) [ClassicSimilarity], result of:
            0.061993413 = score(doc=82,freq=14.0), product of:
              0.122005455 = queryWeight, product of:
                2.1078272 = boost
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.016649531 = queryNorm
              0.50812 = fieldWeight in 82, product of:
                3.7416575 = tf(freq=14.0), with freq of:
                  14.0 = termFreq=14.0
                3.4765 = idf(docFreq=3732, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.29395586 = weight(abstract_txt:trec in 82) [ClassicSimilarity], result of:
            0.29395586 = score(doc=82,freq=17.0), product of:
              0.27223697 = queryWeight, product of:
                2.438903 = boost
                6.704255 = idf(docFreq=147, maxDocs=44421)
                0.016649531 = queryNorm
              1.0797794 = fieldWeight in 82, product of:
                4.1231055 = tf(freq=17.0), with freq of:
                  17.0 = termFreq=17.0
                6.704255 = idf(docFreq=147, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.0436568 = weight(abstract_txt:standards in 82) [ClassicSimilarity], result of:
            0.0436568 = score(doc=82,freq=1.0), product of:
              0.23275168 = queryWeight, product of:
                2.9113324 = boost
                4.8017445 = idf(docFreq=991, maxDocs=44421)
                0.016649531 = queryNorm
              0.18756814 = fieldWeight in 82, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8017445 = idf(docFreq=991, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.052189887 = weight(abstract_txt:effectiveness in 82) [ClassicSimilarity], result of:
            0.052189887 = score(doc=82,freq=1.0), product of:
              0.2621697 = queryWeight, product of:
                3.0898445 = boost
                5.0961695 = idf(docFreq=738, maxDocs=44421)
                0.016649531 = queryNorm
              0.19906911 = fieldWeight in 82, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0961695 = idf(docFreq=738, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.06936647 = weight(abstract_txt:collections in 82) [ClassicSimilarity], result of:
            0.06936647 = score(doc=82,freq=2.0), product of:
              0.26730582 = queryWeight, product of:
                3.4177492 = boost
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.016649531 = queryNorm
              0.2595023 = fieldWeight in 82, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6974936 = idf(docFreq=1100, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
          0.08598047 = weight(abstract_txt:test in 82) [ClassicSimilarity], result of:
            0.08598047 = score(doc=82,freq=2.0), product of:
              0.3084432 = queryWeight, product of:
                3.6713314 = boost
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.016649531 = queryNorm
              0.27875626 = fieldWeight in 82, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.046027 = idf(docFreq=776, maxDocs=44421)
                0.0390625 = fieldNorm(doc=82)
        0.36 = coord(9/25)