Document (#23909)

Author

Keenan, S.
Smeaton, A.F.
Keogh, G.

Title

¬The effect of pool depth on system evaluation in TREC

Source

Journal of the American Society for Information Science and technology. 52(2001) no.7, S.570-574

Year

2001

Abstract

The TREC benchmarking exercise for information retrieval (IR) experiments has provided a forum and an opportunity for IR researchers to evaluate the performance of their approaches to the IR task and has resulted in improvements in IR effectiveness. Typically, retrieval performance has been measured in terms of precision and recall, and comparisons between different IR approaches have been based on these measures. These measures are in turn dependent on the so-called "pool depth" used to discover relevant documents. Whereas there is evidence to suggest that the pool depth size used for TREC evaluations adequately identifies the relevant documents in the entire test data collection, we consider how it affects the evaluations of individual systems. The data used comes from the Sixth TREC conference, TREC-6. By fitting appropriate regression models we explore whether different pool depths confer advantages or disadvantages on different retrieval systems when they are compared. As a consequence of this model fitting, a pair of measures for each retrieval run, which are related to precision and recall, emerge. For each system, these give an extrapolation for the number of relevant documents the system would have been deemed to have retrieved if an indefinitely large pool size had been used, and also a measure of the sensitivity of each system to pool size. We concur that even on the basis of analyses of individual systems, the pool depth of 100 used by TREC is adequate

Theme

Retrievalstudien

Object

TREC

Similar documents (author)

1. Keenan, S.: Concise dictionary of library and information science (1996) 2.30

   2.2958126 = sum of:
     2.2958126 = product of:
       4.591625 = sum of:
         4.591625 = weight(author_txt:keenan in 3436) [ClassicSimilarity], result of:
           4.591625 = score(doc=3436,freq=1.0), product of:
             0.7729811 = queryWeight, product of:
               1.1038065 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.073681496 = queryNorm
             5.9401517 = fieldWeight in 3436, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.625 = fieldNorm(doc=3436)
       0.5 = coord(1/2)
   

2. Keenan, S.: From world bibliography to the Internet in 100 years : FID celebrates its centennial (1995) 2.30

   2.2958126 = sum of:
     2.2958126 = product of:
       4.591625 = sum of:
         4.591625 = weight(author_txt:keenan in 1216) [ClassicSimilarity], result of:
           4.591625 = score(doc=1216,freq=1.0), product of:
             0.7729811 = queryWeight, product of:
               1.1038065 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.073681496 = queryNorm
             5.9401517 = fieldWeight in 1216, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.625 = fieldNorm(doc=1216)
       0.5 = coord(1/2)
   

3. Keenan, S.: Abstracting and indexing services in science and technology (1969) 2.30

   2.2958126 = sum of:
     2.2958126 = product of:
       4.591625 = sum of:
         4.591625 = weight(author_txt:keenan in 235) [ClassicSimilarity], result of:
           4.591625 = score(doc=235,freq=1.0), product of:
             0.7729811 = queryWeight, product of:
               1.1038065 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.073681496 = queryNorm
             5.9401517 = fieldWeight in 235, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.625 = fieldNorm(doc=235)
       0.5 = coord(1/2)
   

4. Keenan, J.P.: ¬Das Gesicht im Spiegel : Auf der Suche nach dem Ursprung des Bewusstseins (2005) 2.30

   2.2958126 = sum of:
     2.2958126 = product of:
       4.591625 = sum of:
         4.591625 = weight(author_txt:keenan in 5844) [ClassicSimilarity], result of:
           4.591625 = score(doc=5844,freq=1.0), product of:
             0.7729811 = queryWeight, product of:
               1.1038065 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.073681496 = queryNorm
             5.9401517 = fieldWeight in 5844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.625 = fieldNorm(doc=5844)
       0.5 = coord(1/2)
   

5. Keenan, S.; Johnston, C.: Concise dictionary of library and information science (2000) 1.84

   1.83665 = sum of:
     1.83665 = product of:
       3.6733 = sum of:
         3.6733 = weight(author_txt:keenan in 2354) [ClassicSimilarity], result of:
           3.6733 = score(doc=2354,freq=1.0), product of:
             0.7729811 = queryWeight, product of:
               1.1038065 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.073681496 = queryNorm
             4.7521214 = fieldWeight in 2354, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.5 = fieldNorm(doc=2354)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Pal, S.; Mitra, M.; Kamps, J.: Evaluation effort, reliability and reusability in XML retrieval (2011) 0.44

   0.4393022 = sum of:
     0.4393022 = product of:
       0.9152129 = sum of:
         0.008429996 = weight(abstract_txt:these in 197) [ClassicSimilarity], result of:
           0.008429996 = score(doc=197,freq=1.0), product of:
             0.048430577 = queryWeight, product of:
               1.0903957 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.013954551 = queryNorm
             0.1740635 = fieldWeight in 197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.020754412 = weight(abstract_txt:systems in 197) [ClassicSimilarity], result of:
           0.020754412 = score(doc=197,freq=4.0), product of:
             0.055627346 = queryWeight, product of:
               1.168607 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.013954551 = queryNorm
             0.37309727 = fieldWeight in 197, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.05087289 = weight(abstract_txt:precision in 197) [ClassicSimilarity], result of:
           0.05087289 = score(doc=197,freq=3.0), product of:
             0.09723475 = queryWeight, product of:
               1.2615063 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.013954551 = queryNorm
             0.5231966 = fieldWeight in 197, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.046879455 = weight(abstract_txt:recall in 197) [ClassicSimilarity], result of:
           0.046879455 = score(doc=197,freq=2.0), product of:
             0.10540213 = queryWeight, product of:
               1.3134192 = boost
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.013954551 = queryNorm
             0.44476765 = fieldWeight in 197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.013374246 = weight(abstract_txt:system in 197) [ClassicSimilarity], result of:
           0.013374246 = score(doc=197,freq=1.0), product of:
             0.072509326 = queryWeight, product of:
               1.5406026 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.013954551 = queryNorm
             0.18444863 = fieldWeight in 197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.029293 = weight(abstract_txt:retrieval in 197) [ClassicSimilarity], result of:
           0.029293 = score(doc=197,freq=4.0), product of:
             0.077037744 = queryWeight, product of:
               1.5879816 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013954551 = queryNorm
             0.3802422 = fieldWeight in 197, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.016460044 = weight(abstract_txt:been in 197) [ClassicSimilarity], result of:
           0.016460044 = score(doc=197,freq=1.0), product of:
             0.08327251 = queryWeight, product of:
               1.6509902 = boost
               3.614442 = idf(docFreq=3251, maxDocs=44421)
               0.013954551 = queryNorm
             0.1976648 = fieldWeight in 197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.614442 = idf(docFreq=3251, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.059025604 = weight(abstract_txt:measures in 197) [ClassicSimilarity], result of:
           0.059025604 = score(doc=197,freq=2.0), product of:
             0.14068627 = queryWeight, product of:
               1.858447 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.013954551 = queryNorm
             0.41955483 = fieldWeight in 197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.023316525 = weight(abstract_txt:used in 197) [ClassicSimilarity], result of:
           0.023316525 = score(doc=197,freq=2.0), product of:
             0.08980149 = queryWeight, product of:
               1.9168603 = boost
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.013954551 = queryNorm
             0.2596452 = fieldWeight in 197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.05351181 = weight(abstract_txt:size in 197) [ClassicSimilarity], result of:
           0.05351181 = score(doc=197,freq=1.0), product of:
             0.16603558 = queryWeight, product of:
               2.018947 = boost
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.013954551 = queryNorm
             0.32229123 = fieldWeight in 197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.15756148 = weight(abstract_txt:trec in 197) [ClassicSimilarity], result of:
           0.15756148 = score(doc=197,freq=1.0), product of:
             0.42974558 = queryWeight, product of:
               4.593514 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.013954551 = queryNorm
             0.36663896 = fieldWeight in 197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
         0.4357335 = weight(abstract_txt:pool in 197) [ClassicSimilarity], result of:
           0.4357335 = score(doc=197,freq=2.0), product of:
             0.7074531 = queryWeight, product of:
               6.365925 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013954551 = queryNorm
             0.6159186 = fieldWeight in 197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.0546875 = fieldNorm(doc=197)
       0.48 = coord(12/25)
   

2. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.34

   0.34140256 = sum of:
     0.34140256 = product of:
       0.8535064 = sum of:
         0.013624931 = weight(abstract_txt:these in 3591) [ClassicSimilarity], result of:
           0.013624931 = score(doc=3591,freq=2.0), product of:
             0.048430577 = queryWeight, product of:
               1.0903957 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.013954551 = queryNorm
             0.2813291 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.011859664 = weight(abstract_txt:systems in 3591) [ClassicSimilarity], result of:
           0.011859664 = score(doc=3591,freq=1.0), product of:
             0.055627346 = queryWeight, product of:
               1.168607 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.013954551 = queryNorm
             0.21319844 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.0335674 = weight(abstract_txt:precision in 3591) [ClassicSimilarity], result of:
           0.0335674 = score(doc=3591,freq=1.0), product of:
             0.09723475 = queryWeight, product of:
               1.2615063 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.013954551 = queryNorm
             0.3452202 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.020860665 = weight(abstract_txt:each in 3591) [ClassicSimilarity], result of:
           0.020860665 = score(doc=3591,freq=1.0), product of:
             0.08105731 = queryWeight, product of:
               1.4106536 = boost
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.013954551 = queryNorm
             0.25735697 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.036279622 = weight(abstract_txt:documents in 3591) [ClassicSimilarity], result of:
           0.036279622 = score(doc=3591,freq=3.0), product of:
             0.08127835 = queryWeight, product of:
               1.4125758 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.013954551 = queryNorm
             0.4463627 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.021616045 = weight(abstract_txt:system in 3591) [ClassicSimilarity], result of:
           0.021616045 = score(doc=3591,freq=2.0), product of:
             0.072509326 = queryWeight, product of:
               1.5406026 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.013954551 = queryNorm
             0.298114 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.02367232 = weight(abstract_txt:retrieval in 3591) [ClassicSimilarity], result of:
           0.02367232 = score(doc=3591,freq=2.0), product of:
             0.077037744 = queryWeight, product of:
               1.5879816 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013954551 = queryNorm
             0.3072821 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.08524216 = weight(abstract_txt:depth in 3591) [ClassicSimilarity], result of:
           0.08524216 = score(doc=3591,freq=1.0), product of:
             0.2280286 = queryWeight, product of:
               2.732048 = boost
               5.981156 = idf(docFreq=304, maxDocs=44421)
               0.013954551 = queryNorm
             0.37382224 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.981156 = idf(docFreq=304, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.25465778 = weight(abstract_txt:trec in 3591) [ClassicSimilarity], result of:
           0.25465778 = score(doc=3591,freq=2.0), product of:
             0.42974558 = queryWeight, product of:
               4.593514 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.013954551 = queryNorm
             0.592578 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.35212582 = weight(abstract_txt:pool in 3591) [ClassicSimilarity], result of:
           0.35212582 = score(doc=3591,freq=1.0), product of:
             0.7074531 = queryWeight, product of:
               6.365925 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013954551 = queryNorm
             0.49773738 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
       0.4 = coord(10/25)
   

3. Losada, D.E.; Parapar, J.; Barreiro, A.: When to stop making relevance judgments? : a study of stopping methods for building information retrieval test collections (2019) 0.33

   0.333907 = sum of:
     0.333907 = product of:
       0.75887954 = sum of:
         0.013624931 = weight(abstract_txt:these in 674) [ClassicSimilarity], result of:
           0.013624931 = score(doc=674,freq=2.0), product of:
             0.048430577 = queryWeight, product of:
               1.0903957 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.013954551 = queryNorm
             0.2813291 = fieldWeight in 674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.009785006 = weight(abstract_txt:have in 674) [ClassicSimilarity], result of:
           0.009785006 = score(doc=674,freq=1.0), product of:
             0.04893439 = queryWeight, product of:
               1.0960525 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.013954551 = queryNorm
             0.19996175 = fieldWeight in 674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.011859664 = weight(abstract_txt:systems in 674) [ClassicSimilarity], result of:
           0.011859664 = score(doc=674,freq=1.0), product of:
             0.055627346 = queryWeight, product of:
               1.168607 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.013954551 = queryNorm
             0.21319844 = fieldWeight in 674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.02071207 = weight(abstract_txt:different in 674) [ClassicSimilarity], result of:
           0.02071207 = score(doc=674,freq=2.0), product of:
             0.06402935 = queryWeight, product of:
               1.2537584 = boost
               3.6597328 = idf(docFreq=3107, maxDocs=44421)
               0.013954551 = queryNorm
             0.32347775 = fieldWeight in 674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6597328 = idf(docFreq=3107, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.04747147 = weight(abstract_txt:precision in 674) [ClassicSimilarity], result of:
           0.04747147 = score(doc=674,freq=2.0), product of:
             0.09723475 = queryWeight, product of:
               1.2615063 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.013954551 = queryNorm
             0.4882151 = fieldWeight in 674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.037884325 = weight(abstract_txt:recall in 674) [ClassicSimilarity], result of:
           0.037884325 = score(doc=674,freq=1.0), product of:
             0.10540213 = queryWeight, product of:
               1.3134192 = boost
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.013954551 = queryNorm
             0.35942656 = fieldWeight in 674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.036279622 = weight(abstract_txt:documents in 674) [ClassicSimilarity], result of:
           0.036279622 = score(doc=674,freq=3.0), product of:
             0.08127835 = queryWeight, product of:
               1.4125758 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.013954551 = queryNorm
             0.4463627 = fieldWeight in 674, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.016738858 = weight(abstract_txt:retrieval in 674) [ClassicSimilarity], result of:
           0.016738858 = score(doc=674,freq=1.0), product of:
             0.077037744 = queryWeight, product of:
               1.5879816 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013954551 = queryNorm
             0.21728125 = fieldWeight in 674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.047699895 = weight(abstract_txt:measures in 674) [ClassicSimilarity], result of:
           0.047699895 = score(doc=674,freq=1.0), product of:
             0.14068627 = queryWeight, product of:
               1.858447 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.013954551 = queryNorm
             0.3390515 = fieldWeight in 674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.018842598 = weight(abstract_txt:used in 674) [ClassicSimilarity], result of:
           0.018842598 = score(doc=674,freq=1.0), product of:
             0.08980149 = queryWeight, product of:
               1.9168603 = boost
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.013954551 = queryNorm
             0.20982501 = fieldWeight in 674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
         0.49798113 = weight(abstract_txt:pool in 674) [ClassicSimilarity], result of:
           0.49798113 = score(doc=674,freq=2.0), product of:
             0.7074531 = queryWeight, product of:
               6.365925 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013954551 = queryNorm
             0.70390695 = fieldWeight in 674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.0625 = fieldNorm(doc=674)
       0.44 = coord(11/25)
   

4. Voorhees, E.M.; Harman, D.K.: ¬The Text REtrieval Conference (2005) 0.32

   0.32164112 = sum of:
     0.32164112 = product of:
       0.67008567 = sum of:
         0.0060214256 = weight(abstract_txt:these in 82) [ClassicSimilarity], result of:
           0.0060214256 = score(doc=82,freq=1.0), product of:
             0.048430577 = queryWeight, product of:
               1.0903957 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.013954551 = queryNorm
             0.12433107 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.016180431 = weight(abstract_txt:have in 82) [ClassicSimilarity], result of:
           0.016180431 = score(doc=82,freq=7.0), product of:
             0.04893439 = queryWeight, product of:
               1.0960525 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.013954551 = queryNorm
             0.33065563 = fieldWeight in 82, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.014432522 = weight(abstract_txt:individual in 82) [ClassicSimilarity], result of:
           0.014432522 = score(doc=82,freq=1.0), product of:
             0.07577358 = queryWeight, product of:
               1.1136215 = boost
               4.8760076 = idf(docFreq=920, maxDocs=44421)
               0.013954551 = queryNorm
             0.19046904 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8760076 = idf(docFreq=920, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.010482561 = weight(abstract_txt:systems in 82) [ClassicSimilarity], result of:
           0.010482561 = score(doc=82,freq=2.0), product of:
             0.055627346 = queryWeight, product of:
               1.168607 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.013954551 = queryNorm
             0.18844259 = fieldWeight in 82, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.009153528 = weight(abstract_txt:different in 82) [ClassicSimilarity], result of:
           0.009153528 = score(doc=82,freq=1.0), product of:
             0.06402935 = queryWeight, product of:
               1.2537584 = boost
               3.6597328 = idf(docFreq=3107, maxDocs=44421)
               0.013954551 = queryNorm
             0.14295831 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6597328 = idf(docFreq=3107, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.013037915 = weight(abstract_txt:each in 82) [ClassicSimilarity], result of:
           0.013037915 = score(doc=82,freq=1.0), product of:
             0.08105731 = queryWeight, product of:
               1.4106536 = boost
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.013954551 = queryNorm
             0.16084811 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.013091281 = weight(abstract_txt:documents in 82) [ClassicSimilarity], result of:
           0.013091281 = score(doc=82,freq=1.0), product of:
             0.08127835 = queryWeight, product of:
               1.4125758 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.013954551 = queryNorm
             0.16106726 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.037178777 = weight(abstract_txt:evaluations in 82) [ClassicSimilarity], result of:
           0.037178777 = score(doc=82,freq=1.0), product of:
             0.14239226 = queryWeight, product of:
               1.5265881 = boost
               6.684188 = idf(docFreq=150, maxDocs=44421)
               0.013954551 = queryNorm
             0.2611011 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.684188 = idf(docFreq=150, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.018533336 = weight(abstract_txt:relevant in 82) [ClassicSimilarity], result of:
           0.018533336 = score(doc=82,freq=1.0), product of:
             0.102476075 = queryWeight, product of:
               1.5861182 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.013954551 = queryNorm
             0.18085524 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.03914442 = weight(abstract_txt:retrieval in 82) [ClassicSimilarity], result of:
           0.03914442 = score(doc=82,freq=14.0), product of:
             0.077037744 = queryWeight, product of:
               1.5879816 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013954551 = queryNorm
             0.50812 = fieldWeight in 82, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.02879908 = weight(abstract_txt:been in 82) [ClassicSimilarity], result of:
           0.02879908 = score(doc=82,freq=6.0), product of:
             0.08327251 = queryWeight, product of:
               1.6509902 = boost
               3.614442 = idf(docFreq=3251, maxDocs=44421)
               0.013954551 = queryNorm
             0.34584138 = fieldWeight in 82, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.614442 = idf(docFreq=3251, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.4640304 = weight(abstract_txt:trec in 82) [ClassicSimilarity], result of:
           0.4640304 = score(doc=82,freq=17.0), product of:
             0.42974558 = queryWeight, product of:
               4.593514 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.013954551 = queryNorm
             1.0797794 = fieldWeight in 82, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
       0.48 = coord(12/25)
   

5. Kantor, P.; Kim, M.H.; Ibraev, U.; Atasoy, K.: Estimating the number of relevant documents in enormous collections (1999) 0.28

   0.2842247 = sum of:
     0.2842247 = product of:
       0.6459652 = sum of:
         0.01963935 = weight(abstract_txt:performance in 690) [ClassicSimilarity], result of:
           0.01963935 = score(doc=690,freq=1.0), product of:
             0.068018615 = queryWeight, product of:
               1.0550975 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.013954551 = queryNorm
             0.28873494 = fieldWeight in 690, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.023092035 = weight(abstract_txt:individual in 690) [ClassicSimilarity], result of:
           0.023092035 = score(doc=690,freq=1.0), product of:
             0.07577358 = queryWeight, product of:
               1.1136215 = boost
               4.8760076 = idf(docFreq=920, maxDocs=44421)
               0.013954551 = queryNorm
             0.30475047 = fieldWeight in 690, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8760076 = idf(docFreq=920, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.02054154 = weight(abstract_txt:systems in 690) [ClassicSimilarity], result of:
           0.02054154 = score(doc=690,freq=3.0), product of:
             0.055627346 = queryWeight, product of:
               1.168607 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.013954551 = queryNorm
             0.36927053 = fieldWeight in 690, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.0335674 = weight(abstract_txt:precision in 690) [ClassicSimilarity], result of:
           0.0335674 = score(doc=690,freq=1.0), product of:
             0.09723475 = queryWeight, product of:
               1.2615063 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.013954551 = queryNorm
             0.3452202 = fieldWeight in 690, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.029501433 = weight(abstract_txt:each in 690) [ClassicSimilarity], result of:
           0.029501433 = score(doc=690,freq=2.0), product of:
             0.08105731 = queryWeight, product of:
               1.4106536 = boost
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.013954551 = queryNorm
             0.3639577 = fieldWeight in 690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.046836793 = weight(abstract_txt:documents in 690) [ClassicSimilarity], result of:
           0.046836793 = score(doc=690,freq=5.0), product of:
             0.08127835 = queryWeight, product of:
               1.4125758 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.013954551 = queryNorm
             0.5762518 = fieldWeight in 690, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.021616045 = weight(abstract_txt:system in 690) [ClassicSimilarity], result of:
           0.021616045 = score(doc=690,freq=2.0), product of:
             0.072509326 = queryWeight, product of:
               1.5406026 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.013954551 = queryNorm
             0.298114 = fieldWeight in 690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.088960014 = weight(abstract_txt:relevant in 690) [ClassicSimilarity], result of:
           0.088960014 = score(doc=690,freq=9.0), product of:
             0.102476075 = queryWeight, product of:
               1.5861182 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.013954551 = queryNorm
             0.8681052 = fieldWeight in 690, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.02367232 = weight(abstract_txt:retrieval in 690) [ClassicSimilarity], result of:
           0.02367232 = score(doc=690,freq=2.0), product of:
             0.077037744 = queryWeight, product of:
               1.5879816 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013954551 = queryNorm
             0.3072821 = fieldWeight in 690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.026647458 = weight(abstract_txt:used in 690) [ClassicSimilarity], result of:
           0.026647458 = score(doc=690,freq=2.0), product of:
             0.08980149 = queryWeight, product of:
               1.9168603 = boost
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.013954551 = queryNorm
             0.29673737 = fieldWeight in 690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3572001 = idf(docFreq=4205, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
         0.3118908 = weight(abstract_txt:trec in 690) [ClassicSimilarity], result of:
           0.3118908 = score(doc=690,freq=3.0), product of:
             0.42974558 = queryWeight, product of:
               4.593514 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.013954551 = queryNorm
             0.7257569 = fieldWeight in 690, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=690)
       0.44 = coord(11/25)