Document (#32911)

Author

Sakai, T.

Title

On the reliability of information retrieval metrics based on graded relevance

Source

Information processing and management. 43(2007) no.2, S.531-548

Year

2007

Abstract

This paper compares 14 information retrieval metrics based on graded relevance, together with 10 traditional metrics based on binary relevance, in terms of stability, sensitivity and resemblance of system rankings. More specifically, we compare these metrics using the Buckley/Voorhees stability method, the Voorhees/Buckley swap method and Kendall's rank correlation, with three data sets comprising test collections and submitted runs from NTCIR. Our experiments show that (Average) Normalised Discounted Cumulative Gain at document cut-off l are the best among the rank-based graded-relevance metrics, provided that l is large. On the other hand, if one requires a recall-based graded-relevance metric that is highly correlated with Average Precision, then Q-measure is the best choice. Moreover, these best graded-relevance metrics are at least as stable and sensitive as Average Precision, and are fairly robust to the choice of gain values.

Footnote

Beitrag in: Special issue on AIRS2005: Information Retrieval Research in Asia

Theme

Retrievalalgorithmen

Similar documents (content)

1. Kekäläinen, J.: Binary and graded relevance in IR evaluations : comparison of the effects on ranking of IR systems (2005) 0.59

   0.59022886 = sum of:
     0.59022886 = product of:
       1.3414292 = sum of:
         0.10397365 = weight(abstract_txt:binary in 1036) [ClassicSimilarity], result of:
           0.10397365 = score(doc=1036,freq=4.0), product of:
             0.09112454 = queryWeight, product of:
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.012478646 = queryNorm
             1.141006 = fieldWeight in 1036, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.05548705 = weight(abstract_txt:fairly in 1036) [ClassicSimilarity], result of:
           0.05548705 = score(doc=1036,freq=1.0), product of:
             0.09517019 = queryWeight, product of:
               1.0219574 = boost
               7.462781 = idf(docFreq=68, maxDocs=44218)
               0.012478646 = queryNorm
             0.58302975 = fieldWeight in 1036, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.462781 = idf(docFreq=68, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.008847223 = weight(abstract_txt:with in 1036) [ClassicSimilarity], result of:
           0.008847223 = score(doc=1036,freq=2.0), product of:
             0.032033782 = queryWeight, product of:
               1.0269448 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.012478646 = queryNorm
             0.27618414 = fieldWeight in 1036, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.14030626 = weight(abstract_txt:discounted in 1036) [ClassicSimilarity], result of:
           0.14030626 = score(doc=1036,freq=2.0), product of:
             0.14019996 = queryWeight, product of:
               1.2403843 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.012478646 = queryNorm
             1.0007583 = fieldWeight in 1036, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.11875964 = weight(abstract_txt:normalised in 1036) [ClassicSimilarity], result of:
           0.11875964 = score(doc=1036,freq=1.0), product of:
             0.15805894 = queryWeight, product of:
               1.3170185 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.012478646 = queryNorm
             0.751363 = fieldWeight in 1036, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.045036063 = weight(abstract_txt:precision in 1036) [ClassicSimilarity], result of:
           0.045036063 = score(doc=1036,freq=1.0), product of:
             0.10433342 = queryWeight, product of:
               1.5132443 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.012478646 = queryNorm
             0.4316552 = fieldWeight in 1036, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.07078047 = weight(abstract_txt:rank in 1036) [ClassicSimilarity], result of:
           0.07078047 = score(doc=1036,freq=1.0), product of:
             0.14103454 = queryWeight, product of:
               1.7593817 = boost
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.012478646 = queryNorm
             0.5018662 = fieldWeight in 1036, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.15205759 = weight(abstract_txt:gain in 1036) [ClassicSimilarity], result of:
           0.15205759 = score(doc=1036,freq=4.0), product of:
             0.14792284 = queryWeight, product of:
               1.8018347 = boost
               6.578893 = idf(docFreq=166, maxDocs=44218)
               0.012478646 = queryNorm
             1.0279521 = fieldWeight in 1036, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.578893 = idf(docFreq=166, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.021626437 = weight(abstract_txt:based in 1036) [ClassicSimilarity], result of:
           0.021626437 = score(doc=1036,freq=1.0), product of:
             0.086833335 = queryWeight, product of:
               2.182783 = boost
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.012478646 = queryNorm
             0.24905685 = fieldWeight in 1036, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.19217636 = weight(abstract_txt:relevance in 1036) [ClassicSimilarity], result of:
           0.19217636 = score(doc=1036,freq=4.0), product of:
             0.24938494 = queryWeight, product of:
               4.0522203 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012478646 = queryNorm
             0.7706013 = fieldWeight in 1036, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
         0.43237853 = weight(abstract_txt:graded in 1036) [ClassicSimilarity], result of:
           0.43237853 = score(doc=1036,freq=1.0), product of:
             0.63964534 = queryWeight, product of:
               5.924299 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012478646 = queryNorm
             0.675966 = fieldWeight in 1036, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.078125 = fieldNorm(doc=1036)
       0.44 = coord(11/25)
   

2. Kekäläinen, J.; Järvelin, K.: Using graded relevance assessments in IR evaluation (2002) 0.26

   0.26350647 = sum of:
     0.26350647 = product of:
       0.8234577 = sum of:
         0.041589458 = weight(abstract_txt:binary in 5225) [ClassicSimilarity], result of:
           0.041589458 = score(doc=5225,freq=1.0), product of:
             0.09112454 = queryWeight, product of:
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.012478646 = queryNorm
             0.4564024 = fieldWeight in 5225, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.007077778 = weight(abstract_txt:with in 5225) [ClassicSimilarity], result of:
           0.007077778 = score(doc=5225,freq=2.0), product of:
             0.032033782 = queryWeight, product of:
               1.0269448 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.012478646 = queryNorm
             0.22094731 = fieldWeight in 5225, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.019477002 = weight(abstract_txt:method in 5225) [ClassicSimilarity], result of:
           0.019477002 = score(doc=5225,freq=1.0), product of:
             0.06923695 = queryWeight, product of:
               1.2327253 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.012478646 = queryNorm
             0.28130937 = fieldWeight in 5225, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.0720577 = weight(abstract_txt:precision in 5225) [ClassicSimilarity], result of:
           0.0720577 = score(doc=5225,freq=4.0), product of:
             0.10433342 = queryWeight, product of:
               1.5132443 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.012478646 = queryNorm
             0.6906483 = fieldWeight in 5225, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.040437277 = weight(abstract_txt:best in 5225) [ClassicSimilarity], result of:
           0.040437277 = score(doc=5225,freq=1.0), product of:
             0.12898569 = queryWeight, product of:
               2.060695 = boost
               5.0160327 = idf(docFreq=796, maxDocs=44218)
               0.012478646 = queryNorm
             0.31350204 = fieldWeight in 5225, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0160327 = idf(docFreq=796, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.0935354 = weight(abstract_txt:average in 5225) [ClassicSimilarity], result of:
           0.0935354 = score(doc=5225,freq=2.0), product of:
             0.17905822 = queryWeight, product of:
               2.4279518 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.012478646 = queryNorm
             0.5223742 = fieldWeight in 5225, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.20338032 = weight(abstract_txt:relevance in 5225) [ClassicSimilarity], result of:
           0.20338032 = score(doc=5225,freq=7.0), product of:
             0.24938494 = queryWeight, product of:
               4.0522203 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012478646 = queryNorm
             0.8155277 = fieldWeight in 5225, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
         0.3459028 = weight(abstract_txt:graded in 5225) [ClassicSimilarity], result of:
           0.3459028 = score(doc=5225,freq=1.0), product of:
             0.63964534 = queryWeight, product of:
               5.924299 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012478646 = queryNorm
             0.5407728 = fieldWeight in 5225, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0625 = fieldNorm(doc=5225)
       0.32 = coord(8/25)
   

3. Talvensaari, T.; Juhola, M.; Laurikkala, J.; Järvelin, K.: Corpus-based cross-language information retrieval in retrieval of highly relevant documents (2007) 0.20

   0.19608578 = sum of:
     0.19608578 = product of:
       0.70030636 = sum of:
         0.007077778 = weight(abstract_txt:with in 139) [ClassicSimilarity], result of:
           0.007077778 = score(doc=139,freq=2.0), product of:
             0.032033782 = queryWeight, product of:
               1.0269448 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.012478646 = queryNorm
             0.22094731 = fieldWeight in 139, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
         0.052025147 = weight(abstract_txt:runs in 139) [ClassicSimilarity], result of:
           0.052025147 = score(doc=139,freq=1.0), product of:
             0.10579263 = queryWeight, product of:
               1.077482 = boost
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.012478646 = queryNorm
             0.49176535 = fieldWeight in 139, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
         0.05095249 = weight(abstract_txt:precision in 139) [ClassicSimilarity], result of:
           0.05095249 = score(doc=139,freq=2.0), product of:
             0.10433342 = queryWeight, product of:
               1.5132443 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.012478646 = queryNorm
             0.4883621 = fieldWeight in 139, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
         0.02446752 = weight(abstract_txt:based in 139) [ClassicSimilarity], result of:
           0.02446752 = score(doc=139,freq=2.0), product of:
             0.086833335 = queryWeight, product of:
               2.182783 = boost
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.012478646 = queryNorm
             0.28177565 = fieldWeight in 139, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
         0.06613952 = weight(abstract_txt:average in 139) [ClassicSimilarity], result of:
           0.06613952 = score(doc=139,freq=1.0), product of:
             0.17905822 = queryWeight, product of:
               2.4279518 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.012478646 = queryNorm
             0.36937436 = fieldWeight in 139, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
         0.15374108 = weight(abstract_txt:relevance in 139) [ClassicSimilarity], result of:
           0.15374108 = score(doc=139,freq=4.0), product of:
             0.24938494 = queryWeight, product of:
               4.0522203 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012478646 = queryNorm
             0.616481 = fieldWeight in 139, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
         0.3459028 = weight(abstract_txt:graded in 139) [ClassicSimilarity], result of:
           0.3459028 = score(doc=139,freq=1.0), product of:
             0.63964534 = queryWeight, product of:
               5.924299 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012478646 = queryNorm
             0.5407728 = fieldWeight in 139, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0625 = fieldNorm(doc=139)
       0.28 = coord(7/25)
   

4. Sun, A.; Bhowmick, S.S.; Nguyen, K.T.N.; Bai, G.: Tag-based social image retrieval : an empirical evaluation (2011) 0.19

   0.18895137 = sum of:
     0.18895137 = product of:
       0.5248649 = sum of:
         0.006193056 = weight(abstract_txt:with in 4938) [ClassicSimilarity], result of:
           0.006193056 = score(doc=4938,freq=2.0), product of:
             0.032033782 = queryWeight, product of:
               1.0269448 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.012478646 = queryNorm
             0.1933289 = fieldWeight in 4938, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.041616082 = weight(abstract_txt:cumulative in 4938) [ClassicSimilarity], result of:
           0.041616082 = score(doc=4938,freq=1.0), product of:
             0.09965106 = queryWeight, product of:
               1.0457389 = boost
               7.636444 = idf(docFreq=57, maxDocs=44218)
               0.012478646 = queryNorm
             0.41761804 = fieldWeight in 4938, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.636444 = idf(docFreq=57, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.06944806 = weight(abstract_txt:discounted in 4938) [ClassicSimilarity], result of:
           0.06944806 = score(doc=4938,freq=1.0), product of:
             0.14019996 = queryWeight, product of:
               1.2403843 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.012478646 = queryNorm
             0.49535006 = fieldWeight in 4938, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.04458343 = weight(abstract_txt:precision in 4938) [ClassicSimilarity], result of:
           0.04458343 = score(doc=4938,freq=2.0), product of:
             0.10433342 = queryWeight, product of:
               1.5132443 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.012478646 = queryNorm
             0.42731684 = fieldWeight in 4938, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.05322016 = weight(abstract_txt:gain in 4938) [ClassicSimilarity], result of:
           0.05322016 = score(doc=4938,freq=1.0), product of:
             0.14792284 = queryWeight, product of:
               1.8018347 = boost
               6.578893 = idf(docFreq=166, maxDocs=44218)
               0.012478646 = queryNorm
             0.35978323 = fieldWeight in 4938, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.578893 = idf(docFreq=166, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.02140908 = weight(abstract_txt:based in 4938) [ClassicSimilarity], result of:
           0.02140908 = score(doc=4938,freq=2.0), product of:
             0.086833335 = queryWeight, product of:
               2.182783 = boost
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.012478646 = queryNorm
             0.24655369 = fieldWeight in 4938, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.05787208 = weight(abstract_txt:average in 4938) [ClassicSimilarity], result of:
           0.05787208 = score(doc=4938,freq=1.0), product of:
             0.17905822 = queryWeight, product of:
               2.4279518 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.012478646 = queryNorm
             0.32320258 = fieldWeight in 4938, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.067261726 = weight(abstract_txt:relevance in 4938) [ClassicSimilarity], result of:
           0.067261726 = score(doc=4938,freq=1.0), product of:
             0.24938494 = queryWeight, product of:
               4.0522203 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012478646 = queryNorm
             0.26971045 = fieldWeight in 4938, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
         0.16326123 = weight(abstract_txt:metrics in 4938) [ClassicSimilarity], result of:
           0.16326123 = score(doc=4938,freq=1.0), product of:
             0.4504158 = queryWeight, product of:
               5.4458385 = boost
               6.627983 = idf(docFreq=158, maxDocs=44218)
               0.012478646 = queryNorm
             0.36246783 = fieldWeight in 4938, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.627983 = idf(docFreq=158, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4938)
       0.36 = coord(9/25)
   

5. Bilal, D.: Ranking, relevance judgment, and precision of information retrieval on children's queries : evaluation of Google, Yahoo!, Bing, Yahoo! Kids, and ask Kids (2012) 0.19

   0.18668607 = sum of:
     0.18668607 = product of:
       0.66673595 = sum of:
         0.03657681 = weight(abstract_txt:metric in 393) [ClassicSimilarity], result of:
           0.03657681 = score(doc=393,freq=1.0), product of:
             0.09143484 = queryWeight, product of:
               1.0017011 = boost
               7.314861 = idf(docFreq=79, maxDocs=44218)
               0.012478646 = queryNorm
             0.40003145 = fieldWeight in 393, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.314861 = idf(docFreq=79, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
         0.0075849127 = weight(abstract_txt:with in 393) [ClassicSimilarity], result of:
           0.0075849127 = score(doc=393,freq=3.0), product of:
             0.032033782 = queryWeight, product of:
               1.0269448 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.012478646 = queryNorm
             0.23677856 = fieldWeight in 393, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
         0.08916686 = weight(abstract_txt:precision in 393) [ClassicSimilarity], result of:
           0.08916686 = score(doc=393,freq=8.0), product of:
             0.10433342 = queryWeight, product of:
               1.5132443 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.012478646 = queryNorm
             0.8546337 = fieldWeight in 393, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
         0.035382617 = weight(abstract_txt:best in 393) [ClassicSimilarity], result of:
           0.035382617 = score(doc=393,freq=1.0), product of:
             0.12898569 = queryWeight, product of:
               2.060695 = boost
               5.0160327 = idf(docFreq=796, maxDocs=44218)
               0.012478646 = queryNorm
             0.27431428 = fieldWeight in 393, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0160327 = idf(docFreq=796, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
         0.10023738 = weight(abstract_txt:average in 393) [ClassicSimilarity], result of:
           0.10023738 = score(doc=393,freq=3.0), product of:
             0.17905822 = queryWeight, product of:
               2.4279518 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.012478646 = queryNorm
             0.55980325 = fieldWeight in 393, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
         0.095122434 = weight(abstract_txt:relevance in 393) [ClassicSimilarity], result of:
           0.095122434 = score(doc=393,freq=2.0), product of:
             0.24938494 = queryWeight, product of:
               4.0522203 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012478646 = queryNorm
             0.38142815 = fieldWeight in 393, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
         0.30266494 = weight(abstract_txt:graded in 393) [ClassicSimilarity], result of:
           0.30266494 = score(doc=393,freq=1.0), product of:
             0.63964534 = queryWeight, product of:
               5.924299 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012478646 = queryNorm
             0.47317618 = fieldWeight in 393, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0546875 = fieldNorm(doc=393)
       0.28 = coord(7/25)