Document (#26823)

Author

Bodoff, D.
Wu, B.
Wong, K.Y.M.

Title

Relevance data for language models using maximum likelihood

Source

Journal of the American Society for Information Science and technology. 54(2003) no.11, S.1050-1061

Year

2003

Abstract

We present a preliminary empirical test of a maximum likelihood approach to using relevance data for training information retrieval (IR) parameters. Similar to language models, our method uses explicitly hypothesized distributions for documents and queries, but we add to this an explicitly hypothesized distribution for relevance judgments. The method unifies document-oriented and query-oriented views. Performance is better than the Rocchio heuristic for document and/or query modification. The maximum likelihood methodology also motivates a heuristic estimate of the MLE optimization. The method can be used to test competing hypotheses regarding the processes of authors' term selection, searchers' term selection, and assessors' relevancy judgments.

Similar documents (author)

1. Bodoff, D.; Wong, S.P.-S.: Documents and queries as random variables : history and implications (2006) 5.34

   5.339114 = sum of:
     5.339114 = sum of:
       2.2742572 = weight(author_txt:wong in 193) [ClassicSimilarity], result of:
         2.2742572 = score(doc=193,freq=1.0), product of:
           0.63390994 = queryWeight, product of:
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.077302516 = queryNorm
           3.587666 = fieldWeight in 193, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.4375 = fieldNorm(doc=193)
       3.0648568 = weight(author_txt:bodoff in 193) [ClassicSimilarity], result of:
         3.0648568 = score(doc=193,freq=1.0), product of:
           0.7734068 = queryWeight, product of:
             1.1045623 = boost
             9.05783 = idf(docFreq=13, maxDocs=44218)
             0.077302516 = queryNorm
           3.9628005 = fieldWeight in 193, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.05783 = idf(docFreq=13, maxDocs=44218)
             0.4375 = fieldNorm(doc=193)
   

2. Bodoff, D.: ¬A re-unification of two competing models for document retrieval (1999) 2.19

   2.1891835 = sum of:
     2.1891835 = product of:
       4.378367 = sum of:
         4.378367 = weight(author_txt:bodoff in 2951) [ClassicSimilarity], result of:
           4.378367 = score(doc=2951,freq=1.0), product of:
             0.7734068 = queryWeight, product of:
               1.1045623 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.077302516 = queryNorm
             5.661144 = fieldWeight in 2951, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.625 = fieldNorm(doc=2951)
       0.5 = coord(1/2)
   

3. Bodoff, D.: Relevance for browsing, relevance for searching (2006) 2.19

   2.1891835 = sum of:
     2.1891835 = product of:
       4.378367 = sum of:
         4.378367 = weight(author_txt:bodoff in 4909) [ClassicSimilarity], result of:
           4.378367 = score(doc=4909,freq=1.0), product of:
             0.7734068 = queryWeight, product of:
               1.1045623 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.077302516 = queryNorm
             5.661144 = fieldWeight in 4909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.625 = fieldNorm(doc=4909)
       0.5 = coord(1/2)
   

4. Bodoff, D.: Test theory for evaluating reliability of IR test collections (2008) 2.19

   2.1891835 = sum of:
     2.1891835 = product of:
       4.378367 = sum of:
         4.378367 = weight(author_txt:bodoff in 2085) [ClassicSimilarity], result of:
           4.378367 = score(doc=2085,freq=1.0), product of:
             0.7734068 = queryWeight, product of:
               1.1045623 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.077302516 = queryNorm
             5.661144 = fieldWeight in 2085, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.625 = fieldNorm(doc=2085)
       0.5 = coord(1/2)
   

5. Bodoff, D.: Emergence of terminological conventions as a searcher-indexer coordination game (2009) 2.19

   2.1891835 = sum of:
     2.1891835 = product of:
       4.378367 = sum of:
         4.378367 = weight(author_txt:bodoff in 3299) [ClassicSimilarity], result of:
           4.378367 = score(doc=3299,freq=1.0), product of:
             0.7734068 = queryWeight, product of:
               1.1045623 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.077302516 = queryNorm
             5.661144 = fieldWeight in 3299, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.625 = fieldNorm(doc=3299)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Zhang, Y.; Xu, W.: Fast exact maximum likelihood estimation for mixture of language model (2008) 0.27

   0.2667769 = sum of:
     0.2667769 = product of:
       0.83367777 = sum of:
         0.05587823 = weight(abstract_txt:estimate in 2082) [ClassicSimilarity], result of:
           0.05587823 = score(doc=2082,freq=1.0), product of:
             0.12422426 = queryWeight, product of:
               7.1970778 = idf(docFreq=89, maxDocs=44218)
               0.017260375 = queryNorm
             0.44981736 = fieldWeight in 2082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1970778 = idf(docFreq=89, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.05371047 = weight(abstract_txt:language in 2082) [ClassicSimilarity], result of:
           0.05371047 = score(doc=2082,freq=6.0), product of:
             0.083890006 = queryWeight, product of:
               1.1621627 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.017260375 = queryNorm
             0.6402488 = fieldWeight in 2082, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.033533804 = weight(abstract_txt:document in 2082) [ClassicSimilarity], result of:
           0.033533804 = score(doc=2082,freq=2.0), product of:
             0.08838256 = queryWeight, product of:
               1.1928754 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017260375 = queryNorm
             0.37941656 = fieldWeight in 2082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.029977815 = weight(abstract_txt:models in 2082) [ClassicSimilarity], result of:
           0.029977815 = score(doc=2082,freq=1.0), product of:
             0.103336714 = queryWeight, product of:
               1.2898498 = boost
               4.6415744 = idf(docFreq=1158, maxDocs=44218)
               0.017260375 = queryNorm
             0.2900984 = fieldWeight in 2082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6415744 = idf(docFreq=1158, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.04662801 = weight(abstract_txt:selection in 2082) [ClassicSimilarity], result of:
           0.04662801 = score(doc=2082,freq=1.0), product of:
             0.13872431 = queryWeight, product of:
               1.494473 = boost
               5.377919 = idf(docFreq=554, maxDocs=44218)
               0.017260375 = queryNorm
             0.33611995 = fieldWeight in 2082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.377919 = idf(docFreq=554, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.09997457 = weight(abstract_txt:explicitly in 2082) [ClassicSimilarity], result of:
           0.09997457 = score(doc=2082,freq=1.0), product of:
             0.23066464 = queryWeight, product of:
               1.9270914 = boost
               6.9347134 = idf(docFreq=116, maxDocs=44218)
               0.017260375 = queryNorm
             0.4334196 = fieldWeight in 2082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9347134 = idf(docFreq=116, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.24159224 = weight(abstract_txt:maximum in 2082) [ClassicSimilarity], result of:
           0.24159224 = score(doc=2082,freq=2.0), product of:
             0.3773958 = queryWeight, product of:
               3.0189502 = boost
               7.24254 = idf(docFreq=85, maxDocs=44218)
               0.017260375 = queryNorm
             0.64015615 = fieldWeight in 2082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.24254 = idf(docFreq=85, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
         0.2723826 = weight(abstract_txt:likelihood in 2082) [ClassicSimilarity], result of:
           0.2723826 = score(doc=2082,freq=2.0), product of:
             0.4088161 = queryWeight, product of:
               3.1421103 = boost
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.017260375 = queryNorm
             0.6662717 = fieldWeight in 2082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.0625 = fieldNorm(doc=2082)
       0.32 = coord(8/25)
   

2. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.21

   0.21387628 = sum of:
     0.21387628 = product of:
       0.76384383 = sum of:
         0.08359029 = weight(abstract_txt:relevancy in 2591) [ClassicSimilarity], result of:
           0.08359029 = score(doc=2591,freq=1.0), product of:
             0.16248573 = queryWeight, product of:
               1.1436797 = boost
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.017260375 = queryNorm
             0.514447 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.047423963 = weight(abstract_txt:document in 2591) [ClassicSimilarity], result of:
           0.047423963 = score(doc=2591,freq=4.0), product of:
             0.08838256 = queryWeight, product of:
               1.1928754 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017260375 = queryNorm
             0.53657603 = fieldWeight in 2591, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.17406975 = weight(abstract_txt:assessors in 2591) [ClassicSimilarity], result of:
           0.17406975 = score(doc=2591,freq=3.0), product of:
             0.1837186 = queryWeight, product of:
               1.2161114 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.017260375 = queryNorm
             0.94748026 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.038530335 = weight(abstract_txt:test in 2591) [ClassicSimilarity], result of:
           0.038530335 = score(doc=2591,freq=1.0), product of:
             0.122158356 = queryWeight, product of:
               1.4024048 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.017260375 = queryNorm
             0.315413 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.05798595 = weight(abstract_txt:method in 2591) [ClassicSimilarity], result of:
           0.05798595 = score(doc=2591,freq=2.0), product of:
             0.14575505 = queryWeight, product of:
               1.8761573 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.017260375 = queryNorm
             0.3978315 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.21952732 = weight(abstract_txt:judgments in 2591) [ClassicSimilarity], result of:
           0.21952732 = score(doc=2591,freq=5.0), product of:
             0.22788918 = queryWeight, product of:
               1.9154625 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.017260375 = queryNorm
             0.9633073 = fieldWeight in 2591, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.14271621 = weight(abstract_txt:relevance in 2591) [ClassicSimilarity], result of:
           0.14271621 = score(doc=2591,freq=7.0), product of:
             0.17499861 = queryWeight, product of:
               2.0557709 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017260375 = queryNorm
             0.8155277 = fieldWeight in 2591, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
       0.28 = coord(7/25)
   

3. Pan, M.; Huang, J.X.; He, T.; Mao, Z.; Ying, Z.; Tu, X.: ¬A simple kernel co-occurrence-based enhancement for pseudo-relevance feedback (2020) 0.17

   0.17193331 = sum of:
     0.17193331 = product of:
       0.5372916 = sum of:
         0.02192721 = weight(abstract_txt:language in 5678) [ClassicSimilarity], result of:
           0.02192721 = score(doc=5678,freq=1.0), product of:
             0.083890006 = queryWeight, product of:
               1.1621627 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.017260375 = queryNorm
             0.26138046 = fieldWeight in 5678, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.023711981 = weight(abstract_txt:document in 5678) [ClassicSimilarity], result of:
           0.023711981 = score(doc=5678,freq=1.0), product of:
             0.08838256 = queryWeight, product of:
               1.1928754 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017260375 = queryNorm
             0.26828802 = fieldWeight in 5678, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.029977815 = weight(abstract_txt:models in 5678) [ClassicSimilarity], result of:
           0.029977815 = score(doc=5678,freq=1.0), product of:
             0.103336714 = queryWeight, product of:
               1.2898498 = boost
               4.6415744 = idf(docFreq=1158, maxDocs=44218)
               0.017260375 = queryNorm
             0.2900984 = fieldWeight in 5678, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6415744 = idf(docFreq=1158, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.18172452 = weight(abstract_txt:rocchio in 5678) [ClassicSimilarity], result of:
           0.18172452 = score(doc=5678,freq=2.0), product of:
             0.21642646 = queryWeight, product of:
               1.3199333 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.017260375 = queryNorm
             0.83965945 = fieldWeight in 5678, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.06440907 = weight(abstract_txt:query in 5678) [ClassicSimilarity], result of:
           0.06440907 = score(doc=5678,freq=4.0), product of:
             0.10839253 = queryWeight, product of:
               1.3210263 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.017260375 = queryNorm
             0.5942206 = fieldWeight in 5678, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.08127002 = weight(abstract_txt:term in 5678) [ClassicSimilarity], result of:
           0.08127002 = score(doc=5678,freq=6.0), product of:
             0.11056673 = queryWeight, product of:
               1.3342094 = boost
               4.8012047 = idf(docFreq=987, maxDocs=44218)
               0.017260375 = queryNorm
             0.73503137 = fieldWeight in 5678, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.8012047 = idf(docFreq=987, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.05798595 = weight(abstract_txt:method in 5678) [ClassicSimilarity], result of:
           0.05798595 = score(doc=5678,freq=2.0), product of:
             0.14575505 = queryWeight, product of:
               1.8761573 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.017260375 = queryNorm
             0.3978315 = fieldWeight in 5678, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
         0.07628503 = weight(abstract_txt:relevance in 5678) [ClassicSimilarity], result of:
           0.07628503 = score(doc=5678,freq=2.0), product of:
             0.17499861 = queryWeight, product of:
               2.0557709 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017260375 = queryNorm
             0.43591788 = fieldWeight in 5678, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5678)
       0.32 = coord(8/25)
   

4. Meij, E.; Rijke, M. de: Thesaurus-based feedback to support mixed search and browsing environments (2007) 0.16

   0.16071813 = sum of:
     0.16071813 = product of:
       0.5022442 = sum of:
         0.08359029 = weight(abstract_txt:relevancy in 2432) [ClassicSimilarity], result of:
           0.08359029 = score(doc=2432,freq=1.0), product of:
             0.16248573 = queryWeight, product of:
               1.1436797 = boost
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.017260375 = queryNorm
             0.514447 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.02192721 = weight(abstract_txt:language in 2432) [ClassicSimilarity], result of:
           0.02192721 = score(doc=2432,freq=1.0), product of:
             0.083890006 = queryWeight, product of:
               1.1621627 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.017260375 = queryNorm
             0.26138046 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.023711981 = weight(abstract_txt:document in 2432) [ClassicSimilarity], result of:
           0.023711981 = score(doc=2432,freq=1.0), product of:
             0.08838256 = queryWeight, product of:
               1.1928754 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017260375 = queryNorm
             0.26828802 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.042395033 = weight(abstract_txt:models in 2432) [ClassicSimilarity], result of:
           0.042395033 = score(doc=2432,freq=2.0), product of:
             0.103336714 = queryWeight, product of:
               1.2898498 = boost
               4.6415744 = idf(docFreq=1158, maxDocs=44218)
               0.017260375 = queryNorm
             0.4102611 = fieldWeight in 2432, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6415744 = idf(docFreq=1158, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.04554409 = weight(abstract_txt:query in 2432) [ClassicSimilarity], result of:
           0.04554409 = score(doc=2432,freq=2.0), product of:
             0.10839253 = queryWeight, product of:
               1.3210263 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.017260375 = queryNorm
             0.4201774 = fieldWeight in 2432, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.038530335 = weight(abstract_txt:test in 2432) [ClassicSimilarity], result of:
           0.038530335 = score(doc=2432,freq=1.0), product of:
             0.122158356 = queryWeight, product of:
               1.4024048 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.017260375 = queryNorm
             0.315413 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.05394166 = weight(abstract_txt:relevance in 2432) [ClassicSimilarity], result of:
           0.05394166 = score(doc=2432,freq=1.0), product of:
             0.17499861 = queryWeight, product of:
               2.0557709 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017260375 = queryNorm
             0.3082405 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.19260359 = weight(abstract_txt:likelihood in 2432) [ClassicSimilarity], result of:
           0.19260359 = score(doc=2432,freq=1.0), product of:
             0.4088161 = queryWeight, product of:
               3.1421103 = boost
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.017260375 = queryNorm
             0.47112525 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
       0.32 = coord(8/25)
   

5. Bodoff, D.: Test theory for evaluating reliability of IR test collections (2008) 0.14

   0.14014979 = sum of:
     0.14014979 = product of:
       0.58395743 = sum of:
         0.05587823 = weight(abstract_txt:estimate in 2085) [ClassicSimilarity], result of:
           0.05587823 = score(doc=2085,freq=1.0), product of:
             0.12422426 = queryWeight, product of:
               7.1970778 = idf(docFreq=89, maxDocs=44218)
               0.017260375 = queryNorm
             0.44981736 = fieldWeight in 2085, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1970778 = idf(docFreq=89, maxDocs=44218)
               0.0625 = fieldNorm(doc=2085)
         0.033533804 = weight(abstract_txt:document in 2085) [ClassicSimilarity], result of:
           0.033533804 = score(doc=2085,freq=2.0), product of:
             0.08838256 = queryWeight, product of:
               1.1928754 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017260375 = queryNorm
             0.37941656 = fieldWeight in 2085, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=2085)
         0.20099844 = weight(abstract_txt:assessors in 2085) [ClassicSimilarity], result of:
           0.20099844 = score(doc=2085,freq=4.0), product of:
             0.1837186 = queryWeight, product of:
               1.2161114 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.017260375 = queryNorm
             1.094056 = fieldWeight in 2085, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=2085)
         0.101941675 = weight(abstract_txt:test in 2085) [ClassicSimilarity], result of:
           0.101941675 = score(doc=2085,freq=7.0), product of:
             0.122158356 = queryWeight, product of:
               1.4024048 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.017260375 = queryNorm
             0.8345043 = fieldWeight in 2085, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0625 = fieldNorm(doc=2085)
         0.0981756 = weight(abstract_txt:judgments in 2085) [ClassicSimilarity], result of:
           0.0981756 = score(doc=2085,freq=1.0), product of:
             0.22788918 = queryWeight, product of:
               1.9154625 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.017260375 = queryNorm
             0.43080413 = fieldWeight in 2085, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=2085)
         0.0934297 = weight(abstract_txt:relevance in 2085) [ClassicSimilarity], result of:
           0.0934297 = score(doc=2085,freq=3.0), product of:
             0.17499861 = queryWeight, product of:
               2.0557709 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017260375 = queryNorm
             0.5338882 = fieldWeight in 2085, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=2085)
       0.24 = coord(6/25)