Document (#26823)

Author

Bodoff, D.
Wu, B.
Wong, K.Y.M.

Title

Relevance data for language models using maximum likelihood

Source

Journal of the American Society for Information Science and technology. 54(2003) no.11, S.1050-1061

Year

2003

Abstract

We present a preliminary empirical test of a maximum likelihood approach to using relevance data for training information retrieval (IR) parameters. Similar to language models, our method uses explicitly hypothesized distributions for documents and queries, but we add to this an explicitly hypothesized distribution for relevance judgments. The method unifies document-oriented and query-oriented views. Performance is better than the Rocchio heuristic for document and/or query modification. The maximum likelihood methodology also motivates a heuristic estimate of the MLE optimization. The method can be used to test competing hypotheses regarding the processes of authors' term selection, searchers' term selection, and assessors' relevancy judgments.

Similar documents (author)

1. Bodoff, D.; Wong, S.P.-S.: Documents and queries as random variables : history and implications (2006) 5.34

   5.3419485 = sum of:
     5.3419485 = sum of:
       2.2756715 = weight(author_txt:wong in 318) [ClassicSimilarity], result of:
         2.2756715 = score(doc=318,freq=1.0), product of:
           0.63395005 = queryWeight, product of:
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.07726425 = queryNorm
           3.58967 = fieldWeight in 318, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.4375 = fieldNorm(doc=318)
       3.0662768 = weight(author_txt:bodoff in 318) [ClassicSimilarity], result of:
         3.0662768 = score(doc=318,freq=1.0), product of:
           0.77337396 = queryWeight, product of:
             1.1045039 = boost
             9.06241 = idf(docFreq=13, maxDocs=44421)
             0.07726425 = queryNorm
           3.9648046 = fieldWeight in 318, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.06241 = idf(docFreq=13, maxDocs=44421)
             0.4375 = fieldNorm(doc=318)
   

2. Bodoff, D.: ¬A re-unification of two competing models for document retrieval (1999) 2.19

   2.1901975 = sum of:
     2.1901975 = product of:
       4.380395 = sum of:
         4.380395 = weight(author_txt:bodoff in 3951) [ClassicSimilarity], result of:
           4.380395 = score(doc=3951,freq=1.0), product of:
             0.77337396 = queryWeight, product of:
               1.1045039 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.07726425 = queryNorm
             5.664006 = fieldWeight in 3951, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.625 = fieldNorm(doc=3951)
       0.5 = coord(1/2)
   

3. Bodoff, D.: Relevance for browsing, relevance for searching (2006) 2.19

   2.1901975 = sum of:
     2.1901975 = product of:
       4.380395 = sum of:
         4.380395 = weight(author_txt:bodoff in 5909) [ClassicSimilarity], result of:
           4.380395 = score(doc=5909,freq=1.0), product of:
             0.77337396 = queryWeight, product of:
               1.1045039 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.07726425 = queryNorm
             5.664006 = fieldWeight in 5909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.625 = fieldNorm(doc=5909)
       0.5 = coord(1/2)
   

4. Bodoff, D.: Test theory for evaluating reliability of IR test collections (2008) 2.19

   2.1901975 = sum of:
     2.1901975 = product of:
       4.380395 = sum of:
         4.380395 = weight(author_txt:bodoff in 3085) [ClassicSimilarity], result of:
           4.380395 = score(doc=3085,freq=1.0), product of:
             0.77337396 = queryWeight, product of:
               1.1045039 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.07726425 = queryNorm
             5.664006 = fieldWeight in 3085, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.625 = fieldNorm(doc=3085)
       0.5 = coord(1/2)
   

5. Bodoff, D.: Emergence of terminological conventions as a searcher-indexer coordination game (2009) 2.19

   2.1901975 = sum of:
     2.1901975 = product of:
       4.380395 = sum of:
         4.380395 = weight(author_txt:bodoff in 286) [ClassicSimilarity], result of:
           4.380395 = score(doc=286,freq=1.0), product of:
             0.77337396 = queryWeight, product of:
               1.1045039 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.07726425 = queryNorm
             5.664006 = fieldWeight in 286, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.625 = fieldNorm(doc=286)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Zhang, Y.; Xu, W.: Fast exact maximum likelihood estimation for mixture of language model (2008) 0.27

   0.26636365 = sum of:
     0.26636365 = product of:
       0.83238643 = sum of:
         0.05579007 = weight(abstract_txt:estimate in 3082) [ClassicSimilarity], result of:
           0.05579007 = score(doc=3082,freq=1.0), product of:
             0.12413987 = queryWeight, product of:
               7.190608 = idf(docFreq=90, maxDocs=44421)
               0.017264169 = queryNorm
             0.449413 = fieldWeight in 3082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.190608 = idf(docFreq=90, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.053343613 = weight(abstract_txt:language in 3082) [ClassicSimilarity], result of:
           0.053343613 = score(doc=3082,freq=6.0), product of:
             0.083538726 = queryWeight, product of:
               1.160121 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.017264169 = queryNorm
             0.6385495 = fieldWeight in 3082, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.033607535 = weight(abstract_txt:document in 3082) [ClassicSimilarity], result of:
           0.033607535 = score(doc=3082,freq=2.0), product of:
             0.088545084 = queryWeight, product of:
               1.1943774 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.017264169 = queryNorm
             0.3795528 = fieldWeight in 3082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.029654954 = weight(abstract_txt:models in 3082) [ClassicSimilarity], result of:
           0.029654954 = score(doc=3082,freq=1.0), product of:
             0.102631696 = queryWeight, product of:
               1.285879 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.017264169 = queryNorm
             0.28894538 = fieldWeight in 3082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.04661252 = weight(abstract_txt:selection in 3082) [ClassicSimilarity], result of:
           0.04661252 = score(doc=3082,freq=1.0), product of:
             0.13874534 = queryWeight, product of:
               1.4950942 = boost
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.017264169 = queryNorm
             0.33595738 = fieldWeight in 3082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.09955192 = weight(abstract_txt:explicitly in 3082) [ClassicSimilarity], result of:
           0.09955192 = score(doc=3082,freq=1.0), product of:
             0.2300999 = queryWeight, product of:
               1.925385 = boost
               6.922344 = idf(docFreq=118, maxDocs=44421)
               0.017264169 = queryNorm
             0.4326465 = fieldWeight in 3082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.922344 = idf(docFreq=118, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.24232195 = weight(abstract_txt:maximum in 3082) [ClassicSimilarity], result of:
           0.24232195 = score(doc=3082,freq=2.0), product of:
             0.37829646 = queryWeight, product of:
               3.0235777 = boost
               7.2471204 = idf(docFreq=85, maxDocs=44421)
               0.017264169 = queryNorm
             0.640561 = fieldWeight in 3082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.2471204 = idf(docFreq=85, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
         0.27150387 = weight(abstract_txt:likelihood in 3082) [ClassicSimilarity], result of:
           0.27150387 = score(doc=3082,freq=2.0), product of:
             0.40808856 = queryWeight, product of:
               3.14038 = boost
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.017264169 = queryNorm
             0.6653062 = fieldWeight in 3082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.0625 = fieldNorm(doc=3082)
       0.32 = coord(8/25)
   

2. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.21

   0.21299657 = sum of:
     0.21299657 = product of:
       0.760702 = sum of:
         0.08382368 = weight(abstract_txt:relevancy in 3591) [ClassicSimilarity], result of:
           0.08382368 = score(doc=3591,freq=1.0), product of:
             0.16284877 = queryWeight, product of:
               1.1453457 = boost
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.017264169 = queryNorm
             0.51473325 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.047528233 = weight(abstract_txt:document in 3591) [ClassicSimilarity], result of:
           0.047528233 = score(doc=3591,freq=4.0), product of:
             0.088545084 = queryWeight, product of:
               1.1943774 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.017264169 = queryNorm
             0.53676873 = fieldWeight in 3591, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.1714893 = weight(abstract_txt:assessors in 3591) [ClassicSimilarity], result of:
           0.1714893 = score(doc=3591,freq=3.0), product of:
             0.18196629 = queryWeight, product of:
               1.2107092 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.017264169 = queryNorm
             0.94242346 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.03856016 = weight(abstract_txt:test in 3591) [ClassicSimilarity], result of:
           0.03856016 = score(doc=3591,freq=1.0), product of:
             0.122266985 = queryWeight, product of:
               1.403505 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.017264169 = queryNorm
             0.3153767 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.057967994 = weight(abstract_txt:method in 3591) [ClassicSimilarity], result of:
           0.057967994 = score(doc=3591,freq=2.0), product of:
             0.14577933 = queryWeight, product of:
               1.8769511 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.017264169 = queryNorm
             0.39764208 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.2186578 = weight(abstract_txt:judgments in 3591) [ClassicSimilarity], result of:
           0.2186578 = score(doc=3591,freq=5.0), product of:
             0.22737184 = queryWeight, product of:
               1.9139372 = boost
               6.881186 = idf(docFreq=123, maxDocs=44421)
               0.017264169 = queryNorm
             0.961675 = fieldWeight in 3591, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.881186 = idf(docFreq=123, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.14267485 = weight(abstract_txt:relevance in 3591) [ClassicSimilarity], result of:
           0.14267485 = score(doc=3591,freq=7.0), product of:
             0.17503008 = queryWeight, product of:
               2.0566545 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.017264169 = queryNorm
             0.8151447 = fieldWeight in 3591, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
       0.28 = coord(7/25)
   

3. Pan, M.; Huang, J.X.; He, T.; Mao, Z.; Ying, Z.; Tu, X.: ¬A simple kernel co-occurrence-based enhancement for pseudo-relevance feedback (2020) 0.17

   0.1718915 = sum of:
     0.1718915 = product of:
       0.53716093 = sum of:
         0.021777436 = weight(abstract_txt:language in 678) [ClassicSimilarity], result of:
           0.021777436 = score(doc=678,freq=1.0), product of:
             0.083538726 = queryWeight, product of:
               1.160121 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.017264169 = queryNorm
             0.26068673 = fieldWeight in 678, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.023764117 = weight(abstract_txt:document in 678) [ClassicSimilarity], result of:
           0.023764117 = score(doc=678,freq=1.0), product of:
             0.088545084 = queryWeight, product of:
               1.1943774 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.017264169 = queryNorm
             0.26838437 = fieldWeight in 678, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.029654954 = weight(abstract_txt:models in 678) [ClassicSimilarity], result of:
           0.029654954 = score(doc=678,freq=1.0), product of:
             0.102631696 = queryWeight, product of:
               1.285879 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.017264169 = queryNorm
             0.28894538 = fieldWeight in 678, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.1821913 = weight(abstract_txt:rocchio in 678) [ClassicSimilarity], result of:
           0.1821913 = score(doc=678,freq=2.0), product of:
             0.2168778 = queryWeight, product of:
               1.3217579 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.017264169 = queryNorm
             0.8400643 = fieldWeight in 678, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.064510785 = weight(abstract_txt:query in 678) [ClassicSimilarity], result of:
           0.064510785 = score(doc=678,freq=4.0), product of:
             0.1085471 = queryWeight, product of:
               1.3224171 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.017264169 = queryNorm
             0.5943115 = fieldWeight in 678, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.08103144 = weight(abstract_txt:term in 678) [ClassicSimilarity], result of:
           0.08103144 = score(doc=678,freq=6.0), product of:
             0.11039141 = queryWeight, product of:
               1.3336043 = boost
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.017264169 = queryNorm
             0.7340376 = fieldWeight in 678, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.794713 = idf(docFreq=998, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.057967994 = weight(abstract_txt:method in 678) [ClassicSimilarity], result of:
           0.057967994 = score(doc=678,freq=2.0), product of:
             0.14577933 = queryWeight, product of:
               1.8769511 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.017264169 = queryNorm
             0.39764208 = fieldWeight in 678, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
         0.07626291 = weight(abstract_txt:relevance in 678) [ClassicSimilarity], result of:
           0.07626291 = score(doc=678,freq=2.0), product of:
             0.17503008 = queryWeight, product of:
               2.0566545 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.017264169 = queryNorm
             0.43571317 = fieldWeight in 678, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=678)
       0.32 = coord(8/25)
   

4. Meij, E.; Rijke, M. de: Thesaurus-based feedback to support mixed search and browsing environments (2007) 0.16

   0.16044419 = sum of:
     0.16044419 = product of:
       0.5013881 = sum of:
         0.08382368 = weight(abstract_txt:relevancy in 3432) [ClassicSimilarity], result of:
           0.08382368 = score(doc=3432,freq=1.0), product of:
             0.16284877 = queryWeight, product of:
               1.1453457 = boost
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.017264169 = queryNorm
             0.51473325 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.021777436 = weight(abstract_txt:language in 3432) [ClassicSimilarity], result of:
           0.021777436 = score(doc=3432,freq=1.0), product of:
             0.083538726 = queryWeight, product of:
               1.160121 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.017264169 = queryNorm
             0.26068673 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.023764117 = weight(abstract_txt:document in 3432) [ClassicSimilarity], result of:
           0.023764117 = score(doc=3432,freq=1.0), product of:
             0.088545084 = queryWeight, product of:
               1.1943774 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.017264169 = queryNorm
             0.26838437 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.041938435 = weight(abstract_txt:models in 3432) [ClassicSimilarity], result of:
           0.041938435 = score(doc=3432,freq=2.0), product of:
             0.102631696 = queryWeight, product of:
               1.285879 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.017264169 = queryNorm
             0.40863046 = fieldWeight in 3432, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.045616012 = weight(abstract_txt:query in 3432) [ClassicSimilarity], result of:
           0.045616012 = score(doc=3432,freq=2.0), product of:
             0.1085471 = queryWeight, product of:
               1.3224171 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.017264169 = queryNorm
             0.42024165 = fieldWeight in 3432, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.03856016 = weight(abstract_txt:test in 3432) [ClassicSimilarity], result of:
           0.03856016 = score(doc=3432,freq=1.0), product of:
             0.122266985 = queryWeight, product of:
               1.403505 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.017264169 = queryNorm
             0.3153767 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.053926025 = weight(abstract_txt:relevance in 3432) [ClassicSimilarity], result of:
           0.053926025 = score(doc=3432,freq=1.0), product of:
             0.17503008 = queryWeight, product of:
               2.0566545 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.017264169 = queryNorm
             0.30809575 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
         0.19198222 = weight(abstract_txt:likelihood in 3432) [ClassicSimilarity], result of:
           0.19198222 = score(doc=3432,freq=1.0), product of:
             0.40808856 = queryWeight, product of:
               3.14038 = boost
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.017264169 = queryNorm
             0.47044253 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.0625 = fieldNorm(doc=3432)
       0.32 = coord(8/25)
   

5. Bodoff, D.: Test theory for evaluating reliability of IR test collections (2008) 0.14

   0.13935032 = sum of:
     0.13935032 = product of:
       0.58062637 = sum of:
         0.05579007 = weight(abstract_txt:estimate in 3085) [ClassicSimilarity], result of:
           0.05579007 = score(doc=3085,freq=1.0), product of:
             0.12413987 = queryWeight, product of:
               7.190608 = idf(docFreq=90, maxDocs=44421)
               0.017264169 = queryNorm
             0.449413 = fieldWeight in 3085, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.190608 = idf(docFreq=90, maxDocs=44421)
               0.0625 = fieldNorm(doc=3085)
         0.033607535 = weight(abstract_txt:document in 3085) [ClassicSimilarity], result of:
           0.033607535 = score(doc=3085,freq=2.0), product of:
             0.088545084 = queryWeight, product of:
               1.1943774 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.017264169 = queryNorm
             0.3795528 = fieldWeight in 3085, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=3085)
         0.19801879 = weight(abstract_txt:assessors in 3085) [ClassicSimilarity], result of:
           0.19801879 = score(doc=3085,freq=4.0), product of:
             0.18196629 = queryWeight, product of:
               1.2107092 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.017264169 = queryNorm
             1.0882169 = fieldWeight in 3085, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=3085)
         0.102020584 = weight(abstract_txt:test in 3085) [ClassicSimilarity], result of:
           0.102020584 = score(doc=3085,freq=7.0), product of:
             0.122266985 = queryWeight, product of:
               1.403505 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.017264169 = queryNorm
             0.8344083 = fieldWeight in 3085, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0625 = fieldNorm(doc=3085)
         0.09778675 = weight(abstract_txt:judgments in 3085) [ClassicSimilarity], result of:
           0.09778675 = score(doc=3085,freq=1.0), product of:
             0.22737184 = queryWeight, product of:
               1.9139372 = boost
               6.881186 = idf(docFreq=123, maxDocs=44421)
               0.017264169 = queryNorm
             0.43007413 = fieldWeight in 3085, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.881186 = idf(docFreq=123, maxDocs=44421)
               0.0625 = fieldNorm(doc=3085)
         0.09340261 = weight(abstract_txt:relevance in 3085) [ClassicSimilarity], result of:
           0.09340261 = score(doc=3085,freq=3.0), product of:
             0.17503008 = queryWeight, product of:
               2.0566545 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.017264169 = queryNorm
             0.53363746 = fieldWeight in 3085, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=3085)
       0.24 = coord(6/25)