Document (#20947)

Author

Burgin, R.

Title

¬The Monte Carlo method and the evaluation of retrieval system performance

Source

Journal of the American Society for Information Science. 50(1999) no.2, S.181-191

Year

1999

Abstract

The ability to distinguish between acceptable and unacceptable levels of retrieval performance and the ability to distinguish between significant and non-significant differences between retrieval result are important to traditional information retrieval experiments. The Monte Carlo method is shown to represent an attractive alternative to the hypergeometric model for identifying the levels at which random retrieval performance is exceeded in retrieval test collections and for overcoming some of the limitations of the hypergeometric model

Theme

Retrievalstudien

Similar documents (author)

1. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 6.01

   6.0137663 = sum of:
     6.0137663 = weight(author_txt:burgin in 3307) [ClassicSimilarity], result of:
       6.0137663 = fieldWeight in 3307, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.622026 = idf(docFreq=7, maxDocs=44421)
         0.625 = fieldNorm(doc=3307)
   

2. Burgin, R.: ¬The retrieval effectiveness of 5 clustering algorithms as a function of indexing exhaustivity (1995) 6.01

   6.0137663 = sum of:
     6.0137663 = weight(author_txt:burgin in 3433) [ClassicSimilarity], result of:
       6.0137663 = fieldWeight in 3433, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.622026 = idf(docFreq=7, maxDocs=44421)
         0.625 = fieldNorm(doc=3433)
   

3. Burgin, M.: Terminological and teleological systems (1996) 6.01

   6.0137663 = sum of:
     6.0137663 = weight(author_txt:burgin in 6391) [ClassicSimilarity], result of:
       6.0137663 = fieldWeight in 6391, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.622026 = idf(docFreq=7, maxDocs=44421)
         0.625 = fieldNorm(doc=6391)
   

4. Burgin, R.: ¬The effect of indexing exhaustivity on retrieval performance (1991) 6.01

   6.0137663 = sum of:
     6.0137663 = weight(author_txt:burgin in 6262) [ClassicSimilarity], result of:
       6.0137663 = fieldWeight in 6262, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.622026 = idf(docFreq=7, maxDocs=44421)
         0.625 = fieldNorm(doc=6262)
   

5. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 3.61

   3.60826 = sum of:
     3.60826 = weight(author_txt:burgin in 325) [ClassicSimilarity], result of:
       3.60826 = fieldWeight in 325, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.622026 = idf(docFreq=7, maxDocs=44421)
         0.375 = fieldNorm(doc=325)
   

Similar documents (content)

1. Hopkins, M.E.; Zavalina, O.L.: Evaluating physicians' serendipitous knowledge discovery in online discovery systems : a new approach (2019) 0.18

   0.18310839 = sum of:
     0.18310839 = product of:
       0.7629516 = sum of:
         0.021659678 = weight(abstract_txt:test in 842) [ClassicSimilarity], result of:
           0.021659678 = score(doc=842,freq=1.0), product of:
             0.06867875 = queryWeight, product of:
               1.0186007 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.013361919 = queryNorm
             0.3153767 = fieldWeight in 842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0625 = fieldNorm(doc=842)
         0.024254756 = weight(abstract_txt:limitations in 842) [ClassicSimilarity], result of:
           0.024254756 = score(doc=842,freq=1.0), product of:
             0.074060336 = queryWeight, product of:
               1.0577563 = boost
               5.2399993 = idf(docFreq=639, maxDocs=44421)
               0.013361919 = queryNorm
             0.32749996 = fieldWeight in 842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2399993 = idf(docFreq=639, maxDocs=44421)
               0.0625 = fieldNorm(doc=842)
         0.030123672 = weight(abstract_txt:model in 842) [ClassicSimilarity], result of:
           0.030123672 = score(doc=842,freq=2.0), product of:
             0.08557095 = queryWeight, product of:
               1.6079441 = boost
               3.9827821 = idf(docFreq=2249, maxDocs=44421)
               0.013361919 = queryNorm
             0.35203153 = fieldWeight in 842, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.9827821 = idf(docFreq=2249, maxDocs=44421)
               0.0625 = fieldNorm(doc=842)
         0.030699069 = weight(abstract_txt:method in 842) [ClassicSimilarity], result of:
           0.030699069 = score(doc=842,freq=1.0), product of:
             0.1091812 = queryWeight, product of:
               1.8162755 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.013361919 = queryNorm
             0.2811754 = fieldWeight in 842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=842)
         0.3281072 = weight(abstract_txt:monte in 842) [ClassicSimilarity], result of:
           0.3281072 = score(doc=842,freq=1.0), product of:
             0.5297548 = queryWeight, product of:
               4.0007854 = boost
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.013361919 = queryNorm
             0.61935675 = fieldWeight in 842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.0625 = fieldNorm(doc=842)
         0.3281072 = weight(abstract_txt:carlo in 842) [ClassicSimilarity], result of:
           0.3281072 = score(doc=842,freq=1.0), product of:
             0.5297548 = queryWeight, product of:
               4.0007854 = boost
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.013361919 = queryNorm
             0.61935675 = fieldWeight in 842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.0625 = fieldNorm(doc=842)
       0.24 = coord(6/25)
   

2. Almerri, J.; McGregor, D.R.: Codon signatures : a document retrieval method (1996) 0.16

   0.16208328 = sum of:
     0.16208328 = product of:
       0.57886887 = sum of:
         0.077988096 = weight(abstract_txt:acceptable in 39) [ClassicSimilarity], result of:
           0.077988096 = score(doc=39,freq=1.0), product of:
             0.13903792 = queryWeight, product of:
               1.4493037 = boost
               7.179679 = idf(docFreq=91, maxDocs=44421)
               0.013361919 = queryNorm
             0.56091243 = fieldWeight in 39, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.179679 = idf(docFreq=91, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
         0.06646544 = weight(abstract_txt:method in 39) [ClassicSimilarity], result of:
           0.06646544 = score(doc=39,freq=3.0), product of:
             0.1091812 = queryWeight, product of:
               1.8162755 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.013361919 = queryNorm
             0.6087626 = fieldWeight in 39, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
         0.036948517 = weight(abstract_txt:between in 39) [ClassicSimilarity], result of:
           0.036948517 = score(doc=39,freq=2.0), product of:
             0.09672591 = queryWeight, product of:
               2.09375 = boost
               3.4573963 = idf(docFreq=3804, maxDocs=44421)
               0.013361919 = queryNorm
             0.38199192 = fieldWeight in 39, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4573963 = idf(docFreq=3804, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
         0.07250009 = weight(abstract_txt:ability in 39) [ClassicSimilarity], result of:
           0.07250009 = score(doc=39,freq=1.0), product of:
             0.16685916 = queryWeight, product of:
               2.2453437 = boost
               5.561583 = idf(docFreq=463, maxDocs=44421)
               0.013361919 = queryNorm
             0.43449867 = fieldWeight in 39, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.561583 = idf(docFreq=463, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
         0.10795757 = weight(abstract_txt:performance in 39) [ClassicSimilarity], result of:
           0.10795757 = score(doc=39,freq=3.0), product of:
             0.17269634 = queryWeight, product of:
               2.7976604 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.013361919 = queryNorm
             0.62512946 = fieldWeight in 39, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
         0.14188035 = weight(abstract_txt:distinguish in 39) [ClassicSimilarity], result of:
           0.14188035 = score(doc=39,freq=1.0), product of:
             0.26105934 = queryWeight, product of:
               2.8085191 = boost
               6.9565353 = idf(docFreq=114, maxDocs=44421)
               0.013361919 = queryNorm
             0.5434793 = fieldWeight in 39, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9565353 = idf(docFreq=114, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
         0.075128764 = weight(abstract_txt:retrieval in 39) [ClassicSimilarity], result of:
           0.075128764 = score(doc=39,freq=2.0), product of:
             0.19559556 = queryWeight, product of:
               4.210638 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013361919 = queryNorm
             0.3841026 = fieldWeight in 39, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.078125 = fieldNorm(doc=39)
       0.28 = coord(7/25)
   

3. Schaalje, G.B.; Blades, N.J.; Funai, T.: ¬An open-set size-adjusted Bayesian classifier for authorship attribution (2013) 0.15

   0.15148489 = sum of:
     0.15148489 = product of:
       0.7574245 = sum of:
         0.03689381 = weight(abstract_txt:model in 2041) [ClassicSimilarity], result of:
           0.03689381 = score(doc=2041,freq=3.0), product of:
             0.08557095 = queryWeight, product of:
               1.6079441 = boost
               3.9827821 = idf(docFreq=2249, maxDocs=44421)
               0.013361919 = queryNorm
             0.4311488 = fieldWeight in 2041, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.9827821 = idf(docFreq=2249, maxDocs=44421)
               0.0625 = fieldNorm(doc=2041)
         0.04341504 = weight(abstract_txt:method in 2041) [ClassicSimilarity], result of:
           0.04341504 = score(doc=2041,freq=2.0), product of:
             0.1091812 = queryWeight, product of:
               1.8162755 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.013361919 = queryNorm
             0.39764208 = fieldWeight in 2041, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=2041)
         0.020901239 = weight(abstract_txt:between in 2041) [ClassicSimilarity], result of:
           0.020901239 = score(doc=2041,freq=1.0), product of:
             0.09672591 = queryWeight, product of:
               2.09375 = boost
               3.4573963 = idf(docFreq=3804, maxDocs=44421)
               0.013361919 = queryNorm
             0.21608727 = fieldWeight in 2041, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4573963 = idf(docFreq=3804, maxDocs=44421)
               0.0625 = fieldNorm(doc=2041)
         0.3281072 = weight(abstract_txt:monte in 2041) [ClassicSimilarity], result of:
           0.3281072 = score(doc=2041,freq=1.0), product of:
             0.5297548 = queryWeight, product of:
               4.0007854 = boost
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.013361919 = queryNorm
             0.61935675 = fieldWeight in 2041, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.0625 = fieldNorm(doc=2041)
         0.3281072 = weight(abstract_txt:carlo in 2041) [ClassicSimilarity], result of:
           0.3281072 = score(doc=2041,freq=1.0), product of:
             0.5297548 = queryWeight, product of:
               4.0007854 = boost
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.013361919 = queryNorm
             0.61935675 = fieldWeight in 2041, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.909708 = idf(docFreq=5, maxDocs=44421)
               0.0625 = fieldNorm(doc=2041)
       0.2 = coord(5/25)
   

4. Rubin, V.L.; Lukoianova, T.: Truth and deception at the rhetorical structure level (2015) 0.14

   0.13800281 = sum of:
     0.13800281 = product of:
       0.38334113 = sum of:
         0.020494627 = weight(abstract_txt:differences in 2816) [ClassicSimilarity], result of:
           0.020494627 = score(doc=2816,freq=1.0), product of:
             0.06619336 = queryWeight, product of:
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.013361919 = queryNorm
             0.30961758 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.021659678 = weight(abstract_txt:test in 2816) [ClassicSimilarity], result of:
           0.021659678 = score(doc=2816,freq=1.0), product of:
             0.06867875 = queryWeight, product of:
               1.0186007 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.013361919 = queryNorm
             0.3153767 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.021300653 = weight(abstract_txt:model in 2816) [ClassicSimilarity], result of:
           0.021300653 = score(doc=2816,freq=1.0), product of:
             0.08557095 = queryWeight, product of:
               1.6079441 = boost
               3.9827821 = idf(docFreq=2249, maxDocs=44421)
               0.013361919 = queryNorm
             0.24892388 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9827821 = idf(docFreq=2249, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.04341504 = weight(abstract_txt:method in 2816) [ClassicSimilarity], result of:
           0.04341504 = score(doc=2816,freq=2.0), product of:
             0.1091812 = queryWeight, product of:
               1.8162755 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.013361919 = queryNorm
             0.39764208 = fieldWeight in 2816, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.036280464 = weight(abstract_txt:significant in 2816) [ClassicSimilarity], result of:
           0.036280464 = score(doc=2816,freq=1.0), product of:
             0.12204302 = queryWeight, product of:
               1.9202787 = boost
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.013361919 = queryNorm
             0.29727602 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.020901239 = weight(abstract_txt:between in 2816) [ClassicSimilarity], result of:
           0.020901239 = score(doc=2816,freq=1.0), product of:
             0.09672591 = queryWeight, product of:
               2.09375 = boost
               3.4573963 = idf(docFreq=3804, maxDocs=44421)
               0.013361919 = queryNorm
             0.21608727 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4573963 = idf(docFreq=3804, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.047785074 = weight(abstract_txt:levels in 2816) [ClassicSimilarity], result of:
           0.047785074 = score(doc=2816,freq=1.0), product of:
             0.14664228 = queryWeight, product of:
               2.1049287 = boost
               5.2137837 = idf(docFreq=656, maxDocs=44421)
               0.013361919 = queryNorm
             0.32586148 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2137837 = idf(docFreq=656, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.05800007 = weight(abstract_txt:ability in 2816) [ClassicSimilarity], result of:
           0.05800007 = score(doc=2816,freq=1.0), product of:
             0.16685916 = queryWeight, product of:
               2.2453437 = boost
               5.561583 = idf(docFreq=463, maxDocs=44421)
               0.013361919 = queryNorm
             0.34759894 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.561583 = idf(docFreq=463, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
         0.11350428 = weight(abstract_txt:distinguish in 2816) [ClassicSimilarity], result of:
           0.11350428 = score(doc=2816,freq=1.0), product of:
             0.26105934 = queryWeight, product of:
               2.8085191 = boost
               6.9565353 = idf(docFreq=114, maxDocs=44421)
               0.013361919 = queryNorm
             0.43478346 = fieldWeight in 2816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9565353 = idf(docFreq=114, maxDocs=44421)
               0.0625 = fieldNorm(doc=2816)
       0.36 = coord(9/25)
   

5. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 0.13

   0.13089895 = sum of:
     0.13089895 = product of:
       0.5454123 = sum of:
         0.045947116 = weight(abstract_txt:test in 325) [ClassicSimilarity], result of:
           0.045947116 = score(doc=325,freq=2.0), product of:
             0.06867875 = queryWeight, product of:
               1.0186007 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.013361919 = queryNorm
             0.669015 = fieldWeight in 325, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.09375 = fieldNorm(doc=325)
         0.042422097 = weight(abstract_txt:represent in 325) [ClassicSimilarity], result of:
           0.042422097 = score(doc=325,freq=1.0), product of:
             0.08204554 = queryWeight, product of:
               1.1133207 = boost
               5.515259 = idf(docFreq=485, maxDocs=44421)
               0.013361919 = queryNorm
             0.5170555 = fieldWeight in 325, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.515259 = idf(docFreq=485, maxDocs=44421)
               0.09375 = fieldNorm(doc=325)
         0.0996197 = weight(abstract_txt:random in 325) [ClassicSimilarity], result of:
           0.0996197 = score(doc=325,freq=2.0), product of:
             0.11504835 = queryWeight, product of:
               1.3183569 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.013361919 = queryNorm
             0.8658942 = fieldWeight in 325, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.09375 = fieldNorm(doc=325)
         0.12414926 = weight(abstract_txt:levels in 325) [ClassicSimilarity], result of:
           0.12414926 = score(doc=325,freq=3.0), product of:
             0.14664228 = queryWeight, product of:
               2.1049287 = boost
               5.2137837 = idf(docFreq=656, maxDocs=44421)
               0.013361919 = queryNorm
             0.846613 = fieldWeight in 325, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.2137837 = idf(docFreq=656, maxDocs=44421)
               0.09375 = fieldNorm(doc=325)
         0.105776384 = weight(abstract_txt:performance in 325) [ClassicSimilarity], result of:
           0.105776384 = score(doc=325,freq=2.0), product of:
             0.17269634 = queryWeight, product of:
               2.7976604 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.013361919 = queryNorm
             0.6124993 = fieldWeight in 325, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.09375 = fieldNorm(doc=325)
         0.12749775 = weight(abstract_txt:retrieval in 325) [ClassicSimilarity], result of:
           0.12749775 = score(doc=325,freq=4.0), product of:
             0.19559556 = queryWeight, product of:
               4.210638 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.013361919 = queryNorm
             0.6518438 = fieldWeight in 325, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=325)
       0.24 = coord(6/25)