Document (#8321)

Author

Dyer, H.

Title

Beyond relevance, precision and recall : the 'worth' of information retrieval results

Source

Journal of document and text management. 2(1994) no.1, S.35-47

Year

1994

Abstract

The existing tools of relevance, recall and precision have their place in the evaluation of information retrieval systems, but do not tell the whole story. They cannot, for example, make allowance for the enquirer already knowing of the existence of a document or the subsequent availability of items. Utility theory goes part way towards meeting the need by assessing the likely usefulness of documents. Arguing that information is not information until it is used, the paper suggests an additional approach to information system evaluation - the 'use-value' of information

Theme

Information

Similar documents (author)

1. Dyer, S.: ¬The dark side of the moon : introducing IT to the school library (1993) 6.19

   6.1935673 = sum of:
     6.1935673 = weight(author_txt:dyer in 5536) [ClassicSimilarity], result of:
       6.1935673 = fieldWeight in 5536, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.909708 = idf(docFreq=5, maxDocs=44421)
         0.625 = fieldNorm(doc=5536)
   

2. Dyer, H.; Morris, A.: Human aspects of library automation (1990) 4.95

   4.954854 = sum of:
     4.954854 = weight(author_txt:dyer in 2177) [ClassicSimilarity], result of:
       4.954854 = fieldWeight in 2177, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.909708 = idf(docFreq=5, maxDocs=44421)
         0.5 = fieldNorm(doc=2177)
   

3. Dyer, H.; Bouchet, M.-L.: ¬A comparison between the perceived value of information retrieved via end-user searching of CD-ROMs and mediated online searching (1995) 4.34

   4.3354974 = sum of:
     4.3354974 = weight(author_txt:dyer in 2485) [ClassicSimilarity], result of:
       4.3354974 = fieldWeight in 2485, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.909708 = idf(docFreq=5, maxDocs=44421)
         0.4375 = fieldNorm(doc=2485)
   

4. Dyer, H. u. A. Gunson (Comp.): ¬A directory of library and information retrieval software for microcomputers (1990) 3.72

   3.7161405 = sum of:
     3.7161405 = weight(author_txt:dyer in 1816) [ClassicSimilarity], result of:
       3.7161405 = fieldWeight in 1816, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.909708 = idf(docFreq=5, maxDocs=44421)
         0.375 = fieldNorm(doc=1816)
   

Similar documents (content)

1. Lesk, M.E.; Salton, G.: Relevance assements and retrieval system evaluation (1969) 0.24

   0.23672156 = sum of:
     0.23672156 = product of:
       0.8454341 = sum of:
         0.07991517 = weight(abstract_txt:cannot in 4219) [ClassicSimilarity], result of:
           0.07991517 = score(doc=4219,freq=1.0), product of:
             0.14313865 = queryWeight, product of:
               1.037419 = boost
               5.9552646 = idf(docFreq=312, maxDocs=44421)
               0.0231687 = queryNorm
             0.55830604 = fieldWeight in 4219, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9552646 = idf(docFreq=312, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
         0.031796753 = weight(abstract_txt:retrieval in 4219) [ClassicSimilarity], result of:
           0.031796753 = score(doc=4219,freq=1.0), product of:
             0.09755943 = queryWeight, product of:
               1.2112265 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0231687 = queryNorm
             0.3259219 = fieldWeight in 4219, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
         0.06802727 = weight(abstract_txt:evaluation in 4219) [ClassicSimilarity], result of:
           0.06802727 = score(doc=4219,freq=1.0), product of:
             0.161983 = queryWeight, product of:
               1.5607216 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0231687 = queryNorm
             0.4199655 = fieldWeight in 4219, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
         0.09065111 = weight(abstract_txt:relevance in 4219) [ClassicSimilarity], result of:
           0.09065111 = score(doc=4219,freq=1.0), product of:
             0.19615354 = queryWeight, product of:
               1.7174686 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0231687 = queryNorm
             0.46214363 = fieldWeight in 4219, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
         0.25505546 = weight(abstract_txt:precision in 4219) [ClassicSimilarity], result of:
           0.25505546 = score(doc=4219,freq=4.0), product of:
             0.2462732 = queryWeight, product of:
               1.9244176 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0231687 = queryNorm
             1.0356606 = fieldWeight in 4219, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
         0.28785682 = weight(abstract_txt:recall in 4219) [ClassicSimilarity], result of:
           0.28785682 = score(doc=4219,freq=4.0), product of:
             0.2669593 = queryWeight, product of:
               2.0036104 = boost
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0231687 = queryNorm
             1.0782797 = fieldWeight in 4219, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
         0.032131486 = weight(abstract_txt:information in 4219) [ClassicSimilarity], result of:
           0.032131486 = score(doc=4219,freq=1.0), product of:
             0.14169082 = queryWeight, product of:
               2.5282629 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0231687 = queryNorm
             0.22677183 = fieldWeight in 4219, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.09375 = fieldNorm(doc=4219)
       0.28 = coord(7/25)
   

2. Wallis, P.; Thom, J.A.: Relevance judgements for assessing recall (1996) 0.19

   0.19110903 = sum of:
     0.19110903 = product of:
       0.79628766 = sum of:
         0.18588592 = weight(abstract_txt:assessing in 5017) [ClassicSimilarity], result of:
           0.18588592 = score(doc=5017,freq=2.0), product of:
             0.17996742 = queryWeight, product of:
               1.163249 = boost
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.0231687 = queryNorm
             1.0328865 = fieldWeight in 5017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.109375 = fieldNorm(doc=5017)
         0.037096214 = weight(abstract_txt:retrieval in 5017) [ClassicSimilarity], result of:
           0.037096214 = score(doc=5017,freq=1.0), product of:
             0.09755943 = queryWeight, product of:
               1.2112265 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0231687 = queryNorm
             0.3802422 = fieldWeight in 5017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.109375 = fieldNorm(doc=5017)
         0.14956668 = weight(abstract_txt:relevance in 5017) [ClassicSimilarity], result of:
           0.14956668 = score(doc=5017,freq=2.0), product of:
             0.19615354 = queryWeight, product of:
               1.7174686 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0231687 = queryNorm
             0.762498 = fieldWeight in 5017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.109375 = fieldNorm(doc=5017)
         0.14878236 = weight(abstract_txt:precision in 5017) [ClassicSimilarity], result of:
           0.14878236 = score(doc=5017,freq=1.0), product of:
             0.2462732 = queryWeight, product of:
               1.9244176 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0231687 = queryNorm
             0.6041354 = fieldWeight in 5017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.109375 = fieldNorm(doc=5017)
         0.23746973 = weight(abstract_txt:recall in 5017) [ClassicSimilarity], result of:
           0.23746973 = score(doc=5017,freq=2.0), product of:
             0.2669593 = queryWeight, product of:
               2.0036104 = boost
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0231687 = queryNorm
             0.8895353 = fieldWeight in 5017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.109375 = fieldNorm(doc=5017)
         0.037486736 = weight(abstract_txt:information in 5017) [ClassicSimilarity], result of:
           0.037486736 = score(doc=5017,freq=1.0), product of:
             0.14169082 = queryWeight, product of:
               2.5282629 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0231687 = queryNorm
             0.26456714 = fieldWeight in 5017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.109375 = fieldNorm(doc=5017)
       0.24 = coord(6/25)
   

3. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 0.16

   0.15774177 = sum of:
     0.15774177 = product of:
       0.6572574 = sum of:
         0.06359351 = weight(abstract_txt:retrieval in 3307) [ClassicSimilarity], result of:
           0.06359351 = score(doc=3307,freq=4.0), product of:
             0.09755943 = queryWeight, product of:
               1.2112265 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0231687 = queryNorm
             0.6518438 = fieldWeight in 3307, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=3307)
         0.06802727 = weight(abstract_txt:evaluation in 3307) [ClassicSimilarity], result of:
           0.06802727 = score(doc=3307,freq=1.0), product of:
             0.161983 = queryWeight, product of:
               1.5607216 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0231687 = queryNorm
             0.4199655 = fieldWeight in 3307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=3307)
         0.22204898 = weight(abstract_txt:relevance in 3307) [ClassicSimilarity], result of:
           0.22204898 = score(doc=3307,freq=6.0), product of:
             0.19615354 = queryWeight, product of:
               1.7174686 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0231687 = queryNorm
             1.1320162 = fieldWeight in 3307, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.09375 = fieldNorm(doc=3307)
         0.12752773 = weight(abstract_txt:precision in 3307) [ClassicSimilarity], result of:
           0.12752773 = score(doc=3307,freq=1.0), product of:
             0.2462732 = queryWeight, product of:
               1.9244176 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0231687 = queryNorm
             0.5178303 = fieldWeight in 3307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.09375 = fieldNorm(doc=3307)
         0.14392841 = weight(abstract_txt:recall in 3307) [ClassicSimilarity], result of:
           0.14392841 = score(doc=3307,freq=1.0), product of:
             0.2669593 = queryWeight, product of:
               2.0036104 = boost
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0231687 = queryNorm
             0.53913987 = fieldWeight in 3307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.09375 = fieldNorm(doc=3307)
         0.032131486 = weight(abstract_txt:information in 3307) [ClassicSimilarity], result of:
           0.032131486 = score(doc=3307,freq=1.0), product of:
             0.14169082 = queryWeight, product of:
               2.5282629 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0231687 = queryNorm
             0.22677183 = fieldWeight in 3307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.09375 = fieldNorm(doc=3307)
       0.24 = coord(6/25)
   

4. Spink, A.; Greisdorf, H.: Regions and levels : Measuring and mapping users' relevance judgements (2001) 0.15

   0.15325072 = sum of:
     0.15325072 = product of:
       0.547324 = sum of:
         0.055831704 = weight(abstract_txt:usefulness in 6586) [ClassicSimilarity], result of:
           0.055831704 = score(doc=6586,freq=1.0), product of:
             0.14767903 = queryWeight, product of:
               1.0537441 = boost
               6.0489783 = idf(docFreq=284, maxDocs=44421)
               0.0231687 = queryNorm
             0.37806115 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0489783 = idf(docFreq=284, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.064721756 = weight(abstract_txt:utility in 6586) [ClassicSimilarity], result of:
           0.064721756 = score(doc=6586,freq=1.0), product of:
             0.16296653 = queryWeight, product of:
               1.1069422 = boost
               6.35436 = idf(docFreq=209, maxDocs=44421)
               0.0231687 = queryNorm
             0.3971475 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.35436 = idf(docFreq=209, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.021197835 = weight(abstract_txt:retrieval in 6586) [ClassicSimilarity], result of:
           0.021197835 = score(doc=6586,freq=1.0), product of:
             0.09755943 = queryWeight, product of:
               1.2112265 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0231687 = queryNorm
             0.21728125 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.06413672 = weight(abstract_txt:evaluation in 6586) [ClassicSimilarity], result of:
           0.06413672 = score(doc=6586,freq=2.0), product of:
             0.161983 = queryWeight, product of:
               1.5607216 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0231687 = queryNorm
             0.39594725 = fieldWeight in 6586, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.22612359 = weight(abstract_txt:relevance in 6586) [ClassicSimilarity], result of:
           0.22612359 = score(doc=6586,freq=14.0), product of:
             0.19615354 = queryWeight, product of:
               1.7174686 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0231687 = queryNorm
             1.1527888 = fieldWeight in 6586, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.085018486 = weight(abstract_txt:precision in 6586) [ClassicSimilarity], result of:
           0.085018486 = score(doc=6586,freq=1.0), product of:
             0.2462732 = queryWeight, product of:
               1.9244176 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0231687 = queryNorm
             0.3452202 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.030293856 = weight(abstract_txt:information in 6586) [ClassicSimilarity], result of:
           0.030293856 = score(doc=6586,freq=2.0), product of:
             0.14169082 = queryWeight, product of:
               2.5282629 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0231687 = queryNorm
             0.21380253 = fieldWeight in 6586, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
       0.28 = coord(7/25)
   

5. Vakkari, P.; Huuskonen, S.: Search effort degrades search output but improves task outcome (2012) 0.14

   0.14349054 = sum of:
     0.14349054 = product of:
       0.5124662 = sum of:
         0.055831704 = weight(abstract_txt:usefulness in 1046) [ClassicSimilarity], result of:
           0.055831704 = score(doc=1046,freq=1.0), product of:
             0.14767903 = queryWeight, product of:
               1.0537441 = boost
               6.0489783 = idf(docFreq=284, maxDocs=44421)
               0.0231687 = queryNorm
             0.37806115 = fieldWeight in 1046, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0489783 = idf(docFreq=284, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
         0.07510926 = weight(abstract_txt:assessing in 1046) [ClassicSimilarity], result of:
           0.07510926 = score(doc=1046,freq=1.0), product of:
             0.17996742 = queryWeight, product of:
               1.163249 = boost
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.0231687 = queryNorm
             0.4173492 = fieldWeight in 1046, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
         0.029978266 = weight(abstract_txt:retrieval in 1046) [ClassicSimilarity], result of:
           0.029978266 = score(doc=1046,freq=2.0), product of:
             0.09755943 = queryWeight, product of:
               1.2112265 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0231687 = queryNorm
             0.3072821 = fieldWeight in 1046, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
         0.06413672 = weight(abstract_txt:evaluation in 1046) [ClassicSimilarity], result of:
           0.06413672 = score(doc=1046,freq=2.0), product of:
             0.161983 = queryWeight, product of:
               1.5607216 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0231687 = queryNorm
             0.39594725 = fieldWeight in 1046, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
         0.17003697 = weight(abstract_txt:precision in 1046) [ClassicSimilarity], result of:
           0.17003697 = score(doc=1046,freq=4.0), product of:
             0.2462732 = queryWeight, product of:
               1.9244176 = boost
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0231687 = queryNorm
             0.6904404 = fieldWeight in 1046, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5235233 = idf(docFreq=481, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
         0.095952265 = weight(abstract_txt:recall in 1046) [ClassicSimilarity], result of:
           0.095952265 = score(doc=1046,freq=1.0), product of:
             0.2669593 = queryWeight, product of:
               2.0036104 = boost
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0231687 = queryNorm
             0.35942656 = fieldWeight in 1046, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.750825 = idf(docFreq=383, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
         0.021420991 = weight(abstract_txt:information in 1046) [ClassicSimilarity], result of:
           0.021420991 = score(doc=1046,freq=1.0), product of:
             0.14169082 = queryWeight, product of:
               2.5282629 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0231687 = queryNorm
             0.15118122 = fieldWeight in 1046, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=1046)
       0.28 = coord(7/25)