Document (#30202)

Author

Maglaughlin, K.L.
Sonnenwald, D.H.

Title

User perspectives an relevance criteria : a comparison among relevant, partially relevant, and not-relevant judgements

Source

Journal of the American Society for Information Science and technology. 53(2002) no.5, S.327-342

Year

2002

Abstract

In this issue Maglaughin and Sonnenwald provided 12 graduate students with searches related to the student's work and asked them to judge the twenty most recent retrieved representations by highlighting passages thought to contribute to relevance, marking out passages detracting from relevance, and providing a relevant, partially relevant or relevant judgement on each. By recorded interview they were asked about how these decisions were made and to describe the three classes of judgement. The union of criteria identified in past studies did not seem to fully capture the information supplied so a new set was produced and coding agreement found to be adequate. Twenty-nine criteria were identified and grouped into six categories based upon the focus of the criterion. Multiple criteria are used for most judgements, and most criteria may have either a positive or negative effect. Content was the most frequently mentioned criterion.

Theme

Retrievalstudien
Benutzerstudien

Similar documents (author)

1. Sonnenwald, D.H.: Scientific collaboration (2007) 5.71

   5.7103243 = sum of:
     5.7103243 = weight(author_txt:sonnenwald in 3178) [ClassicSimilarity], result of:
       5.7103243 = fieldWeight in 3178, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.1365185 = idf(docFreq=12, maxDocs=44421)
         0.625 = fieldNorm(doc=3178)
   

2. Sonnenwald, D.H.: Evolving perspectives of human information behaviour : contexts, situations, social networks and information horizons (1999) 5.71

   5.7103243 = sum of:
     5.7103243 = weight(author_txt:sonnenwald in 1293) [ClassicSimilarity], result of:
       5.7103243 = fieldWeight in 1293, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.1365185 = idf(docFreq=12, maxDocs=44421)
         0.625 = fieldNorm(doc=1293)
   

3. Sonnenwald, D.H.; Lievrouw, L.A.: Collaboration during the design process : a case study of communication, information behaviour, and project performance (1997) 4.57

   4.5682592 = sum of:
     4.5682592 = weight(author_txt:sonnenwald in 1309) [ClassicSimilarity], result of:
       4.5682592 = fieldWeight in 1309, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.1365185 = idf(docFreq=12, maxDocs=44421)
         0.5 = fieldNorm(doc=1309)
   

4. Iivonen, M.; Sonnenwald, D.H.: Navigointi hekutermeja valittaessa (1997) 4.57

   4.5682592 = sum of:
     4.5682592 = weight(author_txt:sonnenwald in 2667) [ClassicSimilarity], result of:
       4.5682592 = fieldWeight in 2667, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.1365185 = idf(docFreq=12, maxDocs=44421)
         0.5 = fieldNorm(doc=2667)
   

5. Iivonen, M.; Sonnenwald, D.H.: From translation to navigation of different discourses : a model of search term selection during the pre-online stage of the search process (1998) 4.57

   4.5682592 = sum of:
     4.5682592 = weight(author_txt:sonnenwald in 3881) [ClassicSimilarity], result of:
       4.5682592 = fieldWeight in 3881, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.1365185 = idf(docFreq=12, maxDocs=44421)
         0.5 = fieldNorm(doc=3881)
   

Similar documents (content)

1. Tang, R.; Solomon, P.: Use of relevance criteria across stages of document evaluation : on the complementarity of experimental and naturalistic studies (2001) 0.27

   0.2655551 = sum of:
     0.2655551 = product of:
       0.8298598 = sum of:
         0.032724652 = weight(abstract_txt:were in 213) [ClassicSimilarity], result of:
           0.032724652 = score(doc=213,freq=3.0), product of:
             0.09420154 = queryWeight, product of:
               1.6394571 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.015667137 = queryNorm
             0.34738976 = fieldWeight in 213, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.060385093 = weight(abstract_txt:asked in 213) [ClassicSimilarity], result of:
           0.060385093 = score(doc=213,freq=1.0), product of:
             0.17855464 = queryWeight, product of:
               1.8429391 = boost
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.015667137 = queryNorm
             0.33818832 = fieldWeight in 213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.06488458 = weight(abstract_txt:relevance in 213) [ClassicSimilarity], result of:
           0.06488458 = score(doc=213,freq=2.0), product of:
             0.1701895 = queryWeight, product of:
               2.2036238 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.015667137 = queryNorm
             0.381249 = fieldWeight in 213, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.10451356 = weight(abstract_txt:judgements in 213) [ClassicSimilarity], result of:
           0.10451356 = score(doc=213,freq=1.0), product of:
             0.2573948 = queryWeight, product of:
               2.2127142 = boost
               7.4248013 = idf(docFreq=71, maxDocs=44421)
               0.015667137 = queryNorm
             0.40604383 = fieldWeight in 213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4248013 = idf(docFreq=71, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.117138825 = weight(abstract_txt:criterion in 213) [ClassicSimilarity], result of:
           0.117138825 = score(doc=213,freq=1.0), product of:
             0.27772728 = queryWeight, product of:
               2.298448 = boost
               7.7124834 = idf(docFreq=53, maxDocs=44421)
               0.015667137 = queryNorm
             0.42177644 = fieldWeight in 213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7124834 = idf(docFreq=53, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.054194007 = weight(abstract_txt:most in 213) [ClassicSimilarity], result of:
           0.054194007 = score(doc=213,freq=3.0), product of:
             0.1451293 = queryWeight, product of:
               2.34973 = boost
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.015667137 = queryNorm
             0.3734188 = fieldWeight in 213, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.1267202 = weight(abstract_txt:judgement in 213) [ClassicSimilarity], result of:
           0.1267202 = score(doc=213,freq=1.0), product of:
             0.2926725 = queryWeight, product of:
               2.3594804 = boost
               7.917278 = idf(docFreq=43, maxDocs=44421)
               0.015667137 = queryNorm
             0.43297613 = fieldWeight in 213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.917278 = idf(docFreq=43, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
         0.26929885 = weight(abstract_txt:criteria in 213) [ClassicSimilarity], result of:
           0.26929885 = score(doc=213,freq=6.0), product of:
             0.36132988 = queryWeight, product of:
               4.145218 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015667137 = queryNorm
             0.7452991 = fieldWeight in 213, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0546875 = fieldNorm(doc=213)
       0.32 = coord(8/25)
   

2. Taylor, A.; Zhang, X.; Amadio, W.J.: Examination of relevance criteria choices and the information search process (2009) 0.22

   0.21739489 = sum of:
     0.21739489 = product of:
       0.905812 = sum of:
         0.03454777 = weight(abstract_txt:identified in 595) [ClassicSimilarity], result of:
           0.03454777 = score(doc=595,freq=1.0), product of:
             0.11257364 = queryWeight, product of:
               1.463335 = boost
               4.9102464 = idf(docFreq=889, maxDocs=44421)
               0.015667137 = queryNorm
             0.3068904 = fieldWeight in 595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9102464 = idf(docFreq=889, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.04318534 = weight(abstract_txt:were in 595) [ClassicSimilarity], result of:
           0.04318534 = score(doc=595,freq=4.0), product of:
             0.09420154 = queryWeight, product of:
               1.6394571 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.015667137 = queryNorm
             0.4584356 = fieldWeight in 595, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.16019762 = weight(abstract_txt:partially in 595) [ClassicSimilarity], result of:
           0.16019762 = score(doc=595,freq=2.0), product of:
             0.24845672 = queryWeight, product of:
               2.1739562 = boost
               7.2947483 = idf(docFreq=81, maxDocs=44421)
               0.015667137 = queryNorm
             0.64477074 = fieldWeight in 595, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.2947483 = idf(docFreq=81, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.13872907 = weight(abstract_txt:relevance in 595) [ClassicSimilarity], result of:
           0.13872907 = score(doc=595,freq=7.0), product of:
             0.1701895 = queryWeight, product of:
               2.2036238 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.015667137 = queryNorm
             0.8151447 = fieldWeight in 595, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.17376992 = weight(abstract_txt:relevant in 595) [ClassicSimilarity], result of:
           0.17376992 = score(doc=595,freq=4.0), product of:
             0.30025727 = queryWeight, product of:
               4.139356 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.015667137 = queryNorm
             0.5787368 = fieldWeight in 595, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.3553823 = weight(abstract_txt:criteria in 595) [ClassicSimilarity], result of:
           0.3553823 = score(doc=595,freq=8.0), product of:
             0.36132988 = queryWeight, product of:
               4.145218 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015667137 = queryNorm
             0.9835397 = fieldWeight in 595, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
       0.24 = coord(6/25)
   

3. Crystal, A.; Greenberg, J.: Relevance criteria identified by health information users during Web searches (2006) 0.21

   0.208419 = sum of:
     0.208419 = product of:
       0.74435353 = sum of:
         0.052587133 = weight(abstract_txt:highlighting in 909) [ClassicSimilarity], result of:
           0.052587133 = score(doc=909,freq=1.0), product of:
             0.11823145 = queryWeight, product of:
               1.0604177 = boost
               7.1165 = idf(docFreq=97, maxDocs=44421)
               0.015667137 = queryNorm
             0.44478124 = fieldWeight in 909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1165 = idf(docFreq=97, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
         0.05983849 = weight(abstract_txt:identified in 909) [ClassicSimilarity], result of:
           0.05983849 = score(doc=909,freq=3.0), product of:
             0.11257364 = queryWeight, product of:
               1.463335 = boost
               4.9102464 = idf(docFreq=889, maxDocs=44421)
               0.015667137 = queryNorm
             0.53154975 = fieldWeight in 909, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.9102464 = idf(docFreq=889, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
         0.030536646 = weight(abstract_txt:were in 909) [ClassicSimilarity], result of:
           0.030536646 = score(doc=909,freq=2.0), product of:
             0.09420154 = queryWeight, product of:
               1.6394571 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.015667137 = queryNorm
             0.3241629 = fieldWeight in 909, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
         0.06901153 = weight(abstract_txt:asked in 909) [ClassicSimilarity], result of:
           0.06901153 = score(doc=909,freq=1.0), product of:
             0.17855464 = queryWeight, product of:
               1.8429391 = boost
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.015667137 = queryNorm
             0.38650092 = fieldWeight in 909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
         0.10486932 = weight(abstract_txt:relevance in 909) [ClassicSimilarity], result of:
           0.10486932 = score(doc=909,freq=4.0), product of:
             0.1701895 = queryWeight, product of:
               2.2036238 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.015667137 = queryNorm
             0.6161915 = fieldWeight in 909, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
         0.050570536 = weight(abstract_txt:most in 909) [ClassicSimilarity], result of:
           0.050570536 = score(doc=909,freq=2.0), product of:
             0.1451293 = queryWeight, product of:
               2.34973 = boost
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.015667137 = queryNorm
             0.3484516 = fieldWeight in 909, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
         0.37693986 = weight(abstract_txt:criteria in 909) [ClassicSimilarity], result of:
           0.37693986 = score(doc=909,freq=9.0), product of:
             0.36132988 = queryWeight, product of:
               4.145218 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015667137 = queryNorm
             1.0432014 = fieldWeight in 909, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0625 = fieldNorm(doc=909)
       0.28 = coord(7/25)
   

4. Janes, J.W.: ¬The binary nature of continous relevance judgements : a study of users' perceptions (1991) 0.17

   0.16565914 = sum of:
     0.16565914 = product of:
       0.69024646 = sum of:
         0.10861731 = weight(abstract_txt:judge in 4844) [ClassicSimilarity], result of:
           0.10861731 = score(doc=4844,freq=1.0), product of:
             0.14633624 = queryWeight, product of:
               1.1797402 = boost
               7.917278 = idf(docFreq=43, maxDocs=44421)
               0.015667137 = queryNorm
             0.7422448 = fieldWeight in 4844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.917278 = idf(docFreq=43, maxDocs=44421)
               0.09375 = fieldNorm(doc=4844)
         0.032389008 = weight(abstract_txt:were in 4844) [ClassicSimilarity], result of:
           0.032389008 = score(doc=4844,freq=1.0), product of:
             0.09420154 = queryWeight, product of:
               1.6394571 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.015667137 = queryNorm
             0.3438267 = fieldWeight in 4844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.09375 = fieldNorm(doc=4844)
         0.103517294 = weight(abstract_txt:asked in 4844) [ClassicSimilarity], result of:
           0.103517294 = score(doc=4844,freq=1.0), product of:
             0.17855464 = queryWeight, product of:
               1.8429391 = boost
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.015667137 = queryNorm
             0.5797514 = fieldWeight in 4844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.09375 = fieldNorm(doc=4844)
         0.13622923 = weight(abstract_txt:relevance in 4844) [ClassicSimilarity], result of:
           0.13622923 = score(doc=4844,freq=3.0), product of:
             0.1701895 = queryWeight, product of:
               2.2036238 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.015667137 = queryNorm
             0.80045617 = fieldWeight in 4844, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.09375 = fieldNorm(doc=4844)
         0.17916611 = weight(abstract_txt:judgements in 4844) [ClassicSimilarity], result of:
           0.17916611 = score(doc=4844,freq=1.0), product of:
             0.2573948 = queryWeight, product of:
               2.2127142 = boost
               7.4248013 = idf(docFreq=71, maxDocs=44421)
               0.015667137 = queryNorm
             0.69607514 = fieldWeight in 4844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4248013 = idf(docFreq=71, maxDocs=44421)
               0.09375 = fieldNorm(doc=4844)
         0.13032745 = weight(abstract_txt:relevant in 4844) [ClassicSimilarity], result of:
           0.13032745 = score(doc=4844,freq=1.0), product of:
             0.30025727 = queryWeight, product of:
               4.139356 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.015667137 = queryNorm
             0.4340526 = fieldWeight in 4844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.09375 = fieldNorm(doc=4844)
       0.24 = coord(6/25)
   

5. Choi, Y.; Rasmussen, E.M.: Users' relevance criteria in image retrieval in American history (2002) 0.16

   0.15922454 = sum of:
     0.15922454 = product of:
       0.6634356 = sum of:
         0.026719565 = weight(abstract_txt:were in 3592) [ClassicSimilarity], result of:
           0.026719565 = score(doc=3592,freq=2.0), product of:
             0.09420154 = queryWeight, product of:
               1.6394571 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.015667137 = queryNorm
             0.28364253 = fieldWeight in 3592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.12138794 = weight(abstract_txt:relevance in 3592) [ClassicSimilarity], result of:
           0.12138794 = score(doc=3592,freq=7.0), product of:
             0.1701895 = queryWeight, product of:
               2.2036238 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.015667137 = queryNorm
             0.71325165 = fieldWeight in 3592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.117138825 = weight(abstract_txt:criterion in 3592) [ClassicSimilarity], result of:
           0.117138825 = score(doc=3592,freq=1.0), product of:
             0.27772728 = queryWeight, product of:
               2.298448 = boost
               7.7124834 = idf(docFreq=53, maxDocs=44421)
               0.015667137 = queryNorm
             0.42177644 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7124834 = idf(docFreq=53, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.031288926 = weight(abstract_txt:most in 3592) [ClassicSimilarity], result of:
           0.031288926 = score(doc=3592,freq=1.0), product of:
             0.1451293 = queryWeight, product of:
               2.34973 = boost
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.015667137 = queryNorm
             0.21559344 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.07602434 = weight(abstract_txt:relevant in 3592) [ClassicSimilarity], result of:
           0.07602434 = score(doc=3592,freq=1.0), product of:
             0.30025727 = queryWeight, product of:
               4.139356 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.015667137 = queryNorm
             0.25319734 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.29087597 = weight(abstract_txt:criteria in 3592) [ClassicSimilarity], result of:
           0.29087597 = score(doc=3592,freq=7.0), product of:
             0.36132988 = queryWeight, product of:
               4.145218 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015667137 = queryNorm
             0.80501497 = fieldWeight in 3592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
       0.24 = coord(6/25)