Document (#30214)

Author

Tang, R.
Solomon, P.

Title

Use of relevance criteria across stages of document evaluation : on the complementarity of experimental and naturalistic studies

Source

Journal of the American Society for Information Science and technology. 52(2001) no.8, S.676-687

Year

2001

Abstract

Tang and Solomon, based upon their review of the history of topical and other than topical criteria in relevance evaluation, decide to look at a two stage model where judgements are first made on surrogate records and then on full document text to determine if a criteria shift takes place and if so in what manner and to what degree. Both a controlled experiment and a naturalistic study were used to study the staging of relevance judgement criteria. In the controlled environment 90 undergraduate Psychology students were instructed to choose papers that would help them meet an assignment from 20 preselected papers on broader topic that included that assigned. They first selected on the basis of citation and abstract, then read the papers, and in each process filled out a questionnaire on the importance of each of 15 criteria at each stage of the two-stage process. In the naturalistic study 9 Ph.D. Psychology students conducted literature searches to support their own research and were asked to think aloud while making their decisions from retrieved surrogates, and later filled out a questionnaire while reading those materials that they selected and then interviewed at the end of the process. Apparently understandability is important at both stages. Importance increased at stage two. Cognitive criteria do not all follow the same pattern across stages. The controlled group thought quality of information was most important in stage one and topicality most important in stage 2. In the naturalistic study topicality was most frequent for stage one and research structure for stage two. A classification of criteria by their functionality is suggested as a better approach. First a division as to whether a criterion is objectively associated with the document as opposed to being subjectively associated with a person's expectations; then a division based on primary (essential) or secondary (for assistance) status.

Theme

Benutzerstudien

Similar documents (author)

1. Tang, R.; Solomon, P.: Toward an understanding of the dynamics of relevance judgement : an analysis of one person's search behavior (1998) 5.90

   5.902169 = sum of:
     5.902169 = sum of:
       2.5722544 = weight(author_txt:tang in 4268) [ClassicSimilarity], result of:
         2.5722544 = score(doc=4268,freq=1.0), product of:
           0.6440381 = queryWeight, product of:
             7.9878955 = idf(docFreq=40, maxDocs=44421)
             0.080626756 = queryNorm
           3.9939477 = fieldWeight in 4268, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.9878955 = idf(docFreq=40, maxDocs=44421)
             0.5 = fieldNorm(doc=4268)
       3.329915 = weight(author_txt:solomon in 4268) [ClassicSimilarity], result of:
         3.329915 = score(doc=4268,freq=1.0), product of:
           0.7649934 = queryWeight, product of:
             1.0898659 = boost
             8.705735 = idf(docFreq=19, maxDocs=44421)
             0.080626756 = queryNorm
           4.3528676 = fieldWeight in 4268, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.705735 = idf(docFreq=19, maxDocs=44421)
             0.5 = fieldNorm(doc=4268)
   

2. Solomon, P.: Children's information retrieval behavior : a case analysis of an OPAC (1993) 2.08

   2.0811968 = sum of:
     2.0811968 = product of:
       4.1623936 = sum of:
         4.1623936 = weight(author_txt:solomon in 4348) [ClassicSimilarity], result of:
           4.1623936 = score(doc=4348,freq=1.0), product of:
             0.7649934 = queryWeight, product of:
               1.0898659 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.080626756 = queryNorm
             5.4410844 = fieldWeight in 4348, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.625 = fieldNorm(doc=4348)
       0.5 = coord(1/2)
   

3. Solomon, P.: On the dynamics of information system use : from novice to? (1992) 2.08

   2.0811968 = sum of:
     2.0811968 = product of:
       4.1623936 = sum of:
         4.1623936 = weight(author_txt:solomon in 4518) [ClassicSimilarity], result of:
           4.1623936 = score(doc=4518,freq=1.0), product of:
             0.7649934 = queryWeight, product of:
               1.0898659 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.080626756 = queryNorm
             5.4410844 = fieldWeight in 4518, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.625 = fieldNorm(doc=4518)
       0.5 = coord(1/2)
   

4. Solomon, P.: Children, technology, and instruction : a case study of elementary school children using an online public access catalog (OPAC) (1994) 2.08

   2.0811968 = sum of:
     2.0811968 = product of:
       4.1623936 = sum of:
         4.1623936 = weight(author_txt:solomon in 1491) [ClassicSimilarity], result of:
           4.1623936 = score(doc=1491,freq=1.0), product of:
             0.7649934 = queryWeight, product of:
               1.0898659 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.080626756 = queryNorm
             5.4410844 = fieldWeight in 1491, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.625 = fieldNorm(doc=1491)
       0.5 = coord(1/2)
   

5. Solomon, P.: Dicovering information bahavior in sense making : I. time and timing (1997) 2.08

   2.0811968 = sum of:
     2.0811968 = product of:
       4.1623936 = sum of:
         4.1623936 = weight(author_txt:solomon in 957) [ClassicSimilarity], result of:
           4.1623936 = score(doc=957,freq=1.0), product of:
             0.7649934 = queryWeight, product of:
               1.0898659 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.080626756 = queryNorm
             5.4410844 = fieldWeight in 957, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.625 = fieldNorm(doc=957)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Wang, P.; White, M.D.: ¬A cognitive model of document use during a research project : Study II: Decisions at the reading and citing stages (1999) 0.35

   0.35291937 = sum of:
     0.35291937 = product of:
       0.80208945 = sum of:
         0.030689495 = weight(abstract_txt:selected in 3940) [ClassicSimilarity], result of:
           0.030689495 = score(doc=3940,freq=1.0), product of:
             0.09384777 = queryWeight, product of:
               1.0277847 = boost
               5.2322173 = idf(docFreq=644, maxDocs=44421)
               0.01745163 = queryNorm
             0.32701358 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2322173 = idf(docFreq=644, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.015853629 = weight(abstract_txt:were in 3940) [ClassicSimilarity], result of:
           0.015853629 = score(doc=3940,freq=1.0), product of:
             0.06916404 = queryWeight, product of:
               1.0806274 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.01745163 = queryNorm
             0.2292178 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.030168721 = weight(abstract_txt:process in 3940) [ClassicSimilarity], result of:
           0.030168721 = score(doc=3940,freq=2.0), product of:
             0.08429901 = queryWeight, product of:
               1.1930182 = boost
               4.048922 = idf(docFreq=2105, maxDocs=44421)
               0.01745163 = queryNorm
             0.35787752 = fieldWeight in 3940, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.048922 = idf(docFreq=2105, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.022438375 = weight(abstract_txt:each in 3940) [ClassicSimilarity], result of:
           0.022438375 = score(doc=3940,freq=1.0), product of:
             0.08718775 = queryWeight, product of:
               1.213287 = boost
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.01745163 = queryNorm
             0.25735697 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.02316868 = weight(abstract_txt:first in 3940) [ClassicSimilarity], result of:
           0.02316868 = score(doc=3940,freq=1.0), product of:
             0.08906944 = queryWeight, product of:
               1.2263097 = boost
               4.1619086 = idf(docFreq=1880, maxDocs=44421)
               0.01745163 = queryNorm
             0.2601193 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1619086 = idf(docFreq=1880, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.013423988 = weight(abstract_txt:their in 3940) [ClassicSimilarity], result of:
           0.013423988 = score(doc=3940,freq=1.0), product of:
             0.068133675 = queryWeight, product of:
               1.2384717 = boost
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.01745163 = queryNorm
             0.19702427 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.05690367 = weight(abstract_txt:document in 3940) [ClassicSimilarity], result of:
           0.05690367 = score(doc=3940,freq=5.0), product of:
             0.09481959 = queryWeight, product of:
               1.2652748 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.01745163 = queryNorm
             0.6001257 = fieldWeight in 3940, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.045160536 = weight(abstract_txt:study in 3940) [ClassicSimilarity], result of:
           0.045160536 = score(doc=3940,freq=7.0), product of:
             0.07996762 = queryWeight, product of:
               1.3417212 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01745163 = queryNorm
             0.5647353 = fieldWeight in 3940, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.08031472 = weight(abstract_txt:stages in 3940) [ClassicSimilarity], result of:
           0.08031472 = score(doc=3940,freq=1.0), product of:
             0.20401306 = queryWeight, product of:
               1.8559438 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.01745163 = queryNorm
             0.3936744 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.28879264 = weight(abstract_txt:criteria in 3940) [ClassicSimilarity], result of:
           0.28879264 = score(doc=3940,freq=5.0), product of:
             0.37141052 = queryWeight, product of:
               3.825175 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.01745163 = queryNorm
             0.7775564 = fieldWeight in 3940, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
         0.19517498 = weight(abstract_txt:stage in 3940) [ClassicSimilarity], result of:
           0.19517498 = score(doc=3940,freq=1.0), product of:
             0.511368 = queryWeight, product of:
               4.798295 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.01745163 = queryNorm
             0.38167226 = fieldWeight in 3940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.0625 = fieldNorm(doc=3940)
       0.44 = coord(11/25)
   

2. Taylor, A.; Zhang, X.; Amadio, W.J.: Examination of relevance criteria choices and the information search process (2009) 0.35

   0.35017595 = sum of:
     0.35017595 = product of:
       0.9727109 = sum of:
         0.02826723 = weight(abstract_txt:importance in 595) [ClassicSimilarity], result of:
           0.02826723 = score(doc=595,freq=1.0), product of:
             0.088842265 = queryWeight, product of:
               5.0907717 = idf(docFreq=742, maxDocs=44421)
               0.01745163 = queryNorm
             0.31817323 = fieldWeight in 595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0907717 = idf(docFreq=742, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.031707257 = weight(abstract_txt:were in 595) [ClassicSimilarity], result of:
           0.031707257 = score(doc=595,freq=4.0), product of:
             0.06916404 = queryWeight, product of:
               1.0806274 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.01745163 = queryNorm
             0.4584356 = fieldWeight in 595, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.04770094 = weight(abstract_txt:process in 595) [ClassicSimilarity], result of:
           0.04770094 = score(doc=595,freq=5.0), product of:
             0.08429901 = queryWeight, product of:
               1.1930182 = boost
               4.048922 = idf(docFreq=2105, maxDocs=44421)
               0.01745163 = queryNorm
             0.5658541 = fieldWeight in 595, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.048922 = idf(docFreq=2105, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.029564511 = weight(abstract_txt:study in 595) [ClassicSimilarity], result of:
           0.029564511 = score(doc=595,freq=3.0), product of:
             0.07996762 = queryWeight, product of:
               1.3417212 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01745163 = queryNorm
             0.36970603 = fieldWeight in 595, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.10185674 = weight(abstract_txt:relevance in 595) [ClassicSimilarity], result of:
           0.10185674 = score(doc=595,freq=7.0), product of:
             0.12495541 = queryWeight, product of:
               1.4524906 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01745163 = queryNorm
             0.8151447 = fieldWeight in 595, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.052067883 = weight(abstract_txt:controlled in 595) [ClassicSimilarity], result of:
           0.052067883 = score(doc=595,freq=1.0), product of:
             0.152818 = queryWeight, product of:
               1.6062866 = boost
               5.4514923 = idf(docFreq=517, maxDocs=44421)
               0.01745163 = queryNorm
             0.34071827 = fieldWeight in 595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4514923 = idf(docFreq=517, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.12107432 = weight(abstract_txt:topicality in 595) [ClassicSimilarity], result of:
           0.12107432 = score(doc=595,freq=1.0), product of:
             0.23431431 = queryWeight, product of:
               1.6240133 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.01745163 = queryNorm
             0.51671755 = fieldWeight in 595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.365297 = weight(abstract_txt:criteria in 595) [ClassicSimilarity], result of:
           0.365297 = score(doc=595,freq=8.0), product of:
             0.37141052 = queryWeight, product of:
               3.825175 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.01745163 = queryNorm
             0.9835397 = fieldWeight in 595, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
         0.19517498 = weight(abstract_txt:stage in 595) [ClassicSimilarity], result of:
           0.19517498 = score(doc=595,freq=1.0), product of:
             0.511368 = queryWeight, product of:
               4.798295 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.01745163 = queryNorm
             0.38167226 = fieldWeight in 595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.0625 = fieldNorm(doc=595)
       0.36 = coord(9/25)
   

3. Choi, Y.; Rasmussen, E.M.: Users' relevance criteria in image retrieval in American history (2002) 0.34

   0.33627155 = sum of:
     0.33627155 = product of:
       0.76425356 = sum of:
         0.024733827 = weight(abstract_txt:importance in 3592) [ClassicSimilarity], result of:
           0.024733827 = score(doc=3592,freq=1.0), product of:
             0.088842265 = queryWeight, product of:
               5.0907717 = idf(docFreq=742, maxDocs=44421)
               0.01745163 = queryNorm
             0.27840158 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0907717 = idf(docFreq=742, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.019617863 = weight(abstract_txt:were in 3592) [ClassicSimilarity], result of:
           0.019617863 = score(doc=3592,freq=2.0), product of:
             0.06916404 = queryWeight, product of:
               1.0806274 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.01745163 = queryNorm
             0.28364253 = fieldWeight in 3592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.01722956 = weight(abstract_txt:most in 3592) [ClassicSimilarity], result of:
           0.01722956 = score(doc=3592,freq=1.0), product of:
             0.0799169 = queryWeight, product of:
               1.1615961 = boost
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.01745163 = queryNorm
             0.21559344 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.019633578 = weight(abstract_txt:each in 3592) [ClassicSimilarity], result of:
           0.019633578 = score(doc=3592,freq=1.0), product of:
             0.08718775 = queryWeight, product of:
               1.213287 = boost
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.01745163 = queryNorm
             0.22518735 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1177115 = idf(docFreq=1965, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.011745989 = weight(abstract_txt:their in 3592) [ClassicSimilarity], result of:
           0.011745989 = score(doc=3592,freq=1.0), product of:
             0.068133675 = queryWeight, product of:
               1.2384717 = boost
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.01745163 = queryNorm
             0.17239624 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.029683193 = weight(abstract_txt:important in 3592) [ClassicSimilarity], result of:
           0.029683193 = score(doc=3592,freq=2.0), product of:
             0.09115621 = queryWeight, product of:
               1.2405919 = boost
               4.21038 = idf(docFreq=1791, maxDocs=44421)
               0.01745163 = queryNorm
             0.32562998 = fieldWeight in 3592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.21038 = idf(docFreq=1791, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.03339667 = weight(abstract_txt:study in 3592) [ClassicSimilarity], result of:
           0.03339667 = score(doc=3592,freq=5.0), product of:
             0.07996762 = queryWeight, product of:
               1.3417212 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01745163 = queryNorm
             0.4176274 = fieldWeight in 3592, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.08912465 = weight(abstract_txt:relevance in 3592) [ClassicSimilarity], result of:
           0.08912465 = score(doc=3592,freq=7.0), product of:
             0.12495541 = queryWeight, product of:
               1.4524906 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01745163 = queryNorm
             0.71325165 = fieldWeight in 3592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.14982182 = weight(abstract_txt:topicality in 3592) [ClassicSimilarity], result of:
           0.14982182 = score(doc=3592,freq=2.0), product of:
             0.23431431 = queryWeight, product of:
               1.6240133 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.01745163 = queryNorm
             0.6394053 = fieldWeight in 3592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.07027538 = weight(abstract_txt:stages in 3592) [ClassicSimilarity], result of:
           0.07027538 = score(doc=3592,freq=1.0), product of:
             0.20401306 = queryWeight, product of:
               1.8559438 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.01745163 = queryNorm
             0.3444651 = fieldWeight in 3592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
         0.29899102 = weight(abstract_txt:criteria in 3592) [ClassicSimilarity], result of:
           0.29899102 = score(doc=3592,freq=7.0), product of:
             0.37141052 = queryWeight, product of:
               3.825175 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.01745163 = queryNorm
             0.80501497 = fieldWeight in 3592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0546875 = fieldNorm(doc=3592)
       0.44 = coord(11/25)
   

4. Bateman, J.: Modelling the importance of end-user relevance criteria (1999) 0.31

   0.30793366 = sum of:
     0.30793366 = product of:
       0.76983416 = sum of:
         0.042840246 = weight(abstract_txt:importance in 606) [ClassicSimilarity], result of:
           0.042840246 = score(doc=606,freq=3.0), product of:
             0.088842265 = queryWeight, product of:
               5.0907717 = idf(docFreq=742, maxDocs=44421)
               0.01745163 = queryNorm
             0.4822057 = fieldWeight in 606, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.0907717 = idf(docFreq=742, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.026853308 = weight(abstract_txt:selected in 606) [ClassicSimilarity], result of:
           0.026853308 = score(doc=606,freq=1.0), product of:
             0.09384777 = queryWeight, product of:
               1.0277847 = boost
               5.2322173 = idf(docFreq=644, maxDocs=44421)
               0.01745163 = queryNorm
             0.2861369 = fieldWeight in 606, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2322173 = idf(docFreq=644, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.02774385 = weight(abstract_txt:were in 606) [ClassicSimilarity], result of:
           0.02774385 = score(doc=606,freq=4.0), product of:
             0.06916404 = queryWeight, product of:
               1.0806274 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.01745163 = queryNorm
             0.40113115 = fieldWeight in 606, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.03445912 = weight(abstract_txt:most in 606) [ClassicSimilarity], result of:
           0.03445912 = score(doc=606,freq=4.0), product of:
             0.0799169 = queryWeight, product of:
               1.1615961 = boost
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.01745163 = queryNorm
             0.43118688 = fieldWeight in 606, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.94228 = idf(docFreq=2342, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.023491979 = weight(abstract_txt:their in 606) [ClassicSimilarity], result of:
           0.023491979 = score(doc=606,freq=4.0), product of:
             0.068133675 = queryWeight, product of:
               1.2384717 = boost
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.01745163 = queryNorm
             0.3447925 = fieldWeight in 606, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.029683193 = weight(abstract_txt:important in 606) [ClassicSimilarity], result of:
           0.029683193 = score(doc=606,freq=2.0), product of:
             0.09115621 = queryWeight, product of:
               1.2405919 = boost
               4.21038 = idf(docFreq=1791, maxDocs=44421)
               0.01745163 = queryNorm
             0.32562998 = fieldWeight in 606, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.21038 = idf(docFreq=1791, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.014935444 = weight(abstract_txt:study in 606) [ClassicSimilarity], result of:
           0.014935444 = score(doc=606,freq=1.0), product of:
             0.07996762 = queryWeight, product of:
               1.3417212 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01745163 = queryNorm
             0.18676865 = fieldWeight in 606, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.10652434 = weight(abstract_txt:relevance in 606) [ClassicSimilarity], result of:
           0.10652434 = score(doc=606,freq=10.0), product of:
             0.12495541 = queryWeight, product of:
               1.4524906 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01745163 = queryNorm
             0.8524988 = fieldWeight in 606, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.10594003 = weight(abstract_txt:topicality in 606) [ClassicSimilarity], result of:
           0.10594003 = score(doc=606,freq=1.0), product of:
             0.23431431 = queryWeight, product of:
               1.6240133 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.01745163 = queryNorm
             0.45212787 = fieldWeight in 606, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
         0.35736266 = weight(abstract_txt:criteria in 606) [ClassicSimilarity], result of:
           0.35736266 = score(doc=606,freq=10.0), product of:
             0.37141052 = queryWeight, product of:
               3.825175 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.01745163 = queryNorm
             0.962177 = fieldWeight in 606, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0546875 = fieldNorm(doc=606)
       0.4 = coord(10/25)
   

5. Ng, K.B.; Kantor, P.B.; Strzalkowski, T.; Wacholder, N.; Tang, R.; Bai, B.; Rittman,; Song, P.; Sun, Y.: Automated judgment of document qualities (2006) 0.23

   0.23386528 = sum of:
     0.23386528 = product of:
       0.730829 = sum of:
         0.02802552 = weight(abstract_txt:were in 307) [ClassicSimilarity], result of:
           0.02802552 = score(doc=307,freq=2.0), product of:
             0.06916404 = queryWeight, product of:
               1.0806274 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.01745163 = queryNorm
             0.40520364 = fieldWeight in 307, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.016779985 = weight(abstract_txt:their in 307) [ClassicSimilarity], result of:
           0.016779985 = score(doc=307,freq=1.0), product of:
             0.068133675 = queryWeight, product of:
               1.2384717 = boost
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.01745163 = queryNorm
             0.24628034 = fieldWeight in 307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.029984554 = weight(abstract_txt:important in 307) [ClassicSimilarity], result of:
           0.029984554 = score(doc=307,freq=1.0), product of:
             0.09115621 = queryWeight, product of:
               1.2405919 = boost
               4.21038 = idf(docFreq=1791, maxDocs=44421)
               0.01745163 = queryNorm
             0.32893595 = fieldWeight in 307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.21038 = idf(docFreq=1791, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.06362024 = weight(abstract_txt:document in 307) [ClassicSimilarity], result of:
           0.06362024 = score(doc=307,freq=4.0), product of:
             0.09481959 = queryWeight, product of:
               1.2652748 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.01745163 = queryNorm
             0.6709609 = fieldWeight in 307, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.021336349 = weight(abstract_txt:study in 307) [ClassicSimilarity], result of:
           0.021336349 = score(doc=307,freq=1.0), product of:
             0.07996762 = queryWeight, product of:
               1.3417212 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01745163 = queryNorm
             0.26681235 = fieldWeight in 307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.048122786 = weight(abstract_txt:relevance in 307) [ClassicSimilarity], result of:
           0.048122786 = score(doc=307,freq=1.0), product of:
             0.12495541 = queryWeight, product of:
               1.4524906 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01745163 = queryNorm
             0.38511968 = fieldWeight in 307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.1003934 = weight(abstract_txt:stages in 307) [ClassicSimilarity], result of:
           0.1003934 = score(doc=307,freq=1.0), product of:
             0.20401306 = queryWeight, product of:
               1.8559438 = boost
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.01745163 = queryNorm
             0.492093 = fieldWeight in 307, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987905 = idf(docFreq=221, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
         0.42256618 = weight(abstract_txt:stage in 307) [ClassicSimilarity], result of:
           0.42256618 = score(doc=307,freq=3.0), product of:
             0.511368 = queryWeight, product of:
               4.798295 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.01745163 = queryNorm
             0.8263446 = fieldWeight in 307, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.078125 = fieldNorm(doc=307)
       0.32 = coord(8/25)