Document (#27118)

Author

Su, L.T.

Title

¬A comprehensive and systematic model of user evaluation of Web search engines : Il. An evaluation by undergraduates

Source

Journal of the American Society for Information Science and technology. 54(2003) no.13, S.1193-1222

Year

2003

Abstract

This paper presents an application of the model described in Part I to the evaluation of Web search engines by undergraduates. The study observed how 36 undergraduate used four major search engines to find information for their own individual problems and how they evaluated these engines based an actual interaction with the search engines. User evaluation was based an 16 performance measures representing five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Non-performance (user-related) measures were also applied. Each participant searched his/ her own topic an all four engines and provided satisfaction ratings for system features and interaction and reasons for satisfaction. Each also made relevance judgements of retrieved items in relation to his/her own information need and participated in post-search Interviews to provide reactions to the search results and overall performance. The study found significant differences in precision PR1 relative recall, user satisfaction with output display, time saving, value of search results, and overall performance among the four engines and also significant engine by discipline interactions an all these measures. In addition, the study found significant differences in user satisfaction with response time among four engines, and significant engine by discipline interaction in user satisfaction with search interface. None of the four search engines dominated in every aspect of the multidimensional evaluation. Content analysis of verbal data identified a number of user criteria and users evaluative comments based an these criteria. Results from both quantitative analysis and content analysis provide insight for system design and development, and useful feedback an strengths and weaknesses of search engines for system improvement

Footnote

Teil I im selben Heft

Theme

Suchmaschinen
Benutzerstudien

Similar documents (content)

1. Su, L.T.; Chen, H.L.: Evaluation of Web search engines by undergraduate students (1999) 1.24

   1.2438774 = sum of:
     1.2438774 = product of:
       1.5548468 = sum of:
         0.014124917 = weight(abstract_txt:these in 546) [ClassicSimilarity], result of:
           0.014124917 = score(doc=546,freq=2.0), product of:
             0.057380352 = queryWeight, product of:
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.018027835 = queryNorm
             0.24616295 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.014127578 = weight(abstract_txt:based in 546) [ClassicSimilarity], result of:
           0.014127578 = score(doc=546,freq=2.0), product of:
             0.05738756 = queryWeight, product of:
               1.0000628 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.018027835 = queryNorm
             0.24617839 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.04284483 = weight(abstract_txt:relevance in 546) [ClassicSimilarity], result of:
           0.04284483 = score(doc=546,freq=3.0), product of:
             0.09175802 = queryWeight, product of:
               1.0325112 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.018027835 = queryNorm
             0.46693277 = fieldWeight in 546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.025104845 = weight(abstract_txt:differences in 546) [ClassicSimilarity], result of:
           0.025104845 = score(doc=546,freq=1.0), product of:
             0.09266674 = queryWeight, product of:
               1.0376114 = boost
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.018027835 = queryNorm
             0.2709154 = fieldWeight in 546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.016806956 = weight(abstract_txt:system in 546) [ClassicSimilarity], result of:
           0.016806956 = score(doc=546,freq=2.0), product of:
             0.06443156 = queryWeight, product of:
               1.0596629 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.018027835 = queryNorm
             0.26084974 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.017449167 = weight(abstract_txt:study in 546) [ClassicSimilarity], result of:
           0.017449167 = score(doc=546,freq=2.0), product of:
             0.066062614 = queryWeight, product of:
               1.0729915 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.018027835 = queryNorm
             0.26413074 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.018439857 = weight(abstract_txt:results in 546) [ClassicSimilarity], result of:
           0.018439857 = score(doc=546,freq=2.0), product of:
             0.06854004 = queryWeight, product of:
               1.0929255 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.018027835 = queryNorm
             0.26903775 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.033888835 = weight(abstract_txt:overall in 546) [ClassicSimilarity], result of:
           0.033888835 = score(doc=546,freq=1.0), product of:
             0.11318528 = queryWeight, product of:
               1.1467474 = boost
               5.474931 = idf(docFreq=505, maxDocs=44421)
               0.018027835 = queryNorm
             0.29941028 = fieldWeight in 546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.474931 = idf(docFreq=505, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.048938185 = weight(abstract_txt:engine in 546) [ClassicSimilarity], result of:
           0.048938185 = score(doc=546,freq=2.0), product of:
             0.11477327 = queryWeight, product of:
               1.1547638 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.018027835 = queryNorm
             0.42639008 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.044686206 = weight(abstract_txt:interaction in 546) [ClassicSimilarity], result of:
           0.044686206 = score(doc=546,freq=1.0), product of:
             0.15579909 = queryWeight, product of:
               1.6477863 = boost
               5.244698 = idf(docFreq=636, maxDocs=44421)
               0.018027835 = queryNorm
             0.28681943 = fieldWeight in 546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.244698 = idf(docFreq=636, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.08565045 = weight(abstract_txt:measures in 546) [ClassicSimilarity], result of:
           0.08565045 = score(doc=546,freq=3.0), product of:
             0.16668452 = queryWeight, product of:
               1.7043785 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.018027835 = queryNorm
             0.51384765 = fieldWeight in 546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.053347327 = weight(abstract_txt:criteria in 546) [ClassicSimilarity], result of:
           0.053347327 = score(doc=546,freq=1.0), product of:
             0.1753306 = queryWeight, product of:
               1.7480235 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.018027835 = queryNorm
             0.30426708 = fieldWeight in 546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.057586905 = weight(abstract_txt:performance in 546) [ClassicSimilarity], result of:
           0.057586905 = score(doc=546,freq=2.0), product of:
             0.16117637 = queryWeight, product of:
               1.9352561 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.018027835 = queryNorm
             0.35729125 = fieldWeight in 546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.044441666 = weight(abstract_txt:significant in 546) [ClassicSimilarity], result of:
           0.044441666 = score(doc=546,freq=1.0), product of:
             0.17085291 = queryWeight, product of:
               1.9925029 = boost
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.018027835 = queryNorm
             0.26011652 = fieldWeight in 546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.14394812 = weight(abstract_txt:four in 546) [ClassicSimilarity], result of:
           0.14394812 = score(doc=546,freq=4.0), product of:
             0.25381437 = queryWeight, product of:
               2.715194 = boost
               5.1852746 = idf(docFreq=675, maxDocs=44421)
               0.018027835 = queryNorm
             0.5671394 = fieldWeight in 546, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.1852746 = idf(docFreq=675, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.09645609 = weight(abstract_txt:evaluation in 546) [ClassicSimilarity], result of:
           0.09645609 = score(doc=546,freq=3.0), product of:
             0.2273205 = queryWeight, product of:
               2.8148334 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.018027835 = queryNorm
             0.42431757 = fieldWeight in 546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.041193776 = weight(abstract_txt:user in 546) [ClassicSimilarity], result of:
           0.041193776 = score(doc=546,freq=1.0), product of:
             0.20464128 = queryWeight, product of:
               3.0838938 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.018027835 = queryNorm
             0.20129749 = fieldWeight in 546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.08729115 = weight(abstract_txt:search in 546) [ClassicSimilarity], result of:
           0.08729115 = score(doc=546,freq=3.0), product of:
             0.25216332 = queryWeight, product of:
               3.8273547 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.018027835 = queryNorm
             0.3461691 = fieldWeight in 546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.3268108 = weight(abstract_txt:satisfaction in 546) [ClassicSimilarity], result of:
           0.3268108 = score(doc=546,freq=4.0), product of:
             0.46591163 = queryWeight, product of:
               4.0298157 = boost
               6.4132004 = idf(docFreq=197, maxDocs=44421)
               0.018027835 = queryNorm
             0.7014438 = fieldWeight in 546, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.4132004 = idf(docFreq=197, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
         0.34170905 = weight(abstract_txt:engines in 546) [ClassicSimilarity], result of:
           0.34170905 = score(doc=546,freq=5.0), product of:
             0.52827066 = queryWeight, product of:
               5.539697 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.018027835 = queryNorm
             0.6468446 = fieldWeight in 546, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0546875 = fieldNorm(doc=546)
       0.8 = coord(20/25)
   

2. Gluck, M.: Exploring the relationship between user satisfaction and relevance in information systems (1996) 0.66

   0.6570743 = sum of:
     0.6570743 = product of:
       1.0951238 = sum of:
         0.011414657 = weight(abstract_txt:these in 4150) [ClassicSimilarity], result of:
           0.011414657 = score(doc=4150,freq=1.0), product of:
             0.057380352 = queryWeight, product of:
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.018027835 = queryNorm
             0.19892971 = fieldWeight in 4150, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.016145803 = weight(abstract_txt:based in 4150) [ClassicSimilarity], result of:
           0.016145803 = score(doc=4150,freq=2.0), product of:
             0.05738756 = queryWeight, product of:
               1.0000628 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.018027835 = queryNorm
             0.28134674 = fieldWeight in 4150, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.074796066 = weight(abstract_txt:relevance in 4150) [ClassicSimilarity], result of:
           0.074796066 = score(doc=4150,freq=7.0), product of:
             0.09175802 = queryWeight, product of:
               1.0325112 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.018027835 = queryNorm
             0.8151447 = fieldWeight in 4150, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.010381602 = weight(abstract_txt:with in 4150) [ClassicSimilarity], result of:
           0.010381602 = score(doc=4150,freq=2.0), product of:
             0.047054425 = queryWeight, product of:
               1.045654 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.018027835 = queryNorm
             0.22062966 = fieldWeight in 4150, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.033269152 = weight(abstract_txt:system in 4150) [ClassicSimilarity], result of:
           0.033269152 = score(doc=4150,freq=6.0), product of:
             0.06443156 = queryWeight, product of:
               1.0596629 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.018027835 = queryNorm
             0.51634866 = fieldWeight in 4150, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.013852289 = weight(abstract_txt:also in 4150) [ClassicSimilarity], result of:
           0.013852289 = score(doc=4150,freq=1.0), product of:
             0.065283336 = queryWeight, product of:
               1.0666442 = boost
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.018027835 = queryNorm
             0.21218722 = fieldWeight in 4150, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.014101057 = weight(abstract_txt:study in 4150) [ClassicSimilarity], result of:
           0.014101057 = score(doc=4150,freq=1.0), product of:
             0.066062614 = queryWeight, product of:
               1.0729915 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.018027835 = queryNorm
             0.21344988 = fieldWeight in 4150, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.014901656 = weight(abstract_txt:results in 4150) [ClassicSimilarity], result of:
           0.014901656 = score(doc=4150,freq=1.0), product of:
             0.06854004 = queryWeight, product of:
               1.0929255 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.018027835 = queryNorm
             0.21741535 = fieldWeight in 4150, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.034305826 = weight(abstract_txt:analysis in 4150) [ClassicSimilarity], result of:
           0.034305826 = score(doc=4150,freq=4.0), product of:
             0.07528005 = queryWeight, product of:
               1.145403 = boost
               3.6456752 = idf(docFreq=3151, maxDocs=44421)
               0.018027835 = queryNorm
             0.4557094 = fieldWeight in 4150, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6456752 = idf(docFreq=3151, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.054772627 = weight(abstract_txt:overall in 4150) [ClassicSimilarity], result of:
           0.054772627 = score(doc=4150,freq=2.0), product of:
             0.11318528 = queryWeight, product of:
               1.1467474 = boost
               5.474931 = idf(docFreq=505, maxDocs=44421)
               0.018027835 = queryNorm
             0.48392007 = fieldWeight in 4150, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.474931 = idf(docFreq=505, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.11302928 = weight(abstract_txt:measures in 4150) [ClassicSimilarity], result of:
           0.11302928 = score(doc=4150,freq=4.0), product of:
             0.16668452 = queryWeight, product of:
               1.7043785 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.018027835 = queryNorm
             0.678103 = fieldWeight in 4150, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.08060488 = weight(abstract_txt:performance in 4150) [ClassicSimilarity], result of:
           0.08060488 = score(doc=4150,freq=3.0), product of:
             0.16117637 = queryWeight, product of:
               1.9352561 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.018027835 = queryNorm
             0.5001036 = fieldWeight in 4150, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.050790474 = weight(abstract_txt:significant in 4150) [ClassicSimilarity], result of:
           0.050790474 = score(doc=4150,freq=1.0), product of:
             0.17085291 = queryWeight, product of:
               1.9925029 = boost
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.018027835 = queryNorm
             0.29727602 = fieldWeight in 4150, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.11531856 = weight(abstract_txt:user in 4150) [ClassicSimilarity], result of:
           0.11531856 = score(doc=4150,freq=6.0), product of:
             0.20464128 = queryWeight, product of:
               3.0838938 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.018027835 = queryNorm
             0.5635156 = fieldWeight in 4150, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
         0.45743987 = weight(abstract_txt:satisfaction in 4150) [ClassicSimilarity], result of:
           0.45743987 = score(doc=4150,freq=6.0), product of:
             0.46591163 = queryWeight, product of:
               4.0298157 = boost
               6.4132004 = idf(docFreq=197, maxDocs=44421)
               0.018027835 = queryNorm
             0.9818168 = fieldWeight in 4150, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.4132004 = idf(docFreq=197, maxDocs=44421)
               0.0625 = fieldNorm(doc=4150)
       0.6 = coord(15/25)
   

3. Su, L.T.: ¬A comprehensive and systematic model of user evaluation of Web search engines : I. Theory and background (2003) 0.63

   0.6295134 = sum of:
     0.6295134 = product of:
       1.1241311 = sum of:
         0.028270256 = weight(abstract_txt:relevance in 164) [ClassicSimilarity], result of:
           0.028270256 = score(doc=164,freq=1.0), product of:
             0.09175802 = queryWeight, product of:
               1.0325112 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.018027835 = queryNorm
             0.30809575 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.01920795 = weight(abstract_txt:system in 164) [ClassicSimilarity], result of:
           0.01920795 = score(doc=164,freq=2.0), product of:
             0.06443156 = queryWeight, product of:
               1.0596629 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.018027835 = queryNorm
             0.298114 = fieldWeight in 164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.014901656 = weight(abstract_txt:results in 164) [ClassicSimilarity], result of:
           0.014901656 = score(doc=164,freq=1.0), product of:
             0.06854004 = queryWeight, product of:
               1.0929255 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.018027835 = queryNorm
             0.21741535 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.017152913 = weight(abstract_txt:analysis in 164) [ClassicSimilarity], result of:
           0.017152913 = score(doc=164,freq=1.0), product of:
             0.07528005 = queryWeight, product of:
               1.145403 = boost
               3.6456752 = idf(docFreq=3151, maxDocs=44421)
               0.018027835 = queryNorm
             0.2278547 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6456752 = idf(docFreq=3151, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.03954803 = weight(abstract_txt:engine in 164) [ClassicSimilarity], result of:
           0.03954803 = score(doc=164,freq=1.0), product of:
             0.11477327 = queryWeight, product of:
               1.1547638 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.018027835 = queryNorm
             0.34457523 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.11330324 = weight(abstract_txt:undergraduates in 164) [ClassicSimilarity], result of:
           0.11330324 = score(doc=164,freq=1.0), product of:
             0.23151854 = queryWeight, product of:
               1.6400824 = boost
               7.8302665 = idf(docFreq=47, maxDocs=44421)
               0.018027835 = queryNorm
             0.48939165 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.8302665 = idf(docFreq=47, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.05651464 = weight(abstract_txt:measures in 164) [ClassicSimilarity], result of:
           0.05651464 = score(doc=164,freq=1.0), product of:
             0.16668452 = queryWeight, product of:
               1.7043785 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.018027835 = queryNorm
             0.3390515 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.060968373 = weight(abstract_txt:criteria in 164) [ClassicSimilarity], result of:
           0.060968373 = score(doc=164,freq=1.0), product of:
             0.1753306 = queryWeight, product of:
               1.7480235 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.018027835 = queryNorm
             0.3477338 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.08225608 = weight(abstract_txt:four in 164) [ClassicSimilarity], result of:
           0.08225608 = score(doc=164,freq=1.0), product of:
             0.25381437 = queryWeight, product of:
               2.715194 = boost
               5.1852746 = idf(docFreq=675, maxDocs=44421)
               0.018027835 = queryNorm
             0.32407966 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1852746 = idf(docFreq=675, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.11023553 = weight(abstract_txt:evaluation in 164) [ClassicSimilarity], result of:
           0.11023553 = score(doc=164,freq=3.0), product of:
             0.2273205 = queryWeight, product of:
               2.8148334 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.018027835 = queryNorm
             0.48493436 = fieldWeight in 164, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.0665792 = weight(abstract_txt:user in 164) [ClassicSimilarity], result of:
           0.0665792 = score(doc=164,freq=2.0), product of:
             0.20464128 = queryWeight, product of:
               3.0838938 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.018027835 = queryNorm
             0.32534587 = fieldWeight in 164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.081454776 = weight(abstract_txt:search in 164) [ClassicSimilarity], result of:
           0.081454776 = score(doc=164,freq=2.0), product of:
             0.25216332 = queryWeight, product of:
               3.8273547 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.018027835 = queryNorm
             0.3230239 = fieldWeight in 164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.18674904 = weight(abstract_txt:satisfaction in 164) [ClassicSimilarity], result of:
           0.18674904 = score(doc=164,freq=1.0), product of:
             0.46591163 = queryWeight, product of:
               4.0298157 = boost
               6.4132004 = idf(docFreq=197, maxDocs=44421)
               0.018027835 = queryNorm
             0.40082502 = fieldWeight in 164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4132004 = idf(docFreq=197, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
         0.24698944 = weight(abstract_txt:engines in 164) [ClassicSimilarity], result of:
           0.24698944 = score(doc=164,freq=2.0), product of:
             0.52827066 = queryWeight, product of:
               5.539697 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.018027835 = queryNorm
             0.46754336 = fieldWeight in 164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=164)
       0.56 = coord(14/25)
   

4. Spink, A.; Jansen, B.J.; Blakely, C.; Koshman, S.: ¬A study of results overlap and uniqueness among major Web search engines (2006) 0.53

   0.5281543 = sum of:
     0.5281543 = product of:
       1.2003508 = sum of:
         0.0099878255 = weight(abstract_txt:these in 1993) [ClassicSimilarity], result of:
           0.0099878255 = score(doc=1993,freq=1.0), product of:
             0.057380352 = queryWeight, product of:
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.018027835 = queryNorm
             0.1740635 = fieldWeight in 1993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.006423289 = weight(abstract_txt:with in 1993) [ClassicSimilarity], result of:
           0.006423289 = score(doc=1993,freq=1.0), product of:
             0.047054425 = queryWeight, product of:
               1.045654 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.018027835 = queryNorm
             0.13650765 = fieldWeight in 1993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.027589558 = weight(abstract_txt:study in 1993) [ClassicSimilarity], result of:
           0.027589558 = score(doc=1993,freq=5.0), product of:
             0.066062614 = queryWeight, product of:
               1.0729915 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.018027835 = queryNorm
             0.4176274 = fieldWeight in 1993, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.03911685 = weight(abstract_txt:results in 1993) [ClassicSimilarity], result of:
           0.03911685 = score(doc=1993,freq=9.0), product of:
             0.06854004 = queryWeight, product of:
               1.0929255 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.018027835 = queryNorm
             0.5707153 = fieldWeight in 1993, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.048938185 = weight(abstract_txt:engine in 1993) [ClassicSimilarity], result of:
           0.048938185 = score(doc=1993,freq=2.0), product of:
             0.11477327 = queryWeight, product of:
               1.1547638 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.018027835 = queryNorm
             0.42639008 = fieldWeight in 1993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.040720094 = weight(abstract_txt:performance in 1993) [ClassicSimilarity], result of:
           0.040720094 = score(doc=1993,freq=1.0), product of:
             0.16117637 = queryWeight, product of:
               1.9352561 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.018027835 = queryNorm
             0.25264308 = fieldWeight in 1993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.062850006 = weight(abstract_txt:significant in 1993) [ClassicSimilarity], result of:
           0.062850006 = score(doc=1993,freq=2.0), product of:
             0.17085291 = queryWeight, product of:
               1.9925029 = boost
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.018027835 = queryNorm
             0.3678603 = fieldWeight in 1993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.14394812 = weight(abstract_txt:four in 1993) [ClassicSimilarity], result of:
           0.14394812 = score(doc=1993,freq=4.0), product of:
             0.25381437 = queryWeight, product of:
               2.715194 = boost
               5.1852746 = idf(docFreq=675, maxDocs=44421)
               0.018027835 = queryNorm
             0.5671394 = fieldWeight in 1993, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.1852746 = idf(docFreq=675, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.041193776 = weight(abstract_txt:user in 1993) [ClassicSimilarity], result of:
           0.041193776 = score(doc=1993,freq=1.0), product of:
             0.20464128 = queryWeight, product of:
               3.0838938 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.018027835 = queryNorm
             0.20129749 = fieldWeight in 1993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.20779452 = weight(abstract_txt:search in 1993) [ClassicSimilarity], result of:
           0.20779452 = score(doc=1993,freq=17.0), product of:
             0.25216332 = queryWeight, product of:
               3.8273547 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.018027835 = queryNorm
             0.8240473 = fieldWeight in 1993, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.5717886 = weight(abstract_txt:engines in 1993) [ClassicSimilarity], result of:
           0.5717886 = score(doc=1993,freq=14.0), product of:
             0.52827066 = queryWeight, product of:
               5.539697 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.018027835 = queryNorm
             1.0823781 = fieldWeight in 1993, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
       0.44 = coord(11/25)
   

5. Can, F.; Nuray, R.; Sevdik, A.B.: Automatic performance evaluation of Web search engines (2004) 0.47

   0.4699985 = sum of:
     0.4699985 = product of:
       1.0681784 = sum of:
         0.020182254 = weight(abstract_txt:based in 3570) [ClassicSimilarity], result of:
           0.020182254 = score(doc=3570,freq=2.0), product of:
             0.05738756 = queryWeight, product of:
               1.0000628 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.018027835 = queryNorm
             0.35168344 = fieldWeight in 3570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.049975224 = weight(abstract_txt:relevance in 3570) [ClassicSimilarity], result of:
           0.049975224 = score(doc=3570,freq=2.0), product of:
             0.09175802 = queryWeight, product of:
               1.0325112 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.018027835 = queryNorm
             0.5446415 = fieldWeight in 3570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.012977003 = weight(abstract_txt:with in 3570) [ClassicSimilarity], result of:
           0.012977003 = score(doc=3570,freq=2.0), product of:
             0.047054425 = queryWeight, product of:
               1.045654 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.018027835 = queryNorm
             0.2757871 = fieldWeight in 3570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.017626321 = weight(abstract_txt:study in 3570) [ClassicSimilarity], result of:
           0.017626321 = score(doc=3570,freq=1.0), product of:
             0.066062614 = queryWeight, product of:
               1.0729915 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.018027835 = queryNorm
             0.26681235 = fieldWeight in 3570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.01862707 = weight(abstract_txt:results in 3570) [ClassicSimilarity], result of:
           0.01862707 = score(doc=3570,freq=1.0), product of:
             0.06854004 = queryWeight, product of:
               1.0929255 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.018027835 = queryNorm
             0.2717692 = fieldWeight in 3570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.04943503 = weight(abstract_txt:engine in 3570) [ClassicSimilarity], result of:
           0.04943503 = score(doc=3570,freq=1.0), product of:
             0.11477327 = queryWeight, product of:
               1.1547638 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.018027835 = queryNorm
             0.43071902 = fieldWeight in 3570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.063488096 = weight(abstract_txt:significant in 3570) [ClassicSimilarity], result of:
           0.063488096 = score(doc=3570,freq=1.0), product of:
             0.17085291 = queryWeight, product of:
               1.9925029 = boost
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.018027835 = queryNorm
             0.37159503 = fieldWeight in 3570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7564163 = idf(docFreq=1037, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.11250866 = weight(abstract_txt:evaluation in 3570) [ClassicSimilarity], result of:
           0.11250866 = score(doc=3570,freq=2.0), product of:
             0.2273205 = queryWeight, product of:
               2.8148334 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.018027835 = queryNorm
             0.49493405 = fieldWeight in 3570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.058848254 = weight(abstract_txt:user in 3570) [ClassicSimilarity], result of:
           0.058848254 = score(doc=3570,freq=1.0), product of:
             0.20464128 = queryWeight, product of:
               3.0838938 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.018027835 = queryNorm
             0.28756785 = fieldWeight in 3570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.17635477 = weight(abstract_txt:search in 3570) [ClassicSimilarity], result of:
           0.17635477 = score(doc=3570,freq=6.0), product of:
             0.25216332 = queryWeight, product of:
               3.8273547 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.018027835 = queryNorm
             0.6993672 = fieldWeight in 3570, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
         0.48815578 = weight(abstract_txt:engines in 3570) [ClassicSimilarity], result of:
           0.48815578 = score(doc=3570,freq=5.0), product of:
             0.52827066 = queryWeight, product of:
               5.539697 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.018027835 = queryNorm
             0.9240638 = fieldWeight in 3570, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.078125 = fieldNorm(doc=3570)
       0.44 = coord(11/25)