Document (#32471)

Author

Bar-Ilan, J.
Keenoy, K.
Yaari, E.
Levene, M.

Title

User rankings of search engine results

Source

Journal of the American Society for Information Science and Technology. 58(2007) no.9, S.1254-1266

Year

2007

Abstract

In this study, we investigate the similarities and differences between rankings of search results by users and search engines. Sixty-seven students took part in a 3-week-long experiment, during which they were asked to identify and rank the top 10 documents from the set of URLs that were retrieved by three major search engines (Google, MSN Search, and Yahoo!) for 12 selected queries. The URLs and accompanying snippets were displayed in random order, without disclosing which search engine(s) retrieved any specific URL for the query. We computed the similarity of the rankings of the users and search engines using four nonparametric correlation measures in [0,1] that complement each other. The findings show that the similarities between the users' choices and the rankings of the search engines are low. We examined the effects of the presentation order of the results, and of the thinking styles of the participants. Presentation order influences the rankings, but overall the results indicate that there is no "average user," and even if the users have the same basic knowledge of a topic, they evaluate information in their own context, which is influenced by cognitive, affective, and physical factors. This is the first large-scale experiment in which users were asked to rank the results of identical queries. The analysis of the experimental results demonstrates the potential for personalized search.

Theme

Suchmaschinen
Benutzerstudien

Object

Google
MSN Search
Yahoo

Similar documents (author)

1. Bar-Ilan, J.; Levene, M.: ¬The hw-rank : an h-index variant for ranking web pages (2015) 5.41

   5.411173 = sum of:
     5.411173 = sum of:
       2.1073067 = weight(author_txt:ilan in 2694) [ClassicSimilarity], result of:
         2.1073067 = score(doc=2694,freq=1.0), product of:
           0.5953491 = queryWeight, product of:
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.07358574 = queryNorm
           3.5396154 = fieldWeight in 2694, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.4375 = fieldNorm(doc=2694)
       3.3038661 = weight(author_txt:levene in 2694) [ClassicSimilarity], result of:
         3.3038661 = score(doc=2694,freq=1.0), product of:
           0.80346715 = queryWeight, product of:
             1.1617113 = boost
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.07358574 = queryNorm
           4.1120114 = fieldWeight in 2694, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.4375 = fieldNorm(doc=2694)
   

2. Bar-Ilan, J.; Levene, M.; Mat-Hassan, M.: Methods for evaluating dynamic changes in search engine rankings : a case study (2006) 3.87

   3.8651233 = sum of:
     3.8651233 = sum of:
       1.5052191 = weight(author_txt:ilan in 741) [ClassicSimilarity], result of:
         1.5052191 = score(doc=741,freq=1.0), product of:
           0.5953491 = queryWeight, product of:
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.07358574 = queryNorm
           2.5282967 = fieldWeight in 741, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.3125 = fieldNorm(doc=741)
       2.3599043 = weight(author_txt:levene in 741) [ClassicSimilarity], result of:
         2.3599043 = score(doc=741,freq=1.0), product of:
           0.80346715 = queryWeight, product of:
             1.1617113 = boost
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.07358574 = queryNorm
           2.937151 = fieldWeight in 741, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.3125 = fieldNorm(doc=741)
   

3. Bar-Ilan, J.; Keenoy, K.; Levene, M.; Yaari, E.: Presentation bias is significant in determining user preference for search results : a user study (2009) 3.87

   3.8651233 = sum of:
     3.8651233 = sum of:
       1.5052191 = weight(author_txt:ilan in 3703) [ClassicSimilarity], result of:
         1.5052191 = score(doc=3703,freq=1.0), product of:
           0.5953491 = queryWeight, product of:
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.07358574 = queryNorm
           2.5282967 = fieldWeight in 3703, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.3125 = fieldNorm(doc=3703)
       2.3599043 = weight(author_txt:levene in 3703) [ClassicSimilarity], result of:
         2.3599043 = score(doc=3703,freq=1.0), product of:
           0.80346715 = queryWeight, product of:
             1.1617113 = boost
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.07358574 = queryNorm
           2.937151 = fieldWeight in 3703, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.3125 = fieldNorm(doc=3703)
   

4. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Testing the stability of "wisdom of crowds" judgments of search results over time and their similarity with the search engine rankings (2016) 3.87

   3.8651233 = sum of:
     3.8651233 = sum of:
       1.5052191 = weight(author_txt:ilan in 4071) [ClassicSimilarity], result of:
         1.5052191 = score(doc=4071,freq=1.0), product of:
           0.5953491 = queryWeight, product of:
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.07358574 = queryNorm
           2.5282967 = fieldWeight in 4071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.3125 = fieldNorm(doc=4071)
       2.3599043 = weight(author_txt:levene in 4071) [ClassicSimilarity], result of:
         2.3599043 = score(doc=4071,freq=1.0), product of:
           0.80346715 = queryWeight, product of:
             1.1617113 = boost
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.07358574 = queryNorm
           2.937151 = fieldWeight in 4071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.3125 = fieldNorm(doc=4071)
   

5. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Analysis of change in users' assessment of search results over time (2017) 3.87

   3.8651233 = sum of:
     3.8651233 = sum of:
       1.5052191 = weight(author_txt:ilan in 4593) [ClassicSimilarity], result of:
         1.5052191 = score(doc=4593,freq=1.0), product of:
           0.5953491 = queryWeight, product of:
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.07358574 = queryNorm
           2.5282967 = fieldWeight in 4593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.090549 = idf(docFreq=36, maxDocs=44421)
             0.3125 = fieldNorm(doc=4593)
       2.3599043 = weight(author_txt:levene in 4593) [ClassicSimilarity], result of:
         2.3599043 = score(doc=4593,freq=1.0), product of:
           0.80346715 = queryWeight, product of:
             1.1617113 = boost
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.07358574 = queryNorm
           2.937151 = fieldWeight in 4593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.398883 = idf(docFreq=9, maxDocs=44421)
             0.3125 = fieldNorm(doc=4593)
   

Similar documents (content)

1. Bar-Ilan, J.; Levene, M.; Mat-Hassan, M.: Methods for evaluating dynamic changes in search engine rankings : a case study (2006) 0.65

   0.6482297 = sum of:
     0.6482297 = product of:
       1.3504786 = sum of:
         0.05524131 = weight(abstract_txt:computed in 741) [ClassicSimilarity], result of:
           0.05524131 = score(doc=741,freq=1.0), product of:
             0.13249424 = queryWeight, product of:
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.017378733 = queryNorm
             0.41693366 = fieldWeight in 741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.011423669 = weight(abstract_txt:that in 741) [ClassicSimilarity], result of:
           0.011423669 = score(doc=741,freq=3.0), product of:
             0.050996184 = queryWeight, product of:
               1.2407959 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017378733 = queryNorm
             0.22401026 = fieldWeight in 741, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.04681562 = weight(abstract_txt:queries in 741) [ClassicSimilarity], result of:
           0.04681562 = score(doc=741,freq=2.0), product of:
             0.11865377 = queryWeight, product of:
               1.3383117 = boost
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.017378733 = queryNorm
             0.39455652 = fieldWeight in 741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.059085973 = weight(abstract_txt:engine in 741) [ClassicSimilarity], result of:
           0.059085973 = score(doc=741,freq=2.0), product of:
             0.13857257 = queryWeight, product of:
               1.4462892 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.017378733 = queryNorm
             0.42639008 = fieldWeight in 741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.045179177 = weight(abstract_txt:experiment in 741) [ClassicSimilarity], result of:
           0.045179177 = score(doc=741,freq=1.0), product of:
             0.14599004 = queryWeight, product of:
               1.4844929 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017378733 = queryNorm
             0.30946752 = fieldWeight in 741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.046565514 = weight(abstract_txt:order in 741) [ClassicSimilarity], result of:
           0.046565514 = score(doc=741,freq=2.0), product of:
             0.13534048 = queryWeight, product of:
               1.7505559 = boost
               4.448705 = idf(docFreq=1411, maxDocs=44421)
               0.017378733 = queryNorm
             0.34406197 = fieldWeight in 741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.448705 = idf(docFreq=1411, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.04260424 = weight(abstract_txt:were in 741) [ClassicSimilarity], result of:
           0.04260424 = score(doc=741,freq=3.0), product of:
             0.12264104 = queryWeight, product of:
               1.9241965 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.017378733 = queryNorm
             0.34738976 = fieldWeight in 741, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.028295266 = weight(abstract_txt:users in 741) [ClassicSimilarity], result of:
           0.028295266 = score(doc=741,freq=1.0), product of:
             0.14503969 = queryWeight, product of:
               2.3395371 = boost
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.017378733 = queryNorm
             0.19508636 = fieldWeight in 741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.0833025 = weight(abstract_txt:results in 741) [ClassicSimilarity], result of:
           0.0833025 = score(doc=741,freq=7.0), product of:
             0.16550487 = queryWeight, product of:
               2.737679 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017378733 = queryNorm
             0.50332355 = fieldWeight in 741, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.19526166 = weight(abstract_txt:engines in 741) [ClassicSimilarity], result of:
           0.19526166 = score(doc=741,freq=7.0), product of:
             0.255125 = queryWeight, product of:
               2.7752876 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.017378733 = queryNorm
             0.76535684 = fieldWeight in 741, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.1642895 = weight(abstract_txt:search in 741) [ClassicSimilarity], result of:
           0.1642895 = score(doc=741,freq=9.0), product of:
             0.27400652 = queryWeight, product of:
               4.314229 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017378733 = queryNorm
             0.5995825 = fieldWeight in 741, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
         0.5724142 = weight(abstract_txt:rankings in 741) [ClassicSimilarity], result of:
           0.5724142 = score(doc=741,freq=8.0), product of:
             0.53842014 = queryWeight, product of:
               4.5076184 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.017378733 = queryNorm
             1.0631367 = fieldWeight in 741, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0546875 = fieldNorm(doc=741)
       0.48 = coord(12/25)
   

2. Bar-Ilan, J.: Comparing rankings of search results on the Web (2005) 0.56

   0.55610174 = sum of:
     0.55610174 = product of:
       1.2638676 = sum of:
         0.08928344 = weight(abstract_txt:computed in 2068) [ClassicSimilarity], result of:
           0.08928344 = score(doc=2068,freq=2.0), product of:
             0.13249424 = queryWeight, product of:
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.017378733 = queryNorm
             0.67386657 = fieldWeight in 2068, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.015075333 = weight(abstract_txt:that in 2068) [ClassicSimilarity], result of:
           0.015075333 = score(doc=2068,freq=4.0), product of:
             0.050996184 = queryWeight, product of:
               1.2407959 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017378733 = queryNorm
             0.2956169 = fieldWeight in 2068, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.05350356 = weight(abstract_txt:queries in 2068) [ClassicSimilarity], result of:
           0.05350356 = score(doc=2068,freq=2.0), product of:
             0.11865377 = queryWeight, product of:
               1.3383117 = boost
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.017378733 = queryNorm
             0.45092174 = fieldWeight in 2068, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.047748677 = weight(abstract_txt:engine in 2068) [ClassicSimilarity], result of:
           0.047748677 = score(doc=2068,freq=1.0), product of:
             0.13857257 = queryWeight, product of:
               1.4462892 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.017378733 = queryNorm
             0.34457523 = fieldWeight in 2068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.052907806 = weight(abstract_txt:retrieved in 2068) [ClassicSimilarity], result of:
           0.052907806 = score(doc=2068,freq=1.0), product of:
             0.14838257 = queryWeight, product of:
               1.4966075 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.017378733 = queryNorm
             0.35656348 = fieldWeight in 2068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.10742516 = weight(abstract_txt:urls in 2068) [ClassicSimilarity], result of:
           0.10742516 = score(doc=2068,freq=1.0), product of:
             0.23792517 = queryWeight, product of:
               1.8951199 = boost
               7.2241306 = idf(docFreq=87, maxDocs=44421)
               0.017378733 = queryNorm
             0.45150816 = fieldWeight in 2068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.2241306 = idf(docFreq=87, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.02811151 = weight(abstract_txt:were in 2068) [ClassicSimilarity], result of:
           0.02811151 = score(doc=2068,freq=1.0), product of:
             0.12264104 = queryWeight, product of:
               1.9241965 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.017378733 = queryNorm
             0.2292178 = fieldWeight in 2068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.08046111 = weight(abstract_txt:results in 2068) [ClassicSimilarity], result of:
           0.08046111 = score(doc=2068,freq=5.0), product of:
             0.16550487 = queryWeight, product of:
               2.737679 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017378733 = queryNorm
             0.4861555 = fieldWeight in 2068, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.22315617 = weight(abstract_txt:engines in 2068) [ClassicSimilarity], result of:
           0.22315617 = score(doc=2068,freq=7.0), product of:
             0.255125 = queryWeight, product of:
               2.7752876 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.017378733 = queryNorm
             0.8746935 = fieldWeight in 2068, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.16558824 = weight(abstract_txt:search in 2068) [ClassicSimilarity], result of:
           0.16558824 = score(doc=2068,freq=7.0), product of:
             0.27400652 = queryWeight, product of:
               4.314229 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017378733 = queryNorm
             0.6043223 = fieldWeight in 2068, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
         0.4006065 = weight(abstract_txt:rankings in 2068) [ClassicSimilarity], result of:
           0.4006065 = score(doc=2068,freq=3.0), product of:
             0.53842014 = queryWeight, product of:
               4.5076184 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.017378733 = queryNorm
             0.7440407 = fieldWeight in 2068, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0625 = fieldNorm(doc=2068)
       0.44 = coord(11/25)
   

3. Bar-Ilan, J.; Keenoy, K.; Levene, M.; Yaari, E.: Presentation bias is significant in determining user preference for search results : a user study (2009) 0.40

   0.40321958 = sum of:
     0.40321958 = product of:
       0.8400408 = sum of:
         0.10363191 = weight(abstract_txt:snippets in 3703) [ClassicSimilarity], result of:
           0.10363191 = score(doc=3703,freq=1.0), product of:
             0.18436936 = queryWeight, product of:
               1.1796302 = boost
               8.993418 = idf(docFreq=14, maxDocs=44421)
               0.017378733 = queryNorm
             0.5620886 = fieldWeight in 3703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.993418 = idf(docFreq=14, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.010659871 = weight(abstract_txt:that in 3703) [ClassicSimilarity], result of:
           0.010659871 = score(doc=3703,freq=2.0), product of:
             0.050996184 = queryWeight, product of:
               1.2407959 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017378733 = queryNorm
             0.20903271 = fieldWeight in 3703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.037832733 = weight(abstract_txt:queries in 3703) [ClassicSimilarity], result of:
           0.037832733 = score(doc=3703,freq=1.0), product of:
             0.11865377 = queryWeight, product of:
               1.3383117 = boost
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.017378733 = queryNorm
             0.31884983 = fieldWeight in 3703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.047748677 = weight(abstract_txt:engine in 3703) [ClassicSimilarity], result of:
           0.047748677 = score(doc=3703,freq=1.0), product of:
             0.13857257 = queryWeight, product of:
               1.4462892 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.017378733 = queryNorm
             0.34457523 = fieldWeight in 3703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.07302058 = weight(abstract_txt:experiment in 3703) [ClassicSimilarity], result of:
           0.07302058 = score(doc=3703,freq=2.0), product of:
             0.14599004 = queryWeight, product of:
               1.4844929 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017378733 = queryNorm
             0.50017506 = fieldWeight in 3703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.052907806 = weight(abstract_txt:retrieved in 3703) [ClassicSimilarity], result of:
           0.052907806 = score(doc=3703,freq=1.0), product of:
             0.14838257 = queryWeight, product of:
               1.4966075 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.017378733 = queryNorm
             0.35656348 = fieldWeight in 3703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.06738462 = weight(abstract_txt:asked in 3703) [ClassicSimilarity], result of:
           0.06738462 = score(doc=3703,freq=1.0), product of:
             0.1743453 = queryWeight, product of:
               1.6222643 = boost
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.017378733 = queryNorm
             0.38650092 = fieldWeight in 3703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.184015 = idf(docFreq=248, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.02811151 = weight(abstract_txt:were in 3703) [ClassicSimilarity], result of:
           0.02811151 = score(doc=3703,freq=1.0), product of:
             0.12264104 = queryWeight, product of:
               1.9241965 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.017378733 = queryNorm
             0.2292178 = fieldWeight in 3703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.045732055 = weight(abstract_txt:users in 3703) [ClassicSimilarity], result of:
           0.045732055 = score(doc=3703,freq=2.0), product of:
             0.14503969 = queryWeight, product of:
               2.3395371 = boost
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.017378733 = queryNorm
             0.31530717 = fieldWeight in 3703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.08814073 = weight(abstract_txt:results in 3703) [ClassicSimilarity], result of:
           0.08814073 = score(doc=3703,freq=6.0), product of:
             0.16550487 = queryWeight, product of:
               2.737679 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017378733 = queryNorm
             0.5325567 = fieldWeight in 3703, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.11928199 = weight(abstract_txt:engines in 3703) [ClassicSimilarity], result of:
           0.11928199 = score(doc=3703,freq=2.0), product of:
             0.255125 = queryWeight, product of:
               2.7752876 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.017378733 = queryNorm
             0.46754336 = fieldWeight in 3703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
         0.16558824 = weight(abstract_txt:search in 3703) [ClassicSimilarity], result of:
           0.16558824 = score(doc=3703,freq=7.0), product of:
             0.27400652 = queryWeight, product of:
               4.314229 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017378733 = queryNorm
             0.6043223 = fieldWeight in 3703, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=3703)
       0.48 = coord(12/25)
   

4. Vaughan, L.: New measurements for search engine evaluation proposed and tested (2004) 0.37

   0.37491313 = sum of:
     0.37491313 = product of:
       1.0414253 = sum of:
         0.011306501 = weight(abstract_txt:that in 3535) [ClassicSimilarity], result of:
           0.011306501 = score(doc=3535,freq=1.0), product of:
             0.050996184 = queryWeight, product of:
               1.2407959 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017378733 = queryNorm
             0.22171268 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.14324604 = weight(abstract_txt:engine in 3535) [ClassicSimilarity], result of:
           0.14324604 = score(doc=3535,freq=4.0), product of:
             0.13857257 = queryWeight, product of:
               1.4462892 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.017378733 = queryNorm
             1.0337257 = fieldWeight in 3535, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.07745002 = weight(abstract_txt:experiment in 3535) [ClassicSimilarity], result of:
           0.07745002 = score(doc=3535,freq=1.0), product of:
             0.14599004 = queryWeight, product of:
               1.4844929 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017378733 = queryNorm
             0.5305158 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.029906107 = weight(abstract_txt:which in 3535) [ClassicSimilarity], result of:
           0.029906107 = score(doc=3535,freq=2.0), product of:
             0.077413276 = queryWeight, product of:
               1.5287597 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.017378733 = queryNorm
             0.38631755 = fieldWeight in 3535, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.042167265 = weight(abstract_txt:were in 3535) [ClassicSimilarity], result of:
           0.042167265 = score(doc=3535,freq=1.0), product of:
             0.12264104 = queryWeight, product of:
               1.9241965 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.017378733 = queryNorm
             0.3438267 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.05397495 = weight(abstract_txt:results in 3535) [ClassicSimilarity], result of:
           0.05397495 = score(doc=3535,freq=1.0), product of:
             0.16550487 = queryWeight, product of:
               2.737679 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017378733 = queryNorm
             0.32612303 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.12651765 = weight(abstract_txt:engines in 3535) [ClassicSimilarity], result of:
           0.12651765 = score(doc=3535,freq=1.0), product of:
             0.255125 = queryWeight, product of:
               2.7752876 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.017378733 = queryNorm
             0.49590462 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.20992143 = weight(abstract_txt:search in 3535) [ClassicSimilarity], result of:
           0.20992143 = score(doc=3535,freq=5.0), product of:
             0.27400652 = queryWeight, product of:
               4.314229 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017378733 = queryNorm
             0.7661184 = fieldWeight in 3535, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.34693542 = weight(abstract_txt:rankings in 3535) [ClassicSimilarity], result of:
           0.34693542 = score(doc=3535,freq=1.0), product of:
             0.53842014 = queryWeight, product of:
               4.5076184 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.017378733 = queryNorm
             0.64435816 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
       0.36 = coord(9/25)
   

5. Spink, A.; Jansen, B.J.; Blakely, C.; Koshman, S.: ¬A study of results overlap and uniqueness among major Web search engines (2006) 0.33

   0.3318768 = sum of:
     0.3318768 = product of:
       0.92188 = sum of:
         0.0065954584 = weight(abstract_txt:that in 1993) [ClassicSimilarity], result of:
           0.0065954584 = score(doc=1993,freq=1.0), product of:
             0.050996184 = queryWeight, product of:
               1.2407959 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017378733 = queryNorm
             0.1293324 = fieldWeight in 1993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.09363124 = weight(abstract_txt:queries in 1993) [ClassicSimilarity], result of:
           0.09363124 = score(doc=1993,freq=8.0), product of:
             0.11865377 = queryWeight, product of:
               1.3383117 = boost
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.017378733 = queryNorm
             0.78911304 = fieldWeight in 1993, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.1015973 = idf(docFreq=734, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.059085973 = weight(abstract_txt:engine in 1993) [ClassicSimilarity], result of:
           0.059085973 = score(doc=1993,freq=2.0), product of:
             0.13857257 = queryWeight, product of:
               1.4462892 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.017378733 = queryNorm
             0.42639008 = fieldWeight in 1993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.06547007 = weight(abstract_txt:retrieved in 1993) [ClassicSimilarity], result of:
           0.06547007 = score(doc=1993,freq=2.0), product of:
             0.14838257 = queryWeight, product of:
               1.4966075 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.017378733 = queryNorm
             0.44122478 = fieldWeight in 1993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.0659189 = weight(abstract_txt:rank in 1993) [ClassicSimilarity], result of:
           0.0659189 = score(doc=1993,freq=1.0), product of:
             0.18780378 = queryWeight, product of:
               1.6837153 = boost
               6.418264 = idf(docFreq=196, maxDocs=44421)
               0.017378733 = queryNorm
             0.35099882 = fieldWeight in 1993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.418264 = idf(docFreq=196, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.034786217 = weight(abstract_txt:were in 1993) [ClassicSimilarity], result of:
           0.034786217 = score(doc=1993,freq=2.0), product of:
             0.12264104 = queryWeight, product of:
               1.9241965 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.017378733 = queryNorm
             0.28364253 = fieldWeight in 1993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.094456166 = weight(abstract_txt:results in 1993) [ClassicSimilarity], result of:
           0.094456166 = score(doc=1993,freq=9.0), product of:
             0.16550487 = queryWeight, product of:
               2.737679 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017378733 = queryNorm
             0.5707153 = fieldWeight in 1993, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.2761417 = weight(abstract_txt:engines in 1993) [ClassicSimilarity], result of:
           0.2761417 = score(doc=1993,freq=14.0), product of:
             0.255125 = queryWeight, product of:
               2.7752876 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.017378733 = queryNorm
             1.0823781 = fieldWeight in 1993, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
         0.22579433 = weight(abstract_txt:search in 1993) [ClassicSimilarity], result of:
           0.22579433 = score(doc=1993,freq=17.0), product of:
             0.27400652 = queryWeight, product of:
               4.314229 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017378733 = queryNorm
             0.8240473 = fieldWeight in 1993, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1993)
       0.36 = coord(9/25)