Document (#30170)

Author

Ruthven, T.
Lalmas, M.
Rijsbergen, K.van

Title

Incorporating user research behavior into relevance feedback

Source

Journal of the American Society for Information Science and technology. 54(2003) no.6, S.528-548

Year

2003

Abstract

Ruthven, Mounia, and van Rijsbergen rank and select terms for query expansion using information gathered on searcher evaluation behavior. Using the TREC Financial Times and Los Angeles Times collections and search topics from TREC-6 placed in simulated work situations, six student subjects each preformed three searches on an experimental system and three on a control system with instructions to search by natural language expression in any way they found comfortable. Searching was analyzed for behavior differences between experimental and control situations, and for effectiveness and perceptions. In three experiments paired t-tests were the analysis tool with controls being a no relevance feedback system, a standard ranking for automatic expansion system, and a standard ranking for interactive expansion while the experimental systems based ranking upon user information on temporal relevance and partial relevance. Two further experiments compare using user behavior (number assessed relevant and similarity of relevant documents) to choose a query expansion technique against a non-selective technique and finally the effect of providing the user with knowledge of the process. When partial relevance data and time of assessment data are incorporated in term ranking more relevant documents were recovered in fewer iterations, however retrieval effectiveness overall was not improved. The subjects, none-the-less, rated the suggested terms as more useful and used them more heavily. Explanations of what the feedback techniques were doing led to higher use of the techniques.

Theme

Retrievalalgorithmen

Similar documents (author)

1. Ruthven, I.; Lalmas, M.; Rijsbergen, K. van: Combining and selecting characteristics of information use (2002) 5.42

   5.420111 = sum of:
     5.420111 = sum of:
       1.6792222 = weight(author_txt:ruthven in 208) [ClassicSimilarity], result of:
         1.6792222 = score(doc=208,freq=1.0), product of:
           0.5497005 = queryWeight, product of:
             8.146119 = idf(docFreq=34, maxDocs=44421)
             0.06748005 = queryNorm
           3.0547948 = fieldWeight in 208, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.146119 = idf(docFreq=34, maxDocs=44421)
             0.375 = fieldNorm(doc=208)
       1.8448746 = weight(author_txt:rijsbergen in 208) [ClassicSimilarity], result of:
         1.8448746 = score(doc=208,freq=1.0), product of:
           0.58528215 = queryWeight, product of:
             1.0318571 = boost
             8.405631 = idf(docFreq=26, maxDocs=44421)
             0.06748005 = queryNorm
           3.1521115 = fieldWeight in 208, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.405631 = idf(docFreq=26, maxDocs=44421)
             0.375 = fieldNorm(doc=208)
       1.8960142 = weight(author_txt:lalmas in 208) [ClassicSimilarity], result of:
         1.8960142 = score(doc=208,freq=1.0), product of:
           0.5960487 = queryWeight, product of:
             1.0413046 = boost
             8.482592 = idf(docFreq=24, maxDocs=44421)
             0.06748005 = queryNorm
           3.1809719 = fieldWeight in 208, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.482592 = idf(docFreq=24, maxDocs=44421)
             0.375 = fieldNorm(doc=208)
   

2. Lalmas, M.; Ruthven, I.: ¬A model for structured document retrieval : empirical investigations (1997) 3.18

   3.177988 = sum of:
     3.177988 = product of:
       4.766982 = sum of:
         2.238963 = weight(author_txt:ruthven in 1727) [ClassicSimilarity], result of:
           2.238963 = score(doc=1727,freq=1.0), product of:
             0.5497005 = queryWeight, product of:
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.06748005 = queryNorm
             4.0730596 = fieldWeight in 1727, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.5 = fieldNorm(doc=1727)
         2.528019 = weight(author_txt:lalmas in 1727) [ClassicSimilarity], result of:
           2.528019 = score(doc=1727,freq=1.0), product of:
             0.5960487 = queryWeight, product of:
               1.0413046 = boost
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.06748005 = queryNorm
             4.241296 = fieldWeight in 1727, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.5 = fieldNorm(doc=1727)
       0.6666667 = coord(2/3)
   

3. Lalmas, M.; Ruthven, I.: Representing and retrieving structured documents using the Dempster-Shafer theory of evidence : modelling and evaluation (1998) 3.18

   3.177988 = sum of:
     3.177988 = product of:
       4.766982 = sum of:
         2.238963 = weight(author_txt:ruthven in 2076) [ClassicSimilarity], result of:
           2.238963 = score(doc=2076,freq=1.0), product of:
             0.5497005 = queryWeight, product of:
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.06748005 = queryNorm
             4.0730596 = fieldWeight in 2076, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.5 = fieldNorm(doc=2076)
         2.528019 = weight(author_txt:lalmas in 2076) [ClassicSimilarity], result of:
           2.528019 = score(doc=2076,freq=1.0), product of:
             0.5960487 = queryWeight, product of:
               1.0413046 = boost
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.06748005 = queryNorm
             4.241296 = fieldWeight in 2076, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.5 = fieldNorm(doc=2076)
       0.6666667 = coord(2/3)
   

4. Ruthven, I.; Lalmas, M.: Selective relevance feedback using term characteristics (1999) 3.18

   3.177988 = sum of:
     3.177988 = product of:
       4.766982 = sum of:
         2.238963 = weight(author_txt:ruthven in 4824) [ClassicSimilarity], result of:
           2.238963 = score(doc=4824,freq=1.0), product of:
             0.5497005 = queryWeight, product of:
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.06748005 = queryNorm
             4.0730596 = fieldWeight in 4824, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.5 = fieldNorm(doc=4824)
         2.528019 = weight(author_txt:lalmas in 4824) [ClassicSimilarity], result of:
           2.528019 = score(doc=4824,freq=1.0), product of:
             0.5960487 = queryWeight, product of:
               1.0413046 = boost
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.06748005 = queryNorm
             4.241296 = fieldWeight in 4824, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.5 = fieldNorm(doc=4824)
       0.6666667 = coord(2/3)
   

5. Rijsbergen, C.J. van; Lalmas, M.: Information calculus for information retrieval (1996) 2.91

   2.9095802 = sum of:
     2.9095802 = product of:
       4.3643703 = sum of:
         2.1523538 = weight(author_txt:rijsbergen in 4269) [ClassicSimilarity], result of:
           2.1523538 = score(doc=4269,freq=1.0), product of:
             0.58528215 = queryWeight, product of:
               1.0318571 = boost
               8.405631 = idf(docFreq=26, maxDocs=44421)
               0.06748005 = queryNorm
             3.6774635 = fieldWeight in 4269, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.405631 = idf(docFreq=26, maxDocs=44421)
               0.4375 = fieldNorm(doc=4269)
         2.2120166 = weight(author_txt:lalmas in 4269) [ClassicSimilarity], result of:
           2.2120166 = score(doc=4269,freq=1.0), product of:
             0.5960487 = queryWeight, product of:
               1.0413046 = boost
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.06748005 = queryNorm
             3.711134 = fieldWeight in 4269, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.482592 = idf(docFreq=24, maxDocs=44421)
               0.4375 = fieldNorm(doc=4269)
       0.6666667 = coord(2/3)
   

Similar documents (content)

1. Vakkari, P.; Sormunen, E.: ¬The influence of relevance levels an the effectiveness of interactive information retrieval (2004) 0.65

   0.6502481 = sum of:
     0.6502481 = product of:
       1.3546836 = sum of:
         0.07323351 = weight(abstract_txt:query in 3884) [ClassicSimilarity], result of:
           0.07323351 = score(doc=3884,freq=3.0), product of:
             0.11382956 = queryWeight, product of:
               1.00542 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023812415 = queryNorm
             0.6433611 = fieldWeight in 3884, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.05206771 = weight(abstract_txt:effectiveness in 3884) [ClassicSimilarity], result of:
           0.05206771 = score(doc=3884,freq=1.0), product of:
             0.13077797 = queryWeight, product of:
               1.0776737 = boost
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.023812415 = queryNorm
             0.39813823 = fieldWeight in 3884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.024376849 = weight(abstract_txt:using in 3884) [ClassicSimilarity], result of:
           0.024376849 = score(doc=3884,freq=1.0), product of:
             0.090261884 = queryWeight, product of:
               1.0965232 = boost
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.023812415 = queryNorm
             0.27006802 = fieldWeight in 3884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.08399382 = weight(abstract_txt:experiments in 3884) [ClassicSimilarity], result of:
           0.08399382 = score(doc=3884,freq=2.0), product of:
             0.14277226 = queryWeight, product of:
               1.1260091 = boost
               5.324741 = idf(docFreq=587, maxDocs=44421)
               0.023812415 = queryNorm
             0.5883063 = fieldWeight in 3884, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.324741 = idf(docFreq=587, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.062739745 = weight(abstract_txt:subjects in 3884) [ClassicSimilarity], result of:
           0.062739745 = score(doc=3884,freq=1.0), product of:
             0.14808717 = queryWeight, product of:
               1.1467763 = boost
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.023812415 = queryNorm
             0.42366767 = fieldWeight in 3884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.029109389 = weight(abstract_txt:were in 3884) [ClassicSimilarity], result of:
           0.029109389 = score(doc=3884,freq=1.0), product of:
             0.101595566 = queryWeight, product of:
               1.1633302 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.023812415 = queryNorm
             0.28652224 = fieldWeight in 3884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.16764997 = weight(abstract_txt:trec in 3884) [ClassicSimilarity], result of:
           0.16764997 = score(doc=3884,freq=2.0), product of:
             0.22633304 = queryWeight, product of:
               1.4177314 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.023812415 = queryNorm
             0.7407225 = fieldWeight in 3884, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.1014383 = weight(abstract_txt:relevant in 3884) [ClassicSimilarity], result of:
           0.1014383 = score(doc=3884,freq=3.0), product of:
             0.16191246 = queryWeight, product of:
               1.4686075 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.023812415 = queryNorm
             0.6265009 = fieldWeight in 3884, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.039238986 = weight(abstract_txt:user in 3884) [ClassicSimilarity], result of:
           0.039238986 = score(doc=3884,freq=1.0), product of:
             0.13645123 = queryWeight, product of:
               1.5567673 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.023812415 = queryNorm
             0.28756785 = fieldWeight in 3884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.17485355 = weight(abstract_txt:feedback in 3884) [ClassicSimilarity], result of:
           0.17485355 = score(doc=3884,freq=2.0), product of:
             0.2664561 = queryWeight, product of:
               1.8839893 = boost
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.023812415 = queryNorm
             0.656219 = fieldWeight in 3884, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.31035534 = weight(abstract_txt:expansion in 3884) [ClassicSimilarity], result of:
           0.31035534 = score(doc=3884,freq=3.0), product of:
             0.37557617 = queryWeight, product of:
               2.5827596 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.023812415 = queryNorm
             0.8263446 = fieldWeight in 3884, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
         0.2356264 = weight(abstract_txt:relevance in 3884) [ClassicSimilarity], result of:
           0.2356264 = score(doc=3884,freq=4.0), product of:
             0.3059132 = queryWeight, product of:
               2.6060882 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.023812415 = queryNorm
             0.77023935 = fieldWeight in 3884, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.078125 = fieldNorm(doc=3884)
       0.48 = coord(12/25)
   

2. Efthimiadis, E.N.: Interactive query expansion : a user-based evaluation in a relevance feedback environment (2000) 0.52

   0.52186817 = sum of:
     0.52186817 = product of:
       1.0872253 = sum of:
         0.12203143 = weight(abstract_txt:query in 6701) [ClassicSimilarity], result of:
           0.12203143 = score(doc=6701,freq=17.0), product of:
             0.11382956 = queryWeight, product of:
               1.00542 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023812415 = queryNorm
             1.0720539 = fieldWeight in 6701, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.016181463 = weight(abstract_txt:more in 6701) [ClassicSimilarity], result of:
           0.016181463 = score(doc=6701,freq=1.0), product of:
             0.08712294 = queryWeight, product of:
               1.0772882 = boost
               3.3962307 = idf(docFreq=4044, maxDocs=44421)
               0.023812415 = queryNorm
             0.18573137 = fieldWeight in 6701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3962307 = idf(docFreq=4044, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.0515444 = weight(abstract_txt:effectiveness in 6701) [ClassicSimilarity], result of:
           0.0515444 = score(doc=6701,freq=2.0), product of:
             0.13077797 = queryWeight, product of:
               1.0776737 = boost
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.023812415 = queryNorm
             0.39413673 = fieldWeight in 6701, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.017063795 = weight(abstract_txt:using in 6701) [ClassicSimilarity], result of:
           0.017063795 = score(doc=6701,freq=1.0), product of:
             0.090261884 = queryWeight, product of:
               1.0965232 = boost
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.023812415 = queryNorm
             0.18904762 = fieldWeight in 6701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.045563404 = weight(abstract_txt:were in 6701) [ClassicSimilarity], result of:
           0.045563404 = score(doc=6701,freq=5.0), product of:
             0.101595566 = queryWeight, product of:
               1.1633302 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.023812415 = queryNorm
             0.44847828 = fieldWeight in 6701, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.035259716 = weight(abstract_txt:three in 6701) [ClassicSimilarity], result of:
           0.035259716 = score(doc=6701,freq=1.0), product of:
             0.14643331 = queryWeight, product of:
               1.3966434 = boost
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.023812415 = queryNorm
             0.24079028 = fieldWeight in 6701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.02113134 = weight(abstract_txt:system in 6701) [ClassicSimilarity], result of:
           0.02113134 = score(doc=6701,freq=1.0), product of:
             0.11456491 = queryWeight, product of:
               1.426464 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.023812415 = queryNorm
             0.18444863 = fieldWeight in 6701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.057976816 = weight(abstract_txt:relevant in 6701) [ClassicSimilarity], result of:
           0.057976816 = score(doc=6701,freq=2.0), product of:
             0.16191246 = queryWeight, product of:
               1.4686075 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.023812415 = queryNorm
             0.35807508 = fieldWeight in 6701, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.03884461 = weight(abstract_txt:user in 6701) [ClassicSimilarity], result of:
           0.03884461 = score(doc=6701,freq=2.0), product of:
             0.13645123 = queryWeight, product of:
               1.5567673 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.023812415 = queryNorm
             0.28467762 = fieldWeight in 6701, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.0865481 = weight(abstract_txt:feedback in 6701) [ClassicSimilarity], result of:
           0.0865481 = score(doc=6701,freq=1.0), product of:
             0.2664561 = queryWeight, product of:
               1.8839893 = boost
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.023812415 = queryNorm
             0.32481185 = fieldWeight in 6701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.45223936 = weight(abstract_txt:expansion in 6701) [ClassicSimilarity], result of:
           0.45223936 = score(doc=6701,freq=13.0), product of:
             0.37557617 = queryWeight, product of:
               2.5827596 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.023812415 = queryNorm
             1.2041216 = fieldWeight in 6701, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
         0.1428409 = weight(abstract_txt:relevance in 6701) [ClassicSimilarity], result of:
           0.1428409 = score(doc=6701,freq=3.0), product of:
             0.3059132 = queryWeight, product of:
               2.6060882 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.023812415 = queryNorm
             0.46693277 = fieldWeight in 6701, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6701)
       0.48 = coord(12/25)
   

3. Robertson, S.E.: OKAPI at TREC-3 (1995) 0.50

   0.49612457 = sum of:
     0.49612457 = product of:
       1.2403114 = sum of:
         0.05073766 = weight(abstract_txt:query in 6694) [ClassicSimilarity], result of:
           0.05073766 = score(doc=6694,freq=1.0), product of:
             0.11382956 = queryWeight, product of:
               1.00542 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023812415 = queryNorm
             0.4457336 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.029252218 = weight(abstract_txt:using in 6694) [ClassicSimilarity], result of:
           0.029252218 = score(doc=6694,freq=1.0), product of:
             0.090261884 = queryWeight, product of:
               1.0965232 = boost
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.023812415 = queryNorm
             0.32408163 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.071271114 = weight(abstract_txt:experiments in 6694) [ClassicSimilarity], result of:
           0.071271114 = score(doc=6694,freq=1.0), product of:
             0.14277226 = queryWeight, product of:
               1.1260091 = boost
               5.324741 = idf(docFreq=587, maxDocs=44421)
               0.023812415 = queryNorm
             0.49919444 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.324741 = idf(docFreq=587, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.24639414 = weight(abstract_txt:trec in 6694) [ClassicSimilarity], result of:
           0.24639414 = score(doc=6694,freq=3.0), product of:
             0.22633304 = queryWeight, product of:
               1.4177314 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.023812415 = queryNorm
             1.0886353 = fieldWeight in 6694, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.03622516 = weight(abstract_txt:system in 6694) [ClassicSimilarity], result of:
           0.03622516 = score(doc=6694,freq=1.0), product of:
             0.11456491 = queryWeight, product of:
               1.426464 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.023812415 = queryNorm
             0.31619766 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.047086783 = weight(abstract_txt:user in 6694) [ClassicSimilarity], result of:
           0.047086783 = score(doc=6694,freq=1.0), product of:
             0.13645123 = queryWeight, product of:
               1.5567673 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.023812415 = queryNorm
             0.34508142 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.14836815 = weight(abstract_txt:feedback in 6694) [ClassicSimilarity], result of:
           0.14836815 = score(doc=6694,freq=1.0), product of:
             0.2664561 = queryWeight, product of:
               1.8839893 = boost
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.023812415 = queryNorm
             0.5568203 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.16551536 = weight(abstract_txt:ranking in 6694) [ClassicSimilarity], result of:
           0.16551536 = score(doc=6694,freq=1.0), product of:
             0.3154547 = queryWeight, product of:
               2.3670285 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.023812415 = queryNorm
             0.52468824 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.3040849 = weight(abstract_txt:expansion in 6694) [ClassicSimilarity], result of:
           0.3040849 = score(doc=6694,freq=2.0), product of:
             0.37557617 = queryWeight, product of:
               2.5827596 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.023812415 = queryNorm
             0.8096491 = fieldWeight in 6694, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
         0.14137584 = weight(abstract_txt:relevance in 6694) [ClassicSimilarity], result of:
           0.14137584 = score(doc=6694,freq=1.0), product of:
             0.3059132 = queryWeight, product of:
               2.6060882 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.023812415 = queryNorm
             0.46214363 = fieldWeight in 6694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.09375 = fieldNorm(doc=6694)
       0.4 = coord(10/25)
   

4. Xu, B.; Lin, H.; Lin, Y.: Assessment of learning to rank methods for query expansion (2016) 0.44

   0.4403391 = sum of:
     0.4403391 = product of:
       1.1008477 = sum of:
         0.047835924 = weight(abstract_txt:query in 3929) [ClassicSimilarity], result of:
           0.047835924 = score(doc=3929,freq=2.0), product of:
             0.11382956 = queryWeight, product of:
               1.00542 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023812415 = queryNorm
             0.42024165 = fieldWeight in 3929, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.01950148 = weight(abstract_txt:using in 3929) [ClassicSimilarity], result of:
           0.01950148 = score(doc=3929,freq=1.0), product of:
             0.090261884 = queryWeight, product of:
               1.0965232 = boost
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.023812415 = queryNorm
             0.21605442 = fieldWeight in 3929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.04029682 = weight(abstract_txt:three in 3929) [ClassicSimilarity], result of:
           0.04029682 = score(doc=3929,freq=1.0), product of:
             0.14643331 = queryWeight, product of:
               1.3966434 = boost
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.023812415 = queryNorm
             0.2751889 = fieldWeight in 3929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4030223 = idf(docFreq=1477, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.09483715 = weight(abstract_txt:trec in 3929) [ClassicSimilarity], result of:
           0.09483715 = score(doc=3929,freq=1.0), product of:
             0.22633304 = queryWeight, product of:
               1.4177314 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.023812415 = queryNorm
             0.41901594 = fieldWeight in 3929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.046852347 = weight(abstract_txt:relevant in 3929) [ClassicSimilarity], result of:
           0.046852347 = score(doc=3929,freq=1.0), product of:
             0.16191246 = queryWeight, product of:
               1.4686075 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.023812415 = queryNorm
             0.2893684 = fieldWeight in 3929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.074213 = weight(abstract_txt:experimental in 3929) [ClassicSimilarity], result of:
           0.074213 = score(doc=3929,freq=1.0), product of:
             0.22001183 = queryWeight, product of:
               1.7119404 = boost
               5.397019 = idf(docFreq=546, maxDocs=44421)
               0.023812415 = queryNorm
             0.33731368 = fieldWeight in 3929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.397019 = idf(docFreq=546, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.1713208 = weight(abstract_txt:feedback in 3929) [ClassicSimilarity], result of:
           0.1713208 = score(doc=3929,freq=3.0), product of:
             0.2664561 = queryWeight, product of:
               1.8839893 = boost
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.023812415 = queryNorm
             0.64296067 = fieldWeight in 3929, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.15604937 = weight(abstract_txt:ranking in 3929) [ClassicSimilarity], result of:
           0.15604937 = score(doc=3929,freq=2.0), product of:
             0.3154547 = queryWeight, product of:
               2.3670285 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.023812415 = queryNorm
             0.4946808 = fieldWeight in 3929, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.28669402 = weight(abstract_txt:expansion in 3929) [ClassicSimilarity], result of:
           0.28669402 = score(doc=3929,freq=4.0), product of:
             0.37557617 = queryWeight, product of:
               2.5827596 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.023812415 = queryNorm
             0.7633445 = fieldWeight in 3929, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
         0.16324675 = weight(abstract_txt:relevance in 3929) [ClassicSimilarity], result of:
           0.16324675 = score(doc=3929,freq=3.0), product of:
             0.3059132 = queryWeight, product of:
               2.6060882 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.023812415 = queryNorm
             0.53363746 = fieldWeight in 3929, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=3929)
       0.4 = coord(10/25)
   

5. Robertson, S.E.; Walker, S.; Hancock-Beaulieu, M.M.: Large test collection experiments of an operational, interactive system : OKAPI at TREC (1995) 0.40

   0.39716238 = sum of:
     0.39716238 = product of:
       1.1032288 = sum of:
         0.07175389 = weight(abstract_txt:query in 33) [ClassicSimilarity], result of:
           0.07175389 = score(doc=33,freq=2.0), product of:
             0.11382956 = queryWeight, product of:
               1.00542 = boost
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.023812415 = queryNorm
             0.6303625 = fieldWeight in 33, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.754492 = idf(docFreq=1039, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.02773965 = weight(abstract_txt:more in 33) [ClassicSimilarity], result of:
           0.02773965 = score(doc=33,freq=1.0), product of:
             0.08712294 = queryWeight, product of:
               1.0772882 = boost
               3.3962307 = idf(docFreq=4044, maxDocs=44421)
               0.023812415 = queryNorm
             0.31839663 = fieldWeight in 33, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3962307 = idf(docFreq=4044, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.071271114 = weight(abstract_txt:experiments in 33) [ClassicSimilarity], result of:
           0.071271114 = score(doc=33,freq=1.0), product of:
             0.14277226 = queryWeight, product of:
               1.1260091 = boost
               5.324741 = idf(docFreq=587, maxDocs=44421)
               0.023812415 = queryNorm
             0.49919444 = fieldWeight in 33, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.324741 = idf(docFreq=587, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.20117997 = weight(abstract_txt:trec in 33) [ClassicSimilarity], result of:
           0.20117997 = score(doc=33,freq=2.0), product of:
             0.22633304 = queryWeight, product of:
               1.4177314 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.023812415 = queryNorm
             0.888867 = fieldWeight in 33, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.03622516 = weight(abstract_txt:system in 33) [ClassicSimilarity], result of:
           0.03622516 = score(doc=33,freq=1.0), product of:
             0.11456491 = queryWeight, product of:
               1.426464 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.023812415 = queryNorm
             0.31619766 = fieldWeight in 33, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.07027852 = weight(abstract_txt:relevant in 33) [ClassicSimilarity], result of:
           0.07027852 = score(doc=33,freq=1.0), product of:
             0.16191246 = queryWeight, product of:
               1.4686075 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.023812415 = queryNorm
             0.4340526 = fieldWeight in 33, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.20982426 = weight(abstract_txt:feedback in 33) [ClassicSimilarity], result of:
           0.20982426 = score(doc=33,freq=2.0), product of:
             0.2664561 = queryWeight, product of:
               1.8839893 = boost
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.023812415 = queryNorm
             0.7874628 = fieldWeight in 33, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.21502051 = weight(abstract_txt:expansion in 33) [ClassicSimilarity], result of:
           0.21502051 = score(doc=33,freq=1.0), product of:
             0.37557617 = queryWeight, product of:
               2.5827596 = boost
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.023812415 = queryNorm
             0.5725084 = fieldWeight in 33, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.106756 = idf(docFreq=268, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
         0.19993563 = weight(abstract_txt:relevance in 33) [ClassicSimilarity], result of:
           0.19993563 = score(doc=33,freq=2.0), product of:
             0.3059132 = queryWeight, product of:
               2.6060882 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.023812415 = queryNorm
             0.65356976 = fieldWeight in 33, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.09375 = fieldNorm(doc=33)
       0.36 = coord(9/25)