Document (#24504)

Author

Bruijn, B. de
Holte, R.
Martin, J.

Title

¬An automated method for studying interactive systems

Source

Knowledge: creation, organization and use. Proceedings of the 62nd Annual Meeting of the American Society for Information Science, 31.10.-4.11.1999. Ed.: L. Woods

Imprint

Medford, NJ : Information Today

Year

1999

Pages

S.751-762

Series

Proceedings of the American Society for Information Science; vol.36

Abstract

Information Retrieval experiments rarely examine more than a small number of user or system characteristics because of the limited availability of human subjects. In this article we present an interaction model, and based on that an experimental method. This automated method helps identify which user and system variables are relevant, which are independent of other variables, and which ranges of the variables are most important. In an automated method there are no human subjects: the human's role is performed by a computer program. This restricts the user variables that can be studied to those whose effect on behaviour can be accurately modeled. To illustrate the automated method, the following experiment will be presented. The subject's task is to hunt down a set of ten similar documents by providing relevance feedback and using query refinement. Performance is measured by precision in the top 10 documents at the end of a trial. Six user and system variables are varied. The experiment runs 600 different combinations of variable values on 50 tasks. This highlights one of the main advantages of an automated study - it can be much larger scale than is normally possible with human subjects. The results of this automated experiment show, as human studies have done previously, that relevance feedback consistently improves retrieval performance. Further, they show that in the context of the present task, the effect saturates after the second iteration. Irrelevant and interacting variables are identified. These observations give specific guidance about designing a follow-up study using human subjects

Similar documents (author)

1. Martin, G.: Report on the Dewey Decimal Classification editorial Policy Committee (EPC) (1994) 4.58

   4.5825124 = sum of:
     4.5825124 = weight(author_txt:martin in 3417) [ClassicSimilarity], result of:
       4.5825124 = fieldWeight in 3417, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.33202 = idf(docFreq=78, maxDocs=44421)
         0.625 = fieldNorm(doc=3417)
   

2. Martin, R.G.: Design considerations for an OPAC workstation : an introduction to specifications and a model configuration (1989) 4.58

   4.5825124 = sum of:
     4.5825124 = weight(author_txt:martin in 4074) [ClassicSimilarity], result of:
       4.5825124 = fieldWeight in 4074, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.33202 = idf(docFreq=78, maxDocs=44421)
         0.625 = fieldNorm(doc=4074)
   

3. Martin, R.G.: OPAC workstation evaluation : the third time around (1992) 4.58

   4.5825124 = sum of:
     4.5825124 = weight(author_txt:martin in 4218) [ClassicSimilarity], result of:
       4.5825124 = fieldWeight in 4218, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.33202 = idf(docFreq=78, maxDocs=44421)
         0.625 = fieldNorm(doc=4218)
   

4. Martin, B.: Knowledge management (2008) 4.58

   4.5825124 = sum of:
     4.5825124 = weight(author_txt:martin in 4229) [ClassicSimilarity], result of:
       4.5825124 = fieldWeight in 4229, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.33202 = idf(docFreq=78, maxDocs=44421)
         0.625 = fieldNorm(doc=4229)
   

5. Martin, M.: Compact disc media evaluation : what we now know about disc quality (1993) 4.58

   4.5825124 = sum of:
     4.5825124 = weight(author_txt:martin in 4526) [ClassicSimilarity], result of:
       4.5825124 = fieldWeight in 4526, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.33202 = idf(docFreq=78, maxDocs=44421)
         0.625 = fieldNorm(doc=4526)
   

Similar documents (content)

1. Sun, Y.; Kantor, P.B.; Morse, E.L.: Using cross-evaluation to evaluate interactive QA systems (2011) 0.30

   0.30311355 = sum of:
     0.30311355 = product of:
       0.6314866 = sum of:
         0.029561521 = weight(abstract_txt:show in 744) [ClassicSimilarity], result of:
           0.029561521 = score(doc=744,freq=1.0), product of:
             0.08597771 = queryWeight, product of:
               1.1180216 = boost
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.017473698 = queryNorm
             0.34382772 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.034192603 = weight(abstract_txt:performance in 744) [ClassicSimilarity], result of:
           0.034192603 = score(doc=744,freq=1.0), product of:
             0.09473769 = queryWeight, product of:
               1.1735961 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.017473698 = queryNorm
             0.36091867 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.012974091 = weight(abstract_txt:that in 744) [ClassicSimilarity], result of:
           0.012974091 = score(doc=744,freq=2.0), product of:
             0.049653817 = queryWeight, product of:
               1.2015686 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017473698 = queryNorm
             0.2612909 = fieldWeight in 744, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.04094431 = weight(abstract_txt:task in 744) [ClassicSimilarity], result of:
           0.04094431 = score(doc=744,freq=1.0), product of:
             0.10683095 = queryWeight, product of:
               1.2462518 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017473698 = queryNorm
             0.38326263 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.041542366 = weight(abstract_txt:relevance in 744) [ClassicSimilarity], result of:
           0.041542366 = score(doc=744,freq=1.0), product of:
             0.10786872 = queryWeight, product of:
               1.2522904 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.017473698 = queryNorm
             0.38511968 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.019958487 = weight(abstract_txt:system in 744) [ClassicSimilarity], result of:
           0.019958487 = score(doc=744,freq=1.0), product of:
             0.07574434 = queryWeight, product of:
               1.2852215 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017473698 = queryNorm
             0.26349807 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.05519248 = weight(abstract_txt:effect in 744) [ClassicSimilarity], result of:
           0.05519248 = score(doc=744,freq=1.0), product of:
             0.13036309 = queryWeight, product of:
               1.376685 = boost
               5.419201 = idf(docFreq=534, maxDocs=44421)
               0.017473698 = queryNorm
             0.42337507 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.419201 = idf(docFreq=534, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.020921305 = weight(abstract_txt:this in 744) [ClassicSimilarity], result of:
           0.020921305 = score(doc=744,freq=3.0), product of:
             0.06425399 = queryWeight, product of:
               1.5281899 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.017473698 = queryNorm
             0.3256032 = fieldWeight in 744, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.034590356 = weight(abstract_txt:user in 744) [ClassicSimilarity], result of:
           0.034590356 = score(doc=744,freq=1.0), product of:
             0.1202859 = queryWeight, product of:
               1.8701637 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.017473698 = queryNorm
             0.28756785 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.09426415 = weight(abstract_txt:experiment in 744) [ClassicSimilarity], result of:
           0.09426415 = score(doc=744,freq=1.0), product of:
             0.21322078 = queryWeight, product of:
               2.156342 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017473698 = queryNorm
             0.44209647 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.11061398 = weight(abstract_txt:subjects in 744) [ClassicSimilarity], result of:
           0.11061398 = score(doc=744,freq=1.0), product of:
             0.26108667 = queryWeight, product of:
               2.7552729 = boost
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.017473698 = queryNorm
             0.42366767 = fieldWeight in 744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
         0.13673092 = weight(abstract_txt:method in 744) [ClassicSimilarity], result of:
           0.13673092 = score(doc=744,freq=3.0), product of:
             0.22460467 = queryWeight, product of:
               2.8571737 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.017473698 = queryNorm
             0.6087626 = fieldWeight in 744, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.078125 = fieldNorm(doc=744)
       0.48 = coord(12/25)
   

2. Jacko, J.A.; Salvendy, G.; Koubek, R.J.: Modelling of menu design in computerized work (1995) 0.30

   0.29921183 = sum of:
     0.29921183 = product of:
       0.831144 = sum of:
         0.034192603 = weight(abstract_txt:performance in 3460) [ClassicSimilarity], result of:
           0.034192603 = score(doc=3460,freq=1.0), product of:
             0.09473769 = queryWeight, product of:
               1.1735961 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.017473698 = queryNorm
             0.36091867 = fieldWeight in 3460, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.009174067 = weight(abstract_txt:that in 3460) [ClassicSimilarity], result of:
           0.009174067 = score(doc=3460,freq=1.0), product of:
             0.049653817 = queryWeight, product of:
               1.2015686 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017473698 = queryNorm
             0.18476056 = fieldWeight in 3460, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.09155426 = weight(abstract_txt:task in 3460) [ClassicSimilarity], result of:
           0.09155426 = score(doc=3460,freq=5.0), product of:
             0.10683095 = queryWeight, product of:
               1.2462518 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017473698 = queryNorm
             0.8570013 = fieldWeight in 3460, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.019958487 = weight(abstract_txt:system in 3460) [ClassicSimilarity], result of:
           0.019958487 = score(doc=3460,freq=1.0), product of:
             0.07574434 = queryWeight, product of:
               1.2852215 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017473698 = queryNorm
             0.26349807 = fieldWeight in 3460, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.012078921 = weight(abstract_txt:this in 3460) [ClassicSimilarity], result of:
           0.012078921 = score(doc=3460,freq=1.0), product of:
             0.06425399 = queryWeight, product of:
               1.5281899 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.017473698 = queryNorm
             0.18798709 = fieldWeight in 3460, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.04891815 = weight(abstract_txt:user in 3460) [ClassicSimilarity], result of:
           0.04891815 = score(doc=3460,freq=2.0), product of:
             0.1202859 = queryWeight, product of:
               1.8701637 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.017473698 = queryNorm
             0.40668234 = fieldWeight in 3460, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.09426415 = weight(abstract_txt:experiment in 3460) [ClassicSimilarity], result of:
           0.09426415 = score(doc=3460,freq=1.0), product of:
             0.21322078 = queryWeight, product of:
               2.156342 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017473698 = queryNorm
             0.44209647 = fieldWeight in 3460, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.1564318 = weight(abstract_txt:subjects in 3460) [ClassicSimilarity], result of:
           0.1564318 = score(doc=3460,freq=2.0), product of:
             0.26108667 = queryWeight, product of:
               2.7552729 = boost
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.017473698 = queryNorm
             0.59915656 = fieldWeight in 3460, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
         0.36457157 = weight(abstract_txt:variables in 3460) [ClassicSimilarity], result of:
           0.36457157 = score(doc=3460,freq=2.0), product of:
             0.52535594 = queryWeight, product of:
               4.7867937 = boost
               6.2809324 = idf(docFreq=225, maxDocs=44421)
               0.017473698 = queryNorm
             0.69395155 = fieldWeight in 3460, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.2809324 = idf(docFreq=225, maxDocs=44421)
               0.078125 = fieldNorm(doc=3460)
       0.36 = coord(9/25)
   

3. Zhang, X.; Liu, J.; Cole, M.; Belkin, N.: Predicting users' domain knowledge in information retrieval using multiple regression analysis of search behaviors (2015) 0.25

   0.24977747 = sum of:
     0.24977747 = product of:
       0.62444365 = sum of:
         0.027505586 = weight(abstract_txt:documents in 2822) [ClassicSimilarity], result of:
           0.027505586 = score(doc=2822,freq=2.0), product of:
             0.075470746 = queryWeight, product of:
               1.047482 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.017473698 = queryNorm
             0.3644536 = fieldWeight in 2822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.010295083 = weight(abstract_txt:which in 2822) [ClassicSimilarity], result of:
           0.010295083 = score(doc=2822,freq=1.0), product of:
             0.056531653 = queryWeight, product of:
               1.1103214 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.017473698 = queryNorm
             0.18211183 = fieldWeight in 2822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.0103792725 = weight(abstract_txt:that in 2822) [ClassicSimilarity], result of:
           0.0103792725 = score(doc=2822,freq=2.0), product of:
             0.049653817 = queryWeight, product of:
               1.2015686 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017473698 = queryNorm
             0.20903271 = fieldWeight in 2822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.0463232 = weight(abstract_txt:task in 2822) [ClassicSimilarity], result of:
           0.0463232 = score(doc=2822,freq=2.0), product of:
             0.10683095 = queryWeight, product of:
               1.2462518 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017473698 = queryNorm
             0.43361217 = fieldWeight in 2822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.015966788 = weight(abstract_txt:system in 2822) [ClassicSimilarity], result of:
           0.015966788 = score(doc=2822,freq=1.0), product of:
             0.07574434 = queryWeight, product of:
               1.2852215 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017473698 = queryNorm
             0.21079844 = fieldWeight in 2822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.009663137 = weight(abstract_txt:this in 2822) [ClassicSimilarity], result of:
           0.009663137 = score(doc=2822,freq=1.0), product of:
             0.06425399 = queryWeight, product of:
               1.5281899 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.017473698 = queryNorm
             0.15038967 = fieldWeight in 2822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.047929805 = weight(abstract_txt:user in 2822) [ClassicSimilarity], result of:
           0.047929805 = score(doc=2822,freq=3.0), product of:
             0.1202859 = queryWeight, product of:
               1.8701637 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.017473698 = queryNorm
             0.3984657 = fieldWeight in 2822, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.07541132 = weight(abstract_txt:experiment in 2822) [ClassicSimilarity], result of:
           0.07541132 = score(doc=2822,freq=1.0), product of:
             0.21322078 = queryWeight, product of:
               2.156342 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017473698 = queryNorm
             0.35367718 = fieldWeight in 2822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.08931226 = weight(abstract_txt:method in 2822) [ClassicSimilarity], result of:
           0.08931226 = score(doc=2822,freq=2.0), product of:
             0.22460467 = queryWeight, product of:
               2.8571737 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.017473698 = queryNorm
             0.39764208 = fieldWeight in 2822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
         0.29165724 = weight(abstract_txt:variables in 2822) [ClassicSimilarity], result of:
           0.29165724 = score(doc=2822,freq=2.0), product of:
             0.52535594 = queryWeight, product of:
               4.7867937 = boost
               6.2809324 = idf(docFreq=225, maxDocs=44421)
               0.017473698 = queryNorm
             0.55516124 = fieldWeight in 2822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.2809324 = idf(docFreq=225, maxDocs=44421)
               0.0625 = fieldNorm(doc=2822)
       0.4 = coord(10/25)
   

4. Kim, S.H.; Eastman, C.M.: ¬An experiment on node size in a hypermedia system (1999) 0.24

   0.23763953 = sum of:
     0.23763953 = product of:
       0.74262357 = sum of:
         0.015442623 = weight(abstract_txt:which in 4673) [ClassicSimilarity], result of:
           0.015442623 = score(doc=4673,freq=1.0), product of:
             0.056531653 = queryWeight, product of:
               1.1103214 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.017473698 = queryNorm
             0.27316773 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.041031126 = weight(abstract_txt:performance in 4673) [ClassicSimilarity], result of:
           0.041031126 = score(doc=4673,freq=1.0), product of:
             0.09473769 = queryWeight, product of:
               1.1735961 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.017473698 = queryNorm
             0.43310243 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.011008881 = weight(abstract_txt:that in 4673) [ClassicSimilarity], result of:
           0.011008881 = score(doc=4673,freq=1.0), product of:
             0.049653817 = queryWeight, product of:
               1.2015686 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017473698 = queryNorm
             0.22171268 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.04913317 = weight(abstract_txt:task in 4673) [ClassicSimilarity], result of:
           0.04913317 = score(doc=4673,freq=1.0), product of:
             0.10683095 = queryWeight, product of:
               1.2462518 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017473698 = queryNorm
             0.45991516 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.023950182 = weight(abstract_txt:system in 4673) [ClassicSimilarity], result of:
           0.023950182 = score(doc=4673,freq=1.0), product of:
             0.07574434 = queryWeight, product of:
               1.2852215 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017473698 = queryNorm
             0.31619766 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.15997158 = weight(abstract_txt:experiment in 4673) [ClassicSimilarity], result of:
           0.15997158 = score(doc=4673,freq=2.0), product of:
             0.21322078 = queryWeight, product of:
               2.156342 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017473698 = queryNorm
             0.7502626 = fieldWeight in 4673, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.13273677 = weight(abstract_txt:subjects in 4673) [ClassicSimilarity], result of:
           0.13273677 = score(doc=4673,freq=1.0), product of:
             0.26108667 = queryWeight, product of:
               2.7552729 = boost
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.017473698 = queryNorm
             0.50840116 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.422946 = idf(docFreq=532, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
         0.3093492 = weight(abstract_txt:variables in 4673) [ClassicSimilarity], result of:
           0.3093492 = score(doc=4673,freq=1.0), product of:
             0.52535594 = queryWeight, product of:
               4.7867937 = boost
               6.2809324 = idf(docFreq=225, maxDocs=44421)
               0.017473698 = queryNorm
             0.5888374 = fieldWeight in 4673, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2809324 = idf(docFreq=225, maxDocs=44421)
               0.09375 = fieldNorm(doc=4673)
       0.32 = coord(8/25)
   

5. Jacucci, G.; Barral, O.; Daee, P.; Wenzel, M.; Serim, B.; Ruotsalo, T.; Pluchino, P.; Freeman, J.; Gamberini, L.; Kaski, S.; Blankertz, B.: Integrating neurophysiologic relevance feedback in intent modeling for information retrieval (2019) 0.23

   0.22935507 = sum of:
     0.22935507 = product of:
       0.5212615 = sum of:
         0.019449387 = weight(abstract_txt:documents in 356) [ClassicSimilarity], result of:
           0.019449387 = score(doc=356,freq=1.0), product of:
             0.075470746 = queryWeight, product of:
               1.047482 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.017473698 = queryNorm
             0.25770763 = fieldWeight in 356, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.022770962 = weight(abstract_txt:present in 356) [ClassicSimilarity], result of:
           0.022770962 = score(doc=356,freq=1.0), product of:
             0.08383569 = queryWeight, product of:
               1.1040068 = boost
               4.3458266 = idf(docFreq=1564, maxDocs=44421)
               0.017473698 = queryNorm
             0.27161416 = fieldWeight in 356, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.3458266 = idf(docFreq=1564, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.023649216 = weight(abstract_txt:show in 356) [ClassicSimilarity], result of:
           0.023649216 = score(doc=356,freq=1.0), product of:
             0.08597771 = queryWeight, product of:
               1.1180216 = boost
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.017473698 = queryNorm
             0.27506217 = fieldWeight in 356, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.04737866 = weight(abstract_txt:performance in 356) [ClassicSimilarity], result of:
           0.04737866 = score(doc=356,freq=3.0), product of:
             0.09473769 = queryWeight, product of:
               1.1735961 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.017473698 = queryNorm
             0.5001036 = fieldWeight in 356, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.012711959 = weight(abstract_txt:that in 356) [ClassicSimilarity], result of:
           0.012711959 = score(doc=356,freq=3.0), product of:
             0.049653817 = queryWeight, product of:
               1.2015686 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.017473698 = queryNorm
             0.25601172 = fieldWeight in 356, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.0463232 = weight(abstract_txt:task in 356) [ClassicSimilarity], result of:
           0.0463232 = score(doc=356,freq=2.0), product of:
             0.10683095 = queryWeight, product of:
               1.2462518 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017473698 = queryNorm
             0.43361217 = fieldWeight in 356, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.07431325 = weight(abstract_txt:relevance in 356) [ClassicSimilarity], result of:
           0.07431325 = score(doc=356,freq=5.0), product of:
             0.10786872 = queryWeight, product of:
               1.2522904 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.017473698 = queryNorm
             0.68892306 = fieldWeight in 356, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.015966788 = weight(abstract_txt:system in 356) [ClassicSimilarity], result of:
           0.015966788 = score(doc=356,freq=1.0), product of:
             0.07574434 = queryWeight, product of:
               1.2852215 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017473698 = queryNorm
             0.21079844 = fieldWeight in 356, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.14238726 = weight(abstract_txt:feedback in 356) [ClassicSimilarity], result of:
           0.14238726 = score(doc=356,freq=6.0), product of:
             0.15659276 = queryWeight, product of:
               1.5088397 = boost
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.017473698 = queryNorm
             0.90928376 = fieldWeight in 356, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.9394164 = idf(docFreq=317, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.009663137 = weight(abstract_txt:this in 356) [ClassicSimilarity], result of:
           0.009663137 = score(doc=356,freq=1.0), product of:
             0.06425399 = queryWeight, product of:
               1.5281899 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.017473698 = queryNorm
             0.15038967 = fieldWeight in 356, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
         0.106647715 = weight(abstract_txt:experiment in 356) [ClassicSimilarity], result of:
           0.106647715 = score(doc=356,freq=2.0), product of:
             0.21322078 = queryWeight, product of:
               2.156342 = boost
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.017473698 = queryNorm
             0.50017506 = fieldWeight in 356, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.658835 = idf(docFreq=420, maxDocs=44421)
               0.0625 = fieldNorm(doc=356)
       0.44 = coord(11/25)