Document (#39772)

Author

Ferro, N.
Silvello, G.
Keskustalo, H.
Pirkola, A.
Järvelin, K.

Title

¬The twist measure for IR evaluation : taking user's effort into account

Source

Journal of the Association for Information Science and Technology. 67(2016) no.3, S.620-648

Year

2016

Abstract

We present a novel measure for ranking evaluation, called Twist (t). It is a measure for informational intents, which handles both binary and graded relevance. t stems from the observation that searching is currently a that searching is currently taken for granted and it is natural for users to assume that search engines are available and work well. As a consequence, users may assume the utility they have in finding relevant documents, which is the focus of traditional measures, as granted. On the contrary, they may feel uneasy when the system returns nonrelevant documents because they are then forced to do additional work to get the desired information, and this causes avoidable effort. The latter is the focus of t, which evaluates the effectiveness of a system from the point of view of the effort required to the users to retrieve the desired information. We provide a formal definition of t, a demonstration of its properties, and introduce the notion of effort/gain plots, which complement traditional utility-based measures. By means of an extensive experimental evaluation, t is shown to grasp different aspects of system performances, to not require extensive and costly assessments, and to be a robust tool for detecting differences between systems.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23416/abstract.

Similar documents (author)

1. Pirkola, A.; Hedlund, T.; Keskustalo, H.; Järvelin, K.: Dictionary-based cross-language information retrieval : problems, methods, and research findings (2001) 2.14

   2.141234 = sum of:
     2.141234 = product of:
       3.5687232 = sum of:
         0.81685096 = weight(author_txt:järvelin in 4908) [ClassicSimilarity], result of:
           0.81685096 = score(doc=4908,freq=1.0), product of:
             0.32723552 = queryWeight, product of:
               7.9878955 = idf(docFreq=40, maxDocs=44421)
               0.040966425 = queryNorm
             2.4962173 = fieldWeight in 4908, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.9878955 = idf(docFreq=40, maxDocs=44421)
               0.3125 = fieldNorm(doc=4908)
         1.3759362 = weight(author_txt:pirkola in 4908) [ClassicSimilarity], result of:
           1.3759362 = score(doc=4908,freq=1.0), product of:
             0.46326634 = queryWeight, product of:
               1.1898307 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.040966425 = queryNorm
             2.9700758 = fieldWeight in 4908, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.3125 = fieldNorm(doc=4908)
         1.3759362 = weight(author_txt:keskustalo in 4908) [ClassicSimilarity], result of:
           1.3759362 = score(doc=4908,freq=1.0), product of:
             0.46326634 = queryWeight, product of:
               1.1898307 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.040966425 = queryNorm
             2.9700758 = fieldWeight in 4908, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.3125 = fieldNorm(doc=4908)
       0.6 = coord(3/5)
   

2. Toivonen, J.; Pirkola, A.; Keskustalo, H.; Visala, K.; Järvelin, K.: Translating cross-lingual spelling variants using transformation rules (2005) 2.14

   2.141234 = sum of:
     2.141234 = product of:
       3.5687232 = sum of:
         0.81685096 = weight(author_txt:järvelin in 2052) [ClassicSimilarity], result of:
           0.81685096 = score(doc=2052,freq=1.0), product of:
             0.32723552 = queryWeight, product of:
               7.9878955 = idf(docFreq=40, maxDocs=44421)
               0.040966425 = queryNorm
             2.4962173 = fieldWeight in 2052, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.9878955 = idf(docFreq=40, maxDocs=44421)
               0.3125 = fieldNorm(doc=2052)
         1.3759362 = weight(author_txt:pirkola in 2052) [ClassicSimilarity], result of:
           1.3759362 = score(doc=2052,freq=1.0), product of:
             0.46326634 = queryWeight, product of:
               1.1898307 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.040966425 = queryNorm
             2.9700758 = fieldWeight in 2052, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.3125 = fieldNorm(doc=2052)
         1.3759362 = weight(author_txt:keskustalo in 2052) [ClassicSimilarity], result of:
           1.3759362 = score(doc=2052,freq=1.0), product of:
             0.46326634 = queryWeight, product of:
               1.1898307 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.040966425 = queryNorm
             2.9700758 = fieldWeight in 2052, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.3125 = fieldNorm(doc=2052)
       0.6 = coord(3/5)
   

3. Ferro, N.; Silvello, G.: NESTOR: a formal model for digital archives (2013) 1.87

   1.8660634 = sum of:
     1.8660634 = product of:
       4.6651583 = sum of:
         2.284364 = weight(author_txt:silvello in 3707) [ClassicSimilarity], result of:
           2.284364 = score(doc=3707,freq=1.0), product of:
             0.47481975 = queryWeight, product of:
               1.2045759 = boost
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.040966425 = queryNorm
             4.811013 = fieldWeight in 3707, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.5 = fieldNorm(doc=3707)
         2.3807945 = weight(author_txt:ferro in 3707) [ClassicSimilarity], result of:
           2.3807945 = score(doc=3707,freq=1.0), product of:
             0.48808992 = queryWeight, product of:
               1.2212925 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.040966425 = queryNorm
             4.8777785 = fieldWeight in 3707, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.5 = fieldNorm(doc=3707)
       0.4 = coord(2/5)
   

4. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 1.87

   1.8660634 = sum of:
     1.8660634 = product of:
       4.6651583 = sum of:
         2.284364 = weight(author_txt:silvello in 35) [ClassicSimilarity], result of:
           2.284364 = score(doc=35,freq=1.0), product of:
             0.47481975 = queryWeight, product of:
               1.2045759 = boost
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.040966425 = queryNorm
             4.811013 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.5 = fieldNorm(doc=35)
         2.3807945 = weight(author_txt:ferro in 35) [ClassicSimilarity], result of:
           2.3807945 = score(doc=35,freq=1.0), product of:
             0.48808992 = queryWeight, product of:
               1.2212925 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.040966425 = queryNorm
             4.8777785 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.5 = fieldNorm(doc=35)
       0.4 = coord(2/5)
   

5. Pirkola, A.; Järvelin, K.: Employing the resolution power of search keys (2001) 1.40

   1.4033839 = sum of:
     1.4033839 = product of:
       3.5084596 = sum of:
         1.3069615 = weight(author_txt:järvelin in 6907) [ClassicSimilarity], result of:
           1.3069615 = score(doc=6907,freq=1.0), product of:
             0.32723552 = queryWeight, product of:
               7.9878955 = idf(docFreq=40, maxDocs=44421)
               0.040966425 = queryNorm
             3.9939477 = fieldWeight in 6907, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.9878955 = idf(docFreq=40, maxDocs=44421)
               0.5 = fieldNorm(doc=6907)
         2.201498 = weight(author_txt:pirkola in 6907) [ClassicSimilarity], result of:
           2.201498 = score(doc=6907,freq=1.0), product of:
             0.46326634 = queryWeight, product of:
               1.1898307 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.040966425 = queryNorm
             4.7521214 = fieldWeight in 6907, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.5 = fieldNorm(doc=6907)
       0.4 = coord(2/5)
   

Similar documents (content)

1. Hert, C.A.; Nilan, M.S.: User-based information retrieval evaluation : an examination of an online public access catalog (1991) 0.14

   0.13803202 = sum of:
     0.13803202 = product of:
       0.49297148 = sum of:
         0.045172516 = weight(abstract_txt:searching in 3670) [ClassicSimilarity], result of:
           0.045172516 = score(doc=3670,freq=1.0), product of:
             0.096355036 = queryWeight, product of:
               1.0369041 = boost
               4.2862926 = idf(docFreq=1660, maxDocs=44421)
               0.021679739 = queryNorm
             0.46881324 = fieldWeight in 3670, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.2862926 = idf(docFreq=1660, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
         0.06566809 = weight(abstract_txt:focus in 3670) [ClassicSimilarity], result of:
           0.06566809 = score(doc=3670,freq=1.0), product of:
             0.123650104 = queryWeight, product of:
               1.1746234 = boost
               4.855588 = idf(docFreq=939, maxDocs=44421)
               0.021679739 = queryNorm
             0.53107995 = fieldWeight in 3670, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.855588 = idf(docFreq=939, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
         0.046687137 = weight(abstract_txt:system in 3670) [ClassicSimilarity], result of:
           0.046687137 = score(doc=3670,freq=2.0), product of:
             0.08949048 = queryWeight, product of:
               1.2238704 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.021679739 = queryNorm
             0.5216995 = fieldWeight in 3670, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
         0.0390604 = weight(abstract_txt:users in 3670) [ClassicSimilarity], result of:
           0.0390604 = score(doc=3670,freq=1.0), product of:
             0.10011053 = queryWeight, product of:
               1.2944548 = boost
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.021679739 = queryNorm
             0.39017272 = fieldWeight in 3670, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
         0.15861584 = weight(abstract_txt:measures in 3670) [ClassicSimilarity], result of:
           0.15861584 = score(doc=3670,freq=3.0), product of:
             0.15434131 = queryWeight, product of:
               1.3123282 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.021679739 = queryNorm
             1.0276953 = fieldWeight in 3670, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
         0.028381368 = weight(abstract_txt:which in 3670) [ClassicSimilarity], result of:
           0.028381368 = score(doc=3670,freq=1.0), product of:
             0.089054756 = queryWeight, product of:
               1.4097592 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.021679739 = queryNorm
             0.3186957 = fieldWeight in 3670, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
         0.1093861 = weight(abstract_txt:evaluation in 3670) [ClassicSimilarity], result of:
           0.1093861 = score(doc=3670,freq=2.0), product of:
             0.15786533 = queryWeight, product of:
               1.6255128 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.021679739 = queryNorm
             0.6929077 = fieldWeight in 3670, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.109375 = fieldNorm(doc=3670)
       0.28 = coord(7/25)
   

2. White, H.D.: Relevance theory and distributions of judgments in document retrieval (2017) 0.13

   0.13013135 = sum of:
     0.13013135 = product of:
       0.542214 = sum of:
         0.025882756 = weight(abstract_txt:they in 99) [ClassicSimilarity], result of:
           0.025882756 = score(doc=99,freq=1.0), product of:
             0.11049799 = queryWeight, product of:
               1.3599539 = boost
               3.7477977 = idf(docFreq=2845, maxDocs=44421)
               0.021679739 = queryNorm
             0.23423736 = fieldWeight in 99, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7477977 = idf(docFreq=2845, maxDocs=44421)
               0.0625 = fieldNorm(doc=99)
         0.016217925 = weight(abstract_txt:which in 99) [ClassicSimilarity], result of:
           0.016217925 = score(doc=99,freq=1.0), product of:
             0.089054756 = queryWeight, product of:
               1.4097592 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.021679739 = queryNorm
             0.18211183 = fieldWeight in 99, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.0625 = fieldNorm(doc=99)
         0.044198662 = weight(abstract_txt:evaluation in 99) [ClassicSimilarity], result of:
           0.044198662 = score(doc=99,freq=1.0), product of:
             0.15786533 = queryWeight, product of:
               1.6255128 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.021679739 = queryNorm
             0.279977 = fieldWeight in 99, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=99)
         0.13056007 = weight(abstract_txt:assume in 99) [ClassicSimilarity], result of:
           0.13056007 = score(doc=99,freq=1.0), product of:
             0.2839164 = queryWeight, product of:
               1.7799042 = boost
               7.357662 = idf(docFreq=76, maxDocs=44421)
               0.021679739 = queryNorm
             0.4598539 = fieldWeight in 99, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.357662 = idf(docFreq=76, maxDocs=44421)
               0.0625 = fieldNorm(doc=99)
         0.07898785 = weight(abstract_txt:measure in 99) [ClassicSimilarity], result of:
           0.07898785 = score(doc=99,freq=1.0), product of:
             0.23248105 = queryWeight, product of:
               1.972608 = boost
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.021679739 = queryNorm
             0.3397604 = fieldWeight in 99, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.0625 = fieldNorm(doc=99)
         0.24636672 = weight(abstract_txt:effort in 99) [ClassicSimilarity], result of:
           0.24636672 = score(doc=99,freq=4.0), product of:
             0.34410816 = queryWeight, product of:
               2.7711744 = boost
               5.727658 = idf(docFreq=392, maxDocs=44421)
               0.021679739 = queryNorm
             0.7159572 = fieldWeight in 99, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.727658 = idf(docFreq=392, maxDocs=44421)
               0.0625 = fieldNorm(doc=99)
       0.24 = coord(6/25)
   

3. Draper, S.W.; Dunlop, M.D.: New IR - new evaluation : the impact of interaction and multimedia on information retrieval and its evaluation (1997) 0.11

   0.10994088 = sum of:
     0.10994088 = product of:
       0.34356526 = sum of:
         0.037524622 = weight(abstract_txt:focus in 3462) [ClassicSimilarity], result of:
           0.037524622 = score(doc=3462,freq=1.0), product of:
             0.123650104 = queryWeight, product of:
               1.1746234 = boost
               4.855588 = idf(docFreq=939, maxDocs=44421)
               0.021679739 = queryNorm
             0.30347425 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.855588 = idf(docFreq=939, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.018864455 = weight(abstract_txt:system in 3462) [ClassicSimilarity], result of:
           0.018864455 = score(doc=3462,freq=1.0), product of:
             0.08949048 = queryWeight, product of:
               1.2238704 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.021679739 = queryNorm
             0.21079844 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.038659766 = weight(abstract_txt:users in 3462) [ClassicSimilarity], result of:
           0.038659766 = score(doc=3462,freq=3.0), product of:
             0.10011053 = queryWeight, product of:
               1.2944548 = boost
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.021679739 = queryNorm
             0.38617083 = fieldWeight in 3462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.05265857 = weight(abstract_txt:currently in 3462) [ClassicSimilarity], result of:
           0.05265857 = score(doc=3462,freq=1.0), product of:
             0.15498737 = queryWeight, product of:
               1.315072 = boost
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.021679739 = queryNorm
             0.3397604 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.025882756 = weight(abstract_txt:they in 3462) [ClassicSimilarity], result of:
           0.025882756 = score(doc=3462,freq=1.0), product of:
             0.11049799 = queryWeight, product of:
               1.3599539 = boost
               3.7477977 = idf(docFreq=2845, maxDocs=44421)
               0.021679739 = queryNorm
             0.23423736 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7477977 = idf(docFreq=2845, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.022935608 = weight(abstract_txt:which in 3462) [ClassicSimilarity], result of:
           0.022935608 = score(doc=3462,freq=2.0), product of:
             0.089054756 = queryWeight, product of:
               1.4097592 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.021679739 = queryNorm
             0.25754502 = fieldWeight in 3462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.07048515 = weight(abstract_txt:extensive in 3462) [ClassicSimilarity], result of:
           0.07048515 = score(doc=3462,freq=1.0), product of:
             0.1882415 = queryWeight, product of:
               1.4493026 = boost
               5.9910407 = idf(docFreq=301, maxDocs=44421)
               0.021679739 = queryNorm
             0.37444004 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9910407 = idf(docFreq=301, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
         0.07655432 = weight(abstract_txt:evaluation in 3462) [ClassicSimilarity], result of:
           0.07655432 = score(doc=3462,freq=3.0), product of:
             0.15786533 = queryWeight, product of:
               1.6255128 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.021679739 = queryNorm
             0.48493436 = fieldWeight in 3462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=3462)
       0.32 = coord(8/25)
   

4. Su, L.T.: ¬The relevance of recall and precision in user evaluation (1994) 0.11

   0.10838503 = sum of:
     0.10838503 = product of:
       0.38708937 = sum of:
         0.022586258 = weight(abstract_txt:searching in 6932) [ClassicSimilarity], result of:
           0.022586258 = score(doc=6932,freq=1.0), product of:
             0.096355036 = queryWeight, product of:
               1.0369041 = boost
               4.2862926 = idf(docFreq=1660, maxDocs=44421)
               0.021679739 = queryNorm
             0.23440662 = fieldWeight in 6932, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.2862926 = idf(docFreq=1660, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
         0.029008674 = weight(abstract_txt:traditional in 6932) [ClassicSimilarity], result of:
           0.029008674 = score(doc=6932,freq=1.0), product of:
             0.11384924 = queryWeight, product of:
               1.1271104 = boost
               4.6591816 = idf(docFreq=1143, maxDocs=44421)
               0.021679739 = queryNorm
             0.254799 = fieldWeight in 6932, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6591816 = idf(docFreq=1143, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
         0.047839023 = weight(abstract_txt:users in 6932) [ClassicSimilarity], result of:
           0.047839023 = score(doc=6932,freq=6.0), product of:
             0.10011053 = queryWeight, product of:
               1.2944548 = boost
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.021679739 = queryNorm
             0.47786203 = fieldWeight in 6932, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
         0.13736534 = weight(abstract_txt:measures in 6932) [ClassicSimilarity], result of:
           0.13736534 = score(doc=6932,freq=9.0), product of:
             0.15434131 = queryWeight, product of:
               1.3123282 = boost
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.021679739 = queryNorm
             0.89001024 = fieldWeight in 6932, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.424824 = idf(docFreq=531, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
         0.014190684 = weight(abstract_txt:which in 6932) [ClassicSimilarity], result of:
           0.014190684 = score(doc=6932,freq=1.0), product of:
             0.089054756 = queryWeight, product of:
               1.4097592 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.021679739 = queryNorm
             0.15934785 = fieldWeight in 6932, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
         0.06698503 = weight(abstract_txt:evaluation in 6932) [ClassicSimilarity], result of:
           0.06698503 = score(doc=6932,freq=3.0), product of:
             0.15786533 = queryWeight, product of:
               1.6255128 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.021679739 = queryNorm
             0.42431757 = fieldWeight in 6932, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
         0.06911437 = weight(abstract_txt:measure in 6932) [ClassicSimilarity], result of:
           0.06911437 = score(doc=6932,freq=1.0), product of:
             0.23248105 = queryWeight, product of:
               1.972608 = boost
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.021679739 = queryNorm
             0.29729035 = fieldWeight in 6932, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.0546875 = fieldNorm(doc=6932)
       0.28 = coord(7/25)
   

5. Vocht, L. De: Exploring semantic relationships in the Web of Data : Semantische relaties verkennen in data op het web (2017) 0.11

   0.1051165 = sum of:
     0.1051165 = product of:
       0.29199028 = sum of:
         0.019560274 = weight(abstract_txt:searching in 232) [ClassicSimilarity], result of:
           0.019560274 = score(doc=232,freq=3.0), product of:
             0.096355036 = queryWeight, product of:
               1.0369041 = boost
               4.2862926 = idf(docFreq=1660, maxDocs=44421)
               0.021679739 = queryNorm
             0.2030021 = fieldWeight in 232, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.2862926 = idf(docFreq=1660, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.059534352 = weight(abstract_txt:intents in 232) [ClassicSimilarity], result of:
           0.059534352 = score(doc=232,freq=1.0), product of:
             0.23165053 = queryWeight, product of:
               1.1368496 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.021679739 = queryNorm
             0.2570007 = fieldWeight in 232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.023217177 = weight(abstract_txt:focus in 232) [ClassicSimilarity], result of:
           0.023217177 = score(doc=232,freq=2.0), product of:
             0.123650104 = queryWeight, product of:
               1.1746234 = boost
               4.855588 = idf(docFreq=939, maxDocs=44421)
               0.021679739 = queryNorm
             0.18776512 = fieldWeight in 232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.855588 = idf(docFreq=939, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.008253198 = weight(abstract_txt:system in 232) [ClassicSimilarity], result of:
           0.008253198 = score(doc=232,freq=1.0), product of:
             0.08949048 = queryWeight, product of:
               1.2238704 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.021679739 = queryNorm
             0.092224315 = fieldWeight in 232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.035208568 = weight(abstract_txt:users in 232) [ClassicSimilarity], result of:
           0.035208568 = score(doc=232,freq=13.0), product of:
             0.10011053 = queryWeight, product of:
               1.2944548 = boost
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.021679739 = queryNorm
             0.35169694 = fieldWeight in 232, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               3.5672934 = idf(docFreq=3408, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.029959708 = weight(abstract_txt:they in 232) [ClassicSimilarity], result of:
           0.029959708 = score(doc=232,freq=7.0), product of:
             0.11049799 = queryWeight, product of:
               1.3599539 = boost
               3.7477977 = idf(docFreq=2845, maxDocs=44421)
               0.021679739 = queryNorm
             0.2711335 = fieldWeight in 232, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.7477977 = idf(docFreq=2845, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.014190684 = weight(abstract_txt:which in 232) [ClassicSimilarity], result of:
           0.014190684 = score(doc=232,freq=4.0), product of:
             0.089054756 = queryWeight, product of:
               1.4097592 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.021679739 = queryNorm
             0.15934785 = fieldWeight in 232, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.027346525 = weight(abstract_txt:evaluation in 232) [ClassicSimilarity], result of:
           0.027346525 = score(doc=232,freq=2.0), product of:
             0.15786533 = queryWeight, product of:
               1.6255128 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.021679739 = queryNorm
             0.17322692 = fieldWeight in 232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
         0.074719794 = weight(abstract_txt:desired in 232) [ClassicSimilarity], result of:
           0.074719794 = score(doc=232,freq=2.0), product of:
             0.26953304 = queryWeight, product of:
               1.7342328 = boost
               7.168868 = idf(docFreq=92, maxDocs=44421)
               0.021679739 = queryNorm
             0.2772194 = fieldWeight in 232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.168868 = idf(docFreq=92, maxDocs=44421)
               0.02734375 = fieldNorm(doc=232)
       0.36 = coord(9/25)