Document (#39772)

Author

Ferro, N.
Silvello, G.
Keskustalo, H.
Pirkola, A.
Järvelin, K.

Title

¬The twist measure for IR evaluation : taking user's effort into account

Source

Journal of the Association for Information Science and Technology. 67(2016) no.3, S.620-648

Year

2016

Abstract

We present a novel measure for ranking evaluation, called Twist (t). It is a measure for informational intents, which handles both binary and graded relevance. t stems from the observation that searching is currently a that searching is currently taken for granted and it is natural for users to assume that search engines are available and work well. As a consequence, users may assume the utility they have in finding relevant documents, which is the focus of traditional measures, as granted. On the contrary, they may feel uneasy when the system returns nonrelevant documents because they are then forced to do additional work to get the desired information, and this causes avoidable effort. The latter is the focus of t, which evaluates the effectiveness of a system from the point of view of the effort required to the users to retrieve the desired information. We provide a formal definition of t, a demonstration of its properties, and introduce the notion of effort/gain plots, which complement traditional utility-based measures. By means of an extensive experimental evaluation, t is shown to grasp different aspects of system performances, to not require extensive and costly assessments, and to be a robust tool for detecting differences between systems.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23416/abstract.

Similar documents (author)

1. Pirkola, A.; Hedlund, T.; Keskustalo, H.; Järvelin, K.: Dictionary-based cross-language information retrieval : problems, methods, and research findings (2001) 2.16

   2.1594896 = sum of:
     2.1594896 = product of:
       3.5991492 = sum of:
         0.8118522 = weight(author_txt:järvelin in 3908) [ClassicSimilarity], result of:
           0.8118522 = score(doc=3908,freq=1.0), product of:
             0.32541957 = queryWeight, product of:
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.04076246 = queryNorm
             2.494786 = fieldWeight in 3908, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.3125 = fieldNorm(doc=3908)
         1.3678917 = weight(author_txt:pirkola in 3908) [ClassicSimilarity], result of:
           1.3678917 = score(doc=3908,freq=1.0), product of:
             0.46077988 = queryWeight, product of:
               1.1899396 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.04076246 = queryNorm
             2.9686446 = fieldWeight in 3908, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.3125 = fieldNorm(doc=3908)
         1.4194055 = weight(author_txt:keskustalo in 3908) [ClassicSimilarity], result of:
           1.4194055 = score(doc=3908,freq=1.0), product of:
             0.47227687 = queryWeight, product of:
               1.2046933 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.04076246 = queryNorm
             3.005452 = fieldWeight in 3908, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.3125 = fieldNorm(doc=3908)
       0.6 = coord(3/5)
   

2. Toivonen, J.; Pirkola, A.; Keskustalo, H.; Visala, K.; Järvelin, K.: Translating cross-lingual spelling variants using transformation rules (2005) 2.16

   2.1594896 = sum of:
     2.1594896 = product of:
       3.5991492 = sum of:
         0.8118522 = weight(author_txt:järvelin in 1052) [ClassicSimilarity], result of:
           0.8118522 = score(doc=1052,freq=1.0), product of:
             0.32541957 = queryWeight, product of:
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.04076246 = queryNorm
             2.494786 = fieldWeight in 1052, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.3125 = fieldNorm(doc=1052)
         1.3678917 = weight(author_txt:pirkola in 1052) [ClassicSimilarity], result of:
           1.3678917 = score(doc=1052,freq=1.0), product of:
             0.46077988 = queryWeight, product of:
               1.1899396 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.04076246 = queryNorm
             2.9686446 = fieldWeight in 1052, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.3125 = fieldNorm(doc=1052)
         1.4194055 = weight(author_txt:keskustalo in 1052) [ClassicSimilarity], result of:
           1.4194055 = score(doc=1052,freq=1.0), product of:
             0.47227687 = queryWeight, product of:
               1.2046933 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.04076246 = queryNorm
             3.005452 = fieldWeight in 1052, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.3125 = fieldNorm(doc=1052)
       0.6 = coord(3/5)
   

3. Ferro, N.; Silvello, G.: NESTOR: a formal model for digital archives (2013) 1.86

   1.8552049 = sum of:
     1.8552049 = product of:
       4.6380124 = sum of:
         2.2710485 = weight(author_txt:silvello in 2707) [ClassicSimilarity], result of:
           2.2710485 = score(doc=2707,freq=1.0), product of:
             0.47227687 = queryWeight, product of:
               1.2046933 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.04076246 = queryNorm
             4.808723 = fieldWeight in 2707, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.5 = fieldNorm(doc=2707)
         2.3669639 = weight(author_txt:ferro in 2707) [ClassicSimilarity], result of:
           2.3669639 = score(doc=2707,freq=1.0), product of:
             0.48548236 = queryWeight, product of:
               1.2214196 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.04076246 = queryNorm
             4.8754888 = fieldWeight in 2707, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.5 = fieldNorm(doc=2707)
       0.4 = coord(2/5)
   

4. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 1.86

   1.8552049 = sum of:
     1.8552049 = product of:
       4.6380124 = sum of:
         2.2710485 = weight(author_txt:silvello in 4035) [ClassicSimilarity], result of:
           2.2710485 = score(doc=4035,freq=1.0), product of:
             0.47227687 = queryWeight, product of:
               1.2046933 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.04076246 = queryNorm
             4.808723 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.5 = fieldNorm(doc=4035)
         2.3669639 = weight(author_txt:ferro in 4035) [ClassicSimilarity], result of:
           2.3669639 = score(doc=4035,freq=1.0), product of:
             0.48548236 = queryWeight, product of:
               1.2214196 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.04076246 = queryNorm
             4.8754888 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.5 = fieldNorm(doc=4035)
       0.4 = coord(2/5)
   

5. Pirkola, A.; Järvelin, K.: Employing the resolution power of search keys (2001) 1.40

   1.395036 = sum of:
     1.395036 = product of:
       3.4875898 = sum of:
         1.2989634 = weight(author_txt:järvelin in 5907) [ClassicSimilarity], result of:
           1.2989634 = score(doc=5907,freq=1.0), product of:
             0.32541957 = queryWeight, product of:
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.04076246 = queryNorm
             3.9916575 = fieldWeight in 5907, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.5 = fieldNorm(doc=5907)
         2.1886265 = weight(author_txt:pirkola in 5907) [ClassicSimilarity], result of:
           2.1886265 = score(doc=5907,freq=1.0), product of:
             0.46077988 = queryWeight, product of:
               1.1899396 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.04076246 = queryNorm
             4.749831 = fieldWeight in 5907, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.5 = fieldNorm(doc=5907)
       0.4 = coord(2/5)
   

Similar documents (content)

1. Hert, C.A.; Nilan, M.S.: User-based information retrieval evaluation : an examination of an online public access catalog (1991) 0.14

   0.13828196 = sum of:
     0.13828196 = product of:
       0.49386412 = sum of:
         0.045056947 = weight(abstract_txt:searching in 3671) [ClassicSimilarity], result of:
           0.045056947 = score(doc=3671,freq=1.0), product of:
             0.09614363 = queryWeight, product of:
               1.0371001 = boost
               4.284727 = idf(docFreq=1655, maxDocs=44218)
               0.021635987 = queryNorm
             0.46864203 = fieldWeight in 3671, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.284727 = idf(docFreq=1655, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
         0.06568921 = weight(abstract_txt:focus in 3671) [ClassicSimilarity], result of:
           0.06568921 = score(doc=3671,freq=1.0), product of:
             0.1236162 = queryWeight, product of:
               1.175975 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.021635987 = queryNorm
             0.53139645 = fieldWeight in 3671, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
         0.046599407 = weight(abstract_txt:system in 3671) [ClassicSimilarity], result of:
           0.046599407 = score(doc=3671,freq=2.0), product of:
             0.08933467 = queryWeight, product of:
               1.2243794 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.021635987 = queryNorm
             0.52162737 = fieldWeight in 3671, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
         0.039084733 = weight(abstract_txt:users in 3671) [ClassicSimilarity], result of:
           0.039084733 = score(doc=3671,freq=1.0), product of:
             0.10010318 = queryWeight, product of:
               1.2960744 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.021635987 = queryNorm
             0.39044446 = fieldWeight in 3671, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
         0.15931122 = weight(abstract_txt:measures in 3671) [ClassicSimilarity], result of:
           0.15931122 = score(doc=3671,freq=3.0), product of:
             0.15471646 = queryWeight, product of:
               1.3156145 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.021635987 = queryNorm
             1.0296979 = fieldWeight in 3671, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
         0.028425237 = weight(abstract_txt:which in 3671) [ClassicSimilarity], result of:
           0.028425237 = score(doc=3671,freq=1.0), product of:
             0.089102946 = queryWeight, product of:
               1.4119568 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.021635987 = queryNorm
             0.31901568 = fieldWeight in 3671, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
         0.10969739 = weight(abstract_txt:evaluation in 3671) [ClassicSimilarity], result of:
           0.10969739 = score(doc=3671,freq=2.0), product of:
             0.15808742 = queryWeight, product of:
               1.6287509 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.021635987 = queryNorm
             0.6939033 = fieldWeight in 3671, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.109375 = fieldNorm(doc=3671)
       0.28 = coord(7/25)
   

2. White, H.D.: Relevance theory and distributions of judgments in document retrieval (2017) 0.13

   0.13016778 = sum of:
     0.13016778 = product of:
       0.5423658 = sum of:
         0.02593272 = weight(abstract_txt:they in 5099) [ClassicSimilarity], result of:
           0.02593272 = score(doc=5099,freq=1.0), product of:
             0.11058609 = queryWeight, product of:
               1.3622483 = boost
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.021635987 = queryNorm
             0.23450254 = fieldWeight in 5099, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.0625 = fieldNorm(doc=5099)
         0.016242992 = weight(abstract_txt:which in 5099) [ClassicSimilarity], result of:
           0.016242992 = score(doc=5099,freq=1.0), product of:
             0.089102946 = queryWeight, product of:
               1.4119568 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.021635987 = queryNorm
             0.18229467 = fieldWeight in 5099, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.0625 = fieldNorm(doc=5099)
         0.04432444 = weight(abstract_txt:evaluation in 5099) [ClassicSimilarity], result of:
           0.04432444 = score(doc=5099,freq=1.0), product of:
             0.15808742 = queryWeight, product of:
               1.6287509 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.021635987 = queryNorm
             0.2803793 = fieldWeight in 5099, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=5099)
         0.1315277 = weight(abstract_txt:assume in 5099) [ClassicSimilarity], result of:
           0.1315277 = score(doc=5099,freq=1.0), product of:
             0.28517812 = queryWeight, product of:
               1.7861524 = boost
               7.3793993 = idf(docFreq=74, maxDocs=44218)
               0.021635987 = queryNorm
             0.46121246 = fieldWeight in 5099, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3793993 = idf(docFreq=74, maxDocs=44218)
               0.0625 = fieldNorm(doc=5099)
         0.078921765 = weight(abstract_txt:measure in 5099) [ClassicSimilarity], result of:
           0.078921765 = score(doc=5099,freq=1.0), product of:
             0.23223786 = queryWeight, product of:
               1.9741154 = boost
               5.437306 = idf(docFreq=522, maxDocs=44218)
               0.021635987 = queryNorm
             0.33983162 = fieldWeight in 5099, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.437306 = idf(docFreq=522, maxDocs=44218)
               0.0625 = fieldNorm(doc=5099)
         0.24541616 = weight(abstract_txt:effort in 5099) [ClassicSimilarity], result of:
           0.24541616 = score(doc=5099,freq=4.0), product of:
             0.3430548 = queryWeight, product of:
               2.7704935 = boost
               5.723078 = idf(docFreq=392, maxDocs=44218)
               0.021635987 = queryNorm
             0.7153847 = fieldWeight in 5099, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.723078 = idf(docFreq=392, maxDocs=44218)
               0.0625 = fieldNorm(doc=5099)
       0.24 = coord(6/25)
   

3. Draper, S.W.; Dunlop, M.D.: New IR - new evaluation : the impact of interaction and multimedia on information retrieval and its evaluation (1997) 0.11

   0.11026086 = sum of:
     0.11026086 = product of:
       0.34456518 = sum of:
         0.037536692 = weight(abstract_txt:focus in 2462) [ClassicSimilarity], result of:
           0.037536692 = score(doc=2462,freq=1.0), product of:
             0.1236162 = queryWeight, product of:
               1.175975 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.021635987 = queryNorm
             0.30365512 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.018829003 = weight(abstract_txt:system in 2462) [ClassicSimilarity], result of:
           0.018829003 = score(doc=2462,freq=1.0), product of:
             0.08933467 = queryWeight, product of:
               1.2243794 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.021635987 = queryNorm
             0.21076928 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.03868385 = weight(abstract_txt:users in 2462) [ClassicSimilarity], result of:
           0.03868385 = score(doc=2462,freq=3.0), product of:
             0.10010318 = queryWeight, product of:
               1.2960744 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.021635987 = queryNorm
             0.3864398 = fieldWeight in 2462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.05267008 = weight(abstract_txt:currently in 2462) [ClassicSimilarity], result of:
           0.05267008 = score(doc=2462,freq=1.0), product of:
             0.15493423 = queryWeight, product of:
               1.31654 = boost
               5.4392195 = idf(docFreq=521, maxDocs=44218)
               0.021635987 = queryNorm
             0.33995122 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4392195 = idf(docFreq=521, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.02593272 = weight(abstract_txt:they in 2462) [ClassicSimilarity], result of:
           0.02593272 = score(doc=2462,freq=1.0), product of:
             0.11058609 = queryWeight, product of:
               1.3622483 = boost
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.021635987 = queryNorm
             0.23450254 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.02297106 = weight(abstract_txt:which in 2462) [ClassicSimilarity], result of:
           0.02297106 = score(doc=2462,freq=2.0), product of:
             0.089102946 = queryWeight, product of:
               1.4119568 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.021635987 = queryNorm
             0.2578036 = fieldWeight in 2462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.071169585 = weight(abstract_txt:extensive in 2462) [ClassicSimilarity], result of:
           0.071169585 = score(doc=2462,freq=1.0), product of:
             0.18936557 = queryWeight, product of:
               1.4554956 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.021635987 = queryNorm
             0.37583172 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
         0.076772176 = weight(abstract_txt:evaluation in 2462) [ClassicSimilarity], result of:
           0.076772176 = score(doc=2462,freq=3.0), product of:
             0.15808742 = queryWeight, product of:
               1.6287509 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.021635987 = queryNorm
             0.48563117 = fieldWeight in 2462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=2462)
       0.32 = coord(8/25)
   

4. Su, L.T.: ¬The relevance of recall and precision in user evaluation (1994) 0.11

   0.10855334 = sum of:
     0.10855334 = product of:
       0.3876905 = sum of:
         0.022528473 = weight(abstract_txt:searching in 6933) [ClassicSimilarity], result of:
           0.022528473 = score(doc=6933,freq=1.0), product of:
             0.09614363 = queryWeight, product of:
               1.0371001 = boost
               4.284727 = idf(docFreq=1655, maxDocs=44218)
               0.021635987 = queryNorm
             0.23432101 = fieldWeight in 6933, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.284727 = idf(docFreq=1655, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
         0.02888083 = weight(abstract_txt:traditional in 6933) [ClassicSimilarity], result of:
           0.02888083 = score(doc=6933,freq=1.0), product of:
             0.11345905 = queryWeight, product of:
               1.1266265 = boost
               4.654601 = idf(docFreq=1143, maxDocs=44218)
               0.021635987 = queryNorm
             0.2545485 = fieldWeight in 6933, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.654601 = idf(docFreq=1143, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
         0.04786883 = weight(abstract_txt:users in 6933) [ClassicSimilarity], result of:
           0.04786883 = score(doc=6933,freq=6.0), product of:
             0.10010318 = queryWeight, product of:
               1.2960744 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.021635987 = queryNorm
             0.4781949 = fieldWeight in 6933, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
         0.13796757 = weight(abstract_txt:measures in 6933) [ClassicSimilarity], result of:
           0.13796757 = score(doc=6933,freq=9.0), product of:
             0.15471646 = queryWeight, product of:
               1.3156145 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.021635987 = queryNorm
             0.8917446 = fieldWeight in 6933, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
         0.014212619 = weight(abstract_txt:which in 6933) [ClassicSimilarity], result of:
           0.014212619 = score(doc=6933,freq=1.0), product of:
             0.089102946 = queryWeight, product of:
               1.4119568 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.021635987 = queryNorm
             0.15950784 = fieldWeight in 6933, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
         0.06717566 = weight(abstract_txt:evaluation in 6933) [ClassicSimilarity], result of:
           0.06717566 = score(doc=6933,freq=3.0), product of:
             0.15808742 = queryWeight, product of:
               1.6287509 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.021635987 = queryNorm
             0.42492726 = fieldWeight in 6933, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
         0.06905655 = weight(abstract_txt:measure in 6933) [ClassicSimilarity], result of:
           0.06905655 = score(doc=6933,freq=1.0), product of:
             0.23223786 = queryWeight, product of:
               1.9741154 = boost
               5.437306 = idf(docFreq=522, maxDocs=44218)
               0.021635987 = queryNorm
             0.29735267 = fieldWeight in 6933, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.437306 = idf(docFreq=522, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6933)
       0.28 = coord(7/25)
   

5. Vocht, L. De: Exploring semantic relationships in the Web of Data : Semantische relaties verkennen in data op het web (2017) 0.11

   0.1057338 = sum of:
     0.1057338 = product of:
       0.293705 = sum of:
         0.01951023 = weight(abstract_txt:searching in 4232) [ClassicSimilarity], result of:
           0.01951023 = score(doc=4232,freq=3.0), product of:
             0.09614363 = queryWeight, product of:
               1.0371001 = boost
               4.284727 = idf(docFreq=1655, maxDocs=44218)
               0.021635987 = queryNorm
             0.20292795 = fieldWeight in 4232, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.284727 = idf(docFreq=1655, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.061379988 = weight(abstract_txt:intents in 4232) [ClassicSimilarity], result of:
           0.061379988 = score(doc=4232,freq=1.0), product of:
             0.23629828 = queryWeight, product of:
               1.1496766 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.021635987 = queryNorm
             0.2597564 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.023224643 = weight(abstract_txt:focus in 4232) [ClassicSimilarity], result of:
           0.023224643 = score(doc=4232,freq=2.0), product of:
             0.1236162 = queryWeight, product of:
               1.175975 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.021635987 = queryNorm
             0.18787701 = fieldWeight in 4232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.008237689 = weight(abstract_txt:system in 4232) [ClassicSimilarity], result of:
           0.008237689 = score(doc=4232,freq=1.0), product of:
             0.08933467 = queryWeight, product of:
               1.2243794 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.021635987 = queryNorm
             0.09221156 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.035230502 = weight(abstract_txt:users in 4232) [ClassicSimilarity], result of:
           0.035230502 = score(doc=4232,freq=13.0), product of:
             0.10010318 = queryWeight, product of:
               1.2960744 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.021635987 = queryNorm
             0.35194188 = fieldWeight in 4232, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.030017542 = weight(abstract_txt:they in 4232) [ClassicSimilarity], result of:
           0.030017542 = score(doc=4232,freq=7.0), product of:
             0.11058609 = queryWeight, product of:
               1.3622483 = boost
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.021635987 = queryNorm
             0.27144048 = fieldWeight in 4232, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.014212619 = weight(abstract_txt:which in 4232) [ClassicSimilarity], result of:
           0.014212619 = score(doc=4232,freq=4.0), product of:
             0.089102946 = queryWeight, product of:
               1.4119568 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.021635987 = queryNorm
             0.15950784 = fieldWeight in 4232, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.027424347 = weight(abstract_txt:evaluation in 4232) [ClassicSimilarity], result of:
           0.027424347 = score(doc=4232,freq=2.0), product of:
             0.15808742 = queryWeight, product of:
               1.6287509 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.021635987 = queryNorm
             0.17347583 = fieldWeight in 4232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
         0.074467406 = weight(abstract_txt:desired in 4232) [ClassicSimilarity], result of:
           0.074467406 = score(doc=4232,freq=2.0), product of:
             0.2687944 = queryWeight, product of:
               1.7340854 = boost
               7.1642876 = idf(docFreq=92, maxDocs=44218)
               0.021635987 = queryNorm
             0.27704227 = fieldWeight in 4232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.1642876 = idf(docFreq=92, maxDocs=44218)
               0.02734375 = fieldNorm(doc=4232)
       0.36 = coord(9/25)