Document (#42050)

Author

Angelini, M.
Fazzini, V.
Ferro, N.
Santucci, G.
Silvello, G.

Title

CLAIRE: A combinatorial visual analytics system for information retrieval evaluation

Source

Information processing and management. 54(2018) no.6, S.1077-1100

Year

2018

Abstract

Information Retrieval (IR) develops complex systems, constituted of several components, which aim at returning and optimally ranking the most relevant documents in response to user queries. In this context, experimental evaluation plays a central role, since it allows for measuring IR systems effectiveness, increasing the understanding of their functioning, and better directing the efforts for improving them. Current evaluation methodologies are limited by two major factors: (i) IR systems are evaluated as "black boxes", since it is not possible to decompose the contributions of the different components, e.g., stop lists, stemmers, and IR models; (ii) given that it is not possible to predict the effectiveness of an IR system, both academia and industry need to explore huge numbers of systems, originated by large combinatorial compositions of their components, to understand how they perform and how these components interact together. We propose a Combinatorial visuaL Analytics system for Information Retrieval Evaluation (CLAIRE) which allows for exploring and making sense of the performances of a large amount of IR systems, in order to quickly and intuitively grasp which system configurations are preferred, what are the contributions of the different components and how these components interact together. The CLAIRE system is then validated against use cases based on several test collections using a wide set of systems, generated by a combinatorial composition of several off-the-shelf components, representing the most common denominator almost always present in English IR systems. In particular, we validate the findings enabled by CLAIRE with respect to consolidated deep statistical analyses and we show that the CLAIRE system allows the generation of new insights, which were not detectable with traditional approaches.

Content

Vgl.: https://doi.org/10.1016/j.ipm.2018.04.006.

Theme

Retrievalstudien

Similar documents (author)

1. Ferro, N.; Silvello, G.: NESTOR: a formal model for digital archives (2013) 6.85

   6.8510103 = sum of:
     6.8510103 = sum of:
       3.354699 = weight(author_txt:silvello in 3707) [ClassicSimilarity], result of:
         3.354699 = score(doc=3707,freq=1.0), product of:
           0.6972957 = queryWeight, product of:
             9.622026 = idf(docFreq=7, maxDocs=44421)
             0.0724687 = queryNorm
           4.811013 = fieldWeight in 3707, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.622026 = idf(docFreq=7, maxDocs=44421)
             0.5 = fieldNorm(doc=3707)
       3.4963117 = weight(author_txt:ferro in 3707) [ClassicSimilarity], result of:
         3.4963117 = score(doc=3707,freq=1.0), product of:
           0.7167836 = queryWeight, product of:
             1.0138776 = boost
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.0724687 = queryNorm
           4.8777785 = fieldWeight in 3707, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.5 = fieldNorm(doc=3707)
   

2. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 6.85

   6.8510103 = sum of:
     6.8510103 = sum of:
       3.354699 = weight(author_txt:silvello in 35) [ClassicSimilarity], result of:
         3.354699 = score(doc=35,freq=1.0), product of:
           0.6972957 = queryWeight, product of:
             9.622026 = idf(docFreq=7, maxDocs=44421)
             0.0724687 = queryNorm
           4.811013 = fieldWeight in 35, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.622026 = idf(docFreq=7, maxDocs=44421)
             0.5 = fieldNorm(doc=35)
       3.4963117 = weight(author_txt:ferro in 35) [ClassicSimilarity], result of:
         3.4963117 = score(doc=35,freq=1.0), product of:
           0.7167836 = queryWeight, product of:
             1.0138776 = boost
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.0724687 = queryNorm
           4.8777785 = fieldWeight in 35, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.5 = fieldNorm(doc=35)
   

3. Ferro, N.; Silvello, G.; Keskustalo, H.; Pirkola, A.; Järvelin, K.: ¬The twist measure for IR evaluation : taking user's effort into account (2016) 4.28

   4.2818813 = sum of:
     4.2818813 = sum of:
       2.0966868 = weight(author_txt:silvello in 3771) [ClassicSimilarity], result of:
         2.0966868 = score(doc=3771,freq=1.0), product of:
           0.6972957 = queryWeight, product of:
             9.622026 = idf(docFreq=7, maxDocs=44421)
             0.0724687 = queryNorm
           3.0068831 = fieldWeight in 3771, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.622026 = idf(docFreq=7, maxDocs=44421)
             0.3125 = fieldNorm(doc=3771)
       2.1851947 = weight(author_txt:ferro in 3771) [ClassicSimilarity], result of:
         2.1851947 = score(doc=3771,freq=1.0), product of:
           0.7167836 = queryWeight, product of:
             1.0138776 = boost
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.0724687 = queryNorm
           3.0486116 = fieldWeight in 3771, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.3125 = fieldNorm(doc=3771)
   

4. Silvello, G.: Learning to cite framework : how to automatically construct citations for hierarchical data (2017) 2.10

   2.0966868 = sum of:
     2.0966868 = product of:
       4.1933737 = sum of:
         4.1933737 = weight(author_txt:silvello in 4647) [ClassicSimilarity], result of:
           4.1933737 = score(doc=4647,freq=1.0), product of:
             0.6972957 = queryWeight, product of:
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.0724687 = queryNorm
             6.0137663 = fieldWeight in 4647, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.625 = fieldNorm(doc=4647)
       0.5 = coord(1/2)
   

5. Silvello, G.: Theory and practice of data citation (2018) 2.10

   2.0966868 = sum of:
     2.0966868 = product of:
       4.1933737 = sum of:
         4.1933737 = weight(author_txt:silvello in 6) [ClassicSimilarity], result of:
           4.1933737 = score(doc=6,freq=1.0), product of:
             0.6972957 = queryWeight, product of:
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.0724687 = queryNorm
             6.0137663 = fieldWeight in 6, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.625 = fieldNorm(doc=6)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 0.27

   0.26782852 = sum of:
     0.26782852 = product of:
       0.7439681 = sum of:
         0.0875382 = weight(abstract_txt:stemmers in 35) [ClassicSimilarity], result of:
           0.0875382 = score(doc=35,freq=1.0), product of:
             0.15455173 = queryWeight, product of:
               1.014955 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.016802866 = queryNorm
             0.56640065 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.020966671 = weight(abstract_txt:retrieval in 35) [ClassicSimilarity], result of:
           0.020966671 = score(doc=35,freq=2.0), product of:
             0.06823265 = queryWeight, product of:
               1.1680639 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.016802866 = queryNorm
             0.3072821 = fieldWeight in 35, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.059097268 = weight(abstract_txt:together in 35) [ClassicSimilarity], result of:
           0.059097268 = score(doc=35,freq=3.0), product of:
             0.103901856 = queryWeight, product of:
               1.1768917 = boost
               5.254162 = idf(docFreq=630, maxDocs=44421)
               0.016802866 = queryNorm
             0.5687797 = fieldWeight in 35, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.254162 = idf(docFreq=630, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.011638604 = weight(abstract_txt:which in 35) [ClassicSimilarity], result of:
           0.011638604 = score(doc=35,freq=1.0), product of:
             0.063909106 = queryWeight, product of:
               1.3053327 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.016802866 = queryNorm
             0.18211183 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.06640015 = weight(abstract_txt:interact in 35) [ClassicSimilarity], result of:
           0.06640015 = score(doc=35,freq=1.0), product of:
             0.16195641 = queryWeight, product of:
               1.4693455 = boost
               6.559804 = idf(docFreq=170, maxDocs=44421)
               0.016802866 = queryNorm
             0.40998775 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.559804 = idf(docFreq=170, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.073251076 = weight(abstract_txt:evaluation in 35) [ClassicSimilarity], result of:
           0.073251076 = score(doc=35,freq=3.0), product of:
             0.1510536 = queryWeight, product of:
               2.006806 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.016802866 = queryNorm
             0.48493436 = fieldWeight in 35, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.046896506 = weight(abstract_txt:system in 35) [ClassicSimilarity], result of:
           0.046896506 = score(doc=35,freq=3.0), product of:
             0.1284436 = queryWeight, product of:
               2.2664268 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.016802866 = queryNorm
             0.36511362 = fieldWeight in 35, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.056602698 = weight(abstract_txt:systems in 35) [ClassicSimilarity], result of:
           0.056602698 = score(doc=35,freq=3.0), product of:
             0.15328246 = queryWeight, product of:
               2.6742692 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.016802866 = queryNorm
             0.36927053 = fieldWeight in 35, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
         0.32157695 = weight(abstract_txt:components in 35) [ClassicSimilarity], result of:
           0.32157695 = score(doc=35,freq=5.0), product of:
             0.41163102 = queryWeight, product of:
               4.382409 = boost
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.016802866 = queryNorm
             0.7812263 = fieldWeight in 35, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.0625 = fieldNorm(doc=35)
       0.36 = coord(9/25)
   

2. Keller, K.; Breuer, T.; Schaer, P.: Evaluation of temporal change in IR test collections (2024) 0.16

   0.15913945 = sum of:
     0.15913945 = product of:
       0.5683552 = sum of:
         0.05392481 = weight(abstract_txt:effectiveness in 2344) [ClassicSimilarity], result of:
           0.05392481 = score(doc=2344,freq=3.0), product of:
             0.097747155 = queryWeight, product of:
               1.1415026 = boost
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.016802866 = queryNorm
             0.5516765 = fieldWeight in 2344, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.041933343 = weight(abstract_txt:retrieval in 2344) [ClassicSimilarity], result of:
           0.041933343 = score(doc=2344,freq=8.0), product of:
             0.06823265 = queryWeight, product of:
               1.1680639 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.016802866 = queryNorm
             0.6145642 = fieldWeight in 2344, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.057718646 = weight(abstract_txt:allows in 2344) [ClassicSimilarity], result of:
           0.057718646 = score(doc=2344,freq=1.0), product of:
             0.16885994 = queryWeight, product of:
               1.8375273 = boost
               5.4690194 = idf(docFreq=508, maxDocs=44421)
               0.016802866 = queryNorm
             0.3418137 = fieldWeight in 2344, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4690194 = idf(docFreq=508, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.073251076 = weight(abstract_txt:evaluation in 2344) [ClassicSimilarity], result of:
           0.073251076 = score(doc=2344,freq=3.0), product of:
             0.1510536 = queryWeight, product of:
               2.006806 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.016802866 = queryNorm
             0.48493436 = fieldWeight in 2344, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.02707571 = weight(abstract_txt:system in 2344) [ClassicSimilarity], result of:
           0.02707571 = score(doc=2344,freq=1.0), product of:
             0.1284436 = queryWeight, product of:
               2.2664268 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.016802866 = queryNorm
             0.21079844 = fieldWeight in 2344, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.06535916 = weight(abstract_txt:systems in 2344) [ClassicSimilarity], result of:
           0.06535916 = score(doc=2344,freq=4.0), product of:
             0.15328246 = queryWeight, product of:
               2.6742692 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.016802866 = queryNorm
             0.42639688 = fieldWeight in 2344, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.24909244 = weight(abstract_txt:components in 2344) [ClassicSimilarity], result of:
           0.24909244 = score(doc=2344,freq=3.0), product of:
             0.41163102 = queryWeight, product of:
               4.382409 = boost
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.016802866 = queryNorm
             0.60513526 = fieldWeight in 2344, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
       0.28 = coord(7/25)
   

3. Greengrass, M.: Conflation methods for searching databases of Latin text (1996) 0.12

   0.123671934 = sum of:
     0.123671934 = product of:
       0.51529974 = sum of:
         0.1313073 = weight(abstract_txt:stemmers in 56) [ClassicSimilarity], result of:
           0.1313073 = score(doc=56,freq=1.0), product of:
             0.15455173 = queryWeight, product of:
               1.014955 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.016802866 = queryNorm
             0.849601 = fieldWeight in 56, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.09375 = fieldNorm(doc=56)
         0.031450007 = weight(abstract_txt:retrieval in 56) [ClassicSimilarity], result of:
           0.031450007 = score(doc=56,freq=2.0), product of:
             0.06823265 = queryWeight, product of:
               1.1680639 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.016802866 = queryNorm
             0.46092314 = fieldWeight in 56, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=56)
         0.017457906 = weight(abstract_txt:which in 56) [ClassicSimilarity], result of:
           0.017457906 = score(doc=56,freq=1.0), product of:
             0.063909106 = queryWeight, product of:
               1.3053327 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.016802866 = queryNorm
             0.27316773 = fieldWeight in 56, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.09375 = fieldNorm(doc=56)
         0.07034476 = weight(abstract_txt:system in 56) [ClassicSimilarity], result of:
           0.07034476 = score(doc=56,freq=3.0), product of:
             0.1284436 = queryWeight, product of:
               2.2664268 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.016802866 = queryNorm
             0.5476704 = fieldWeight in 56, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.09375 = fieldNorm(doc=56)
         0.049019374 = weight(abstract_txt:systems in 56) [ClassicSimilarity], result of:
           0.049019374 = score(doc=56,freq=1.0), product of:
             0.15328246 = queryWeight, product of:
               2.6742692 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.016802866 = queryNorm
             0.31979766 = fieldWeight in 56, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.09375 = fieldNorm(doc=56)
         0.21572039 = weight(abstract_txt:components in 56) [ClassicSimilarity], result of:
           0.21572039 = score(doc=56,freq=1.0), product of:
             0.41163102 = queryWeight, product of:
               4.382409 = boost
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.016802866 = queryNorm
             0.5240625 = fieldWeight in 56, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.09375 = fieldNorm(doc=56)
       0.24 = coord(6/25)
   

4. Progress in visual information access and retrieval (1999) 0.12

   0.1154258 = sum of:
     0.1154258 = product of:
       0.3206272 = sum of:
         0.01697477 = weight(abstract_txt:since in 964) [ClassicSimilarity], result of:
           0.01697477 = score(doc=964,freq=2.0), product of:
             0.08984474 = queryWeight, product of:
               1.0943877 = boost
               4.8858275 = idf(docFreq=911, maxDocs=44421)
               0.016802866 = queryNorm
             0.18893448 = fieldWeight in 964, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.8858275 = idf(docFreq=911, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.031775925 = weight(abstract_txt:retrieval in 964) [ClassicSimilarity], result of:
           0.031775925 = score(doc=964,freq=24.0), product of:
             0.06823265 = queryWeight, product of:
               1.1680639 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.016802866 = queryNorm
             0.4656997 = fieldWeight in 964, product of:
               4.8989797 = tf(freq=24.0), with freq of:
                 24.0 = termFreq=24.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.083173394 = weight(abstract_txt:visual in 964) [ClassicSimilarity], result of:
           0.083173394 = score(doc=964,freq=21.0), product of:
             0.11836317 = queryWeight, product of:
               1.2561258 = boost
               5.6078978 = idf(docFreq=442, maxDocs=44421)
               0.016802866 = queryNorm
             0.70269656 = fieldWeight in 964, product of:
               4.582576 = tf(freq=21.0), with freq of:
                 21.0 = termFreq=21.0
               5.6078978 = idf(docFreq=442, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.011385811 = weight(abstract_txt:which in 964) [ClassicSimilarity], result of:
           0.011385811 = score(doc=964,freq=5.0), product of:
             0.063909106 = queryWeight, product of:
               1.3053327 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.016802866 = queryNorm
             0.17815632 = fieldWeight in 964, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.02242884 = weight(abstract_txt:contributions in 964) [ClassicSimilarity], result of:
           0.02242884 = score(doc=964,freq=1.0), product of:
             0.13630275 = queryWeight, product of:
               1.3479605 = boost
               6.017888 = idf(docFreq=293, maxDocs=44421)
               0.016802866 = queryNorm
             0.16455163 = fieldWeight in 964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.017888 = idf(docFreq=293, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.0145260105 = weight(abstract_txt:several in 964) [ClassicSimilarity], result of:
           0.0145260105 = score(doc=964,freq=1.0), product of:
             0.11679603 = queryWeight, product of:
               1.5282152 = boost
               4.548416 = idf(docFreq=1277, maxDocs=44421)
               0.016802866 = queryNorm
             0.124370754 = fieldWeight in 964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.548416 = idf(docFreq=1277, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.03700509 = weight(abstract_txt:evaluation in 964) [ClassicSimilarity], result of:
           0.03700509 = score(doc=964,freq=4.0), product of:
             0.1510536 = queryWeight, product of:
               2.006806 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.016802866 = queryNorm
             0.24497987 = fieldWeight in 964, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.04043892 = weight(abstract_txt:systems in 964) [ClassicSimilarity], result of:
           0.04043892 = score(doc=964,freq=8.0), product of:
             0.15328246 = queryWeight, product of:
               2.6742692 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.016802866 = queryNorm
             0.2638196 = fieldWeight in 964, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
         0.06291845 = weight(abstract_txt:components in 964) [ClassicSimilarity], result of:
           0.06291845 = score(doc=964,freq=1.0), product of:
             0.41163102 = queryWeight, product of:
               4.382409 = boost
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.016802866 = queryNorm
             0.15285157 = fieldWeight in 964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.02734375 = fieldNorm(doc=964)
       0.36 = coord(9/25)
   

5. Tague-Sutcliffe, J.M.: Some perspectives on the evaluation of information retrieval systems (1996) 0.11

   0.11375923 = sum of:
     0.11375923 = product of:
       0.4739968 = sum of:
         0.03486542 = weight(abstract_txt:possible in 4231) [ClassicSimilarity], result of:
           0.03486542 = score(doc=4231,freq=1.0), product of:
             0.080442935 = queryWeight, product of:
               1.0355446 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.016802866 = queryNorm
             0.43341807 = fieldWeight in 4231, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.09375 = fieldNorm(doc=4231)
         0.044477027 = weight(abstract_txt:retrieval in 4231) [ClassicSimilarity], result of:
           0.044477027 = score(doc=4231,freq=4.0), product of:
             0.06823265 = queryWeight, product of:
               1.1680639 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.016802866 = queryNorm
             0.6518438 = fieldWeight in 4231, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=4231)
         0.017457906 = weight(abstract_txt:which in 4231) [ClassicSimilarity], result of:
           0.017457906 = score(doc=4231,freq=1.0), product of:
             0.063909106 = queryWeight, product of:
               1.3053327 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.016802866 = queryNorm
             0.27316773 = fieldWeight in 4231, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.09375 = fieldNorm(doc=4231)
         0.0634373 = weight(abstract_txt:evaluation in 4231) [ClassicSimilarity], result of:
           0.0634373 = score(doc=4231,freq=1.0), product of:
             0.1510536 = queryWeight, product of:
               2.006806 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.016802866 = queryNorm
             0.4199655 = fieldWeight in 4231, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=4231)
         0.09803875 = weight(abstract_txt:systems in 4231) [ClassicSimilarity], result of:
           0.09803875 = score(doc=4231,freq=4.0), product of:
             0.15328246 = queryWeight, product of:
               2.6742692 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.016802866 = queryNorm
             0.6395953 = fieldWeight in 4231, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.09375 = fieldNorm(doc=4231)
         0.21572039 = weight(abstract_txt:components in 4231) [ClassicSimilarity], result of:
           0.21572039 = score(doc=4231,freq=1.0), product of:
             0.41163102 = queryWeight, product of:
               4.382409 = boost
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.016802866 = queryNorm
             0.5240625 = fieldWeight in 4231, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.59 = idf(docFreq=450, maxDocs=44421)
               0.09375 = fieldNorm(doc=4231)
       0.24 = coord(6/25)