Document (#32964)

Author

Tsai, C.-F.
McGarry, K.
Tait, J.

Title

Qualitative evaluation of automatic assignment of keywords to images

Source

Information processing and management. 42(2006) no.1, S.136-154

Year

2006

Abstract

In image retrieval, most systems lack user-centred evaluation since they are assessed by some chosen ground truth dataset. The results reported through precision and recall assessed against the ground truth are thought of as being an acceptable surrogate for the judgment of real users. Much current research focuses on automatically assigning keywords to images for enhancing retrieval effectiveness. However, evaluation methods are usually based on system-level assessment, e.g. classification accuracy based on some chosen ground truth dataset. In this paper, we present a qualitative evaluation methodology for automatic image indexing systems. The automatic indexing task is formulated as one of image annotation, or automatic metadata generation for images. The evaluation is composed of two individual methods. First, the automatic indexing annotation results are assessed by human subjects. Second, the subjects are asked to annotate some chosen images as the test set whose annotations are used as ground truth. Then, the system is tested by the test set whose annotation results are judged against the ground truth. Only one of these methods is reported for most systems on which user-centred evaluation are conducted. We believe that both methods need to be considered for full evaluation. We also provide an example evaluation of our system based on this methodology. According to this study, our proposed evaluation methodology is able to provide deeper understanding of the system's performance.

Theme

Automatisches Indexieren

Form

Bilder

Similar documents (author)

1. Tait, J.: CALS and its implications for the library and information retrieval communities (1994) 1.21

   1.2096583 = sum of:
     1.2096583 = product of:
       3.6289747 = sum of:
         3.6289747 = weight(author_txt:tait in 1661) [ClassicSimilarity], result of:
           3.6289747 = score(doc=1661,freq=1.0), product of:
             0.61807245 = queryWeight, product of:
               1.0733342 = boost
               9.394302 = idf(docFreq=9, maxDocs=44218)
               0.061297093 = queryNorm
             5.871439 = fieldWeight in 1661, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.394302 = idf(docFreq=9, maxDocs=44218)
               0.625 = fieldNorm(doc=1661)
       0.33333334 = coord(1/3)
   

2. Tait, J.A.: Authors and titles : an analytical study of the author concept in codes of cataloguing rules in the English language, from that of the British Museum in 1841 to the Anglo-American Cataloguing Rules 1967 (1969) 1.21

   1.2096583 = sum of:
     1.2096583 = product of:
       3.6289747 = sum of:
         3.6289747 = weight(author_txt:tait in 2161) [ClassicSimilarity], result of:
           3.6289747 = score(doc=2161,freq=1.0), product of:
             0.61807245 = queryWeight, product of:
               1.0733342 = boost
               9.394302 = idf(docFreq=9, maxDocs=44218)
               0.061297093 = queryNorm
             5.871439 = fieldWeight in 2161, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.394302 = idf(docFreq=9, maxDocs=44218)
               0.625 = fieldNorm(doc=2161)
       0.33333334 = coord(1/3)
   

3. McGarry, D.: Report from the section on classification and indexing : 1989-90 (1990) 0.98

   0.9782678 = sum of:
     0.9782678 = product of:
       2.9348032 = sum of:
         2.9348032 = weight(author_txt:mcgarry in 475) [ClassicSimilarity], result of:
           2.9348032 = score(doc=475,freq=1.0), product of:
             0.5364996 = queryWeight, product of:
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.061297093 = queryNorm
             5.47028 = fieldWeight in 475, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.625 = fieldNorm(doc=475)
       0.33333334 = coord(1/3)
   

4. McGarry, D.: Displays of bibliographic records in call number order : functions of the displays and data elements needed (1992) 0.98

   0.9782678 = sum of:
     0.9782678 = product of:
       2.9348032 = sum of:
         2.9348032 = weight(author_txt:mcgarry in 2384) [ClassicSimilarity], result of:
           2.9348032 = score(doc=2384,freq=1.0), product of:
             0.5364996 = queryWeight, product of:
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.061297093 = queryNorm
             5.47028 = fieldWeight in 2384, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.625 = fieldNorm(doc=2384)
       0.33333334 = coord(1/3)
   

5. McGarry, D.: Report from the section on classification and indexing : 1991-1992 (1992) 0.98

   0.9782678 = sum of:
     0.9782678 = product of:
       2.9348032 = sum of:
         2.9348032 = weight(author_txt:mcgarry in 2608) [ClassicSimilarity], result of:
           2.9348032 = score(doc=2608,freq=1.0), product of:
             0.5364996 = queryWeight, product of:
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.061297093 = queryNorm
             5.47028 = fieldWeight in 2608, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.625 = fieldNorm(doc=2608)
       0.33333334 = coord(1/3)
   

Similar documents (content)

1. Ostermann, F.O.; Tomko, M.; Purves, R.: User evaluation of automatically generated keywords and toponyms for geo-referenced images (2013) 0.34

   0.33556506 = sum of:
     0.33556506 = product of:
       0.8389126 = sum of:
         0.01945607 = weight(abstract_txt:results in 663) [ClassicSimilarity], result of:
           0.01945607 = score(doc=663,freq=2.0), product of:
             0.06320897 = queryWeight, product of:
               1.035078 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017535742 = queryNorm
             0.30780554 = fieldWeight in 663, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.016207168 = weight(abstract_txt:some in 663) [ClassicSimilarity], result of:
           0.016207168 = score(doc=663,freq=1.0), product of:
             0.07050556 = queryWeight, product of:
               1.0931895 = boost
               3.6779325 = idf(docFreq=3037, maxDocs=44218)
               0.017535742 = queryNorm
             0.22987078 = fieldWeight in 663, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6779325 = idf(docFreq=3037, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.05180366 = weight(abstract_txt:qualitative in 663) [ClassicSimilarity], result of:
           0.05180366 = score(doc=663,freq=2.0), product of:
             0.10607633 = queryWeight, product of:
               1.0948317 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.017535742 = queryNorm
             0.4883621 = fieldWeight in 663, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.0944441 = weight(abstract_txt:keywords in 663) [ClassicSimilarity], result of:
           0.0944441 = score(doc=663,freq=4.0), product of:
             0.1256468 = queryWeight, product of:
               1.1915543 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.017535742 = queryNorm
             0.75166345 = fieldWeight in 663, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.087584555 = weight(abstract_txt:image in 663) [ClassicSimilarity], result of:
           0.087584555 = score(doc=663,freq=3.0), product of:
             0.15054391 = queryWeight, product of:
               1.5974063 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.017535742 = queryNorm
             0.5817874 = fieldWeight in 663, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.043799963 = weight(abstract_txt:methods in 663) [ClassicSimilarity], result of:
           0.043799963 = score(doc=663,freq=2.0), product of:
             0.11950076 = queryWeight, product of:
               1.6433818 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.017535742 = queryNorm
             0.36652455 = fieldWeight in 663, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.11289123 = weight(abstract_txt:annotation in 663) [ClassicSimilarity], result of:
           0.11289123 = score(doc=663,freq=1.0), product of:
             0.25715327 = queryWeight, product of:
               2.0877562 = boost
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.017535742 = queryNorm
             0.43900365 = fieldWeight in 663, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.18376085 = weight(abstract_txt:images in 663) [ClassicSimilarity], result of:
           0.18376085 = score(doc=663,freq=7.0), product of:
             0.20473912 = queryWeight, product of:
               2.151065 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.017535742 = queryNorm
             0.8975366 = fieldWeight in 663, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.076146714 = weight(abstract_txt:automatic in 663) [ClassicSimilarity], result of:
           0.076146714 = score(doc=663,freq=1.0), product of:
             0.2344961 = queryWeight, product of:
               2.573807 = boost
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.017535742 = queryNorm
             0.32472485 = fieldWeight in 663, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
         0.15281835 = weight(abstract_txt:evaluation in 663) [ClassicSimilarity], result of:
           0.15281835 = score(doc=663,freq=3.0), product of:
             0.31467986 = queryWeight, product of:
               4.000174 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.017535742 = queryNorm
             0.48563117 = fieldWeight in 663, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=663)
       0.4 = coord(10/25)
   

2. Hu, X.; Lee, J.H.; Bainbridge, D.; Choi, K.; Organisciak, P.; Downie, J.S.: ¬The MIREX grand challenge : a framework of holistic user-experience evaluation in music information retrieval (2017) 0.26

   0.25880858 = sum of:
     0.25880858 = product of:
       0.9243164 = sum of:
         0.022085274 = weight(abstract_txt:system in 3321) [ClassicSimilarity], result of:
           0.022085274 = score(doc=3321,freq=2.0), product of:
             0.059274845 = queryWeight, product of:
               1.002349 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.017535742 = queryNorm
             0.372591 = fieldWeight in 3321, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
         0.036163583 = weight(abstract_txt:systems in 3321) [ClassicSimilarity], result of:
           0.036163583 = score(doc=3321,freq=5.0), product of:
             0.060674038 = queryWeight, product of:
               1.0141103 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.017535742 = queryNorm
             0.5960306 = fieldWeight in 3321, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
         0.0171969 = weight(abstract_txt:results in 3321) [ClassicSimilarity], result of:
           0.0171969 = score(doc=3321,freq=1.0), product of:
             0.06320897 = queryWeight, product of:
               1.035078 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017535742 = queryNorm
             0.27206424 = fieldWeight in 3321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
         0.053725652 = weight(abstract_txt:against in 3321) [ClassicSimilarity], result of:
           0.053725652 = score(doc=3321,freq=1.0), product of:
             0.1180056 = queryWeight, product of:
               1.1547539 = boost
               5.8275905 = idf(docFreq=353, maxDocs=44218)
               0.017535742 = queryNorm
             0.4552805 = fieldWeight in 3321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8275905 = idf(docFreq=353, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
         0.23063225 = weight(abstract_txt:ground in 3321) [ClassicSimilarity], result of:
           0.23063225 = score(doc=3321,freq=1.0), product of:
             0.4230336 = queryWeight, product of:
               3.4569697 = boost
               6.9783883 = idf(docFreq=111, maxDocs=44218)
               0.017535742 = queryNorm
             0.5451866 = fieldWeight in 3321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9783883 = idf(docFreq=111, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
         0.2727204 = weight(abstract_txt:truth in 3321) [ClassicSimilarity], result of:
           0.2727204 = score(doc=3321,freq=1.0), product of:
             0.47304952 = queryWeight, product of:
               3.6556234 = boost
               7.3793993 = idf(docFreq=74, maxDocs=44218)
               0.017535742 = queryNorm
             0.57651556 = fieldWeight in 3321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3793993 = idf(docFreq=74, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
         0.29179236 = weight(abstract_txt:evaluation in 3321) [ClassicSimilarity], result of:
           0.29179236 = score(doc=3321,freq=7.0), product of:
             0.31467986 = queryWeight, product of:
               4.000174 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.017535742 = queryNorm
             0.9272673 = fieldWeight in 3321, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=3321)
       0.28 = coord(7/25)
   

3. Névéol, A.; Deserno, T.M.; Darmoni, S.J.; Güld, M.O.; Aronson, A.R.: Natural language processing versus content-based image analysis for medical document retrieval (2009) 0.20

   0.19621712 = sum of:
     0.19621712 = product of:
       0.5450475 = sum of:
         0.027912794 = weight(abstract_txt:test in 2702) [ClassicSimilarity], result of:
           0.027912794 = score(doc=2702,freq=1.0), product of:
             0.088496014 = queryWeight, product of:
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.017535742 = queryNorm
             0.315413 = fieldWeight in 2702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.012493316 = weight(abstract_txt:system in 2702) [ClassicSimilarity], result of:
           0.012493316 = score(doc=2702,freq=1.0), product of:
             0.059274845 = queryWeight, product of:
               1.002349 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.017535742 = queryNorm
             0.21076928 = fieldWeight in 2702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.012938277 = weight(abstract_txt:systems in 2702) [ClassicSimilarity], result of:
           0.012938277 = score(doc=2702,freq=1.0), product of:
             0.060674038 = queryWeight, product of:
               1.0141103 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.017535742 = queryNorm
             0.2132424 = fieldWeight in 2702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.04643021 = weight(abstract_txt:indexing in 2702) [ClassicSimilarity], result of:
           0.04643021 = score(doc=2702,freq=3.0), product of:
             0.09860796 = queryWeight, product of:
               1.292825 = boost
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.017535742 = queryNorm
             0.47085664 = fieldWeight in 2702, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.15170091 = weight(abstract_txt:image in 2702) [ClassicSimilarity], result of:
           0.15170091 = score(doc=2702,freq=9.0), product of:
             0.15054391 = queryWeight, product of:
               1.5974063 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.017535742 = queryNorm
             1.0076854 = fieldWeight in 2702, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.030971251 = weight(abstract_txt:methods in 2702) [ClassicSimilarity], result of:
           0.030971251 = score(doc=2702,freq=1.0), product of:
             0.11950076 = queryWeight, product of:
               1.6433818 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.017535742 = queryNorm
             0.259172 = fieldWeight in 2702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.09822431 = weight(abstract_txt:images in 2702) [ClassicSimilarity], result of:
           0.09822431 = score(doc=2702,freq=2.0), product of:
             0.20473912 = queryWeight, product of:
               2.151065 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.017535742 = queryNorm
             0.4797535 = fieldWeight in 2702, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.076146714 = weight(abstract_txt:automatic in 2702) [ClassicSimilarity], result of:
           0.076146714 = score(doc=2702,freq=1.0), product of:
             0.2344961 = queryWeight, product of:
               2.573807 = boost
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.017535742 = queryNorm
             0.32472485 = fieldWeight in 2702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
         0.088229716 = weight(abstract_txt:evaluation in 2702) [ClassicSimilarity], result of:
           0.088229716 = score(doc=2702,freq=1.0), product of:
             0.31467986 = queryWeight, product of:
               4.000174 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.017535742 = queryNorm
             0.2803793 = fieldWeight in 2702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=2702)
       0.36 = coord(9/25)
   

4. Rasmussen, E.M.: Indexing images (1997) 0.19

   0.18860371 = sum of:
     0.18860371 = product of:
       0.6735847 = sum of:
         0.032345694 = weight(abstract_txt:systems in 2215) [ClassicSimilarity], result of:
           0.032345694 = score(doc=2215,freq=4.0), product of:
             0.060674038 = queryWeight, product of:
               1.0141103 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.017535742 = queryNorm
             0.53310597 = fieldWeight in 2215, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.0820778 = weight(abstract_txt:indexing in 2215) [ClassicSimilarity], result of:
           0.0820778 = score(doc=2215,freq=6.0), product of:
             0.09860796 = queryWeight, product of:
               1.292825 = boost
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.017535742 = queryNorm
             0.83236486 = fieldWeight in 2215, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.14133897 = weight(abstract_txt:image in 2215) [ClassicSimilarity], result of:
           0.14133897 = score(doc=2215,freq=5.0), product of:
             0.15054391 = queryWeight, product of:
               1.5974063 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.017535742 = queryNorm
             0.9388554 = fieldWeight in 2215, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.038714062 = weight(abstract_txt:methods in 2215) [ClassicSimilarity], result of:
           0.038714062 = score(doc=2215,freq=1.0), product of:
             0.11950076 = queryWeight, product of:
               1.6433818 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.017535742 = queryNorm
             0.32396498 = fieldWeight in 2215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.17363769 = weight(abstract_txt:images in 2215) [ClassicSimilarity], result of:
           0.17363769 = score(doc=2215,freq=4.0), product of:
             0.20473912 = queryWeight, product of:
               2.151065 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.017535742 = queryNorm
             0.8480924 = fieldWeight in 2215, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.095183395 = weight(abstract_txt:automatic in 2215) [ClassicSimilarity], result of:
           0.095183395 = score(doc=2215,freq=1.0), product of:
             0.2344961 = queryWeight, product of:
               2.573807 = boost
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.017535742 = queryNorm
             0.40590608 = fieldWeight in 2215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.110287145 = weight(abstract_txt:evaluation in 2215) [ClassicSimilarity], result of:
           0.110287145 = score(doc=2215,freq=1.0), product of:
             0.31467986 = queryWeight, product of:
               4.000174 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.017535742 = queryNorm
             0.35047412 = fieldWeight in 2215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
       0.28 = coord(7/25)
   

5. Gil-Leiva, I.: SISA-automatic indexing system for scientific articles : experiments with location heuristics rules versus TF-IDF rules (2017) 0.17

   0.174775 = sum of:
     0.174775 = product of:
       0.5461719 = sum of:
         0.015616645 = weight(abstract_txt:system in 3622) [ClassicSimilarity], result of:
           0.015616645 = score(doc=3622,freq=1.0), product of:
             0.059274845 = queryWeight, product of:
               1.002349 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.017535742 = queryNorm
             0.2634616 = fieldWeight in 3622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.016172847 = weight(abstract_txt:systems in 3622) [ClassicSimilarity], result of:
           0.016172847 = score(doc=3622,freq=1.0), product of:
             0.060674038 = queryWeight, product of:
               1.0141103 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.017535742 = queryNorm
             0.26655298 = fieldWeight in 3622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.0171969 = weight(abstract_txt:results in 3622) [ClassicSimilarity], result of:
           0.0171969 = score(doc=3622,freq=1.0), product of:
             0.06320897 = queryWeight, product of:
               1.035078 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017535742 = queryNorm
             0.27206424 = fieldWeight in 3622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.02025896 = weight(abstract_txt:some in 3622) [ClassicSimilarity], result of:
           0.02025896 = score(doc=3622,freq=1.0), product of:
             0.07050556 = queryWeight, product of:
               1.0931895 = boost
               3.6779325 = idf(docFreq=3037, maxDocs=44218)
               0.017535742 = queryNorm
             0.28733847 = fieldWeight in 3622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6779325 = idf(docFreq=3037, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.059027568 = weight(abstract_txt:keywords in 3622) [ClassicSimilarity], result of:
           0.059027568 = score(doc=3622,freq=1.0), product of:
             0.1256468 = queryWeight, product of:
               1.1915543 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.017535742 = queryNorm
             0.46978965 = fieldWeight in 3622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.094775274 = weight(abstract_txt:indexing in 3622) [ClassicSimilarity], result of:
           0.094775274 = score(doc=3622,freq=8.0), product of:
             0.09860796 = queryWeight, product of:
               1.292825 = boost
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.017535742 = queryNorm
             0.9611321 = fieldWeight in 3622, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.21283655 = weight(abstract_txt:automatic in 3622) [ClassicSimilarity], result of:
           0.21283655 = score(doc=3622,freq=5.0), product of:
             0.2344961 = queryWeight, product of:
               2.573807 = boost
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.017535742 = queryNorm
             0.9076336 = fieldWeight in 3622, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
         0.110287145 = weight(abstract_txt:evaluation in 3622) [ClassicSimilarity], result of:
           0.110287145 = score(doc=3622,freq=1.0), product of:
             0.31467986 = queryWeight, product of:
               4.000174 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.017535742 = queryNorm
             0.35047412 = fieldWeight in 3622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=3622)
       0.32 = coord(8/25)