Document (#32964)

Author

Tsai, C.-F.
McGarry, K.
Tait, J.

Title

Qualitative evaluation of automatic assignment of keywords to images

Source

Information processing and management. 42(2006) no.1, S.136-154

Year

2006

Abstract

In image retrieval, most systems lack user-centred evaluation since they are assessed by some chosen ground truth dataset. The results reported through precision and recall assessed against the ground truth are thought of as being an acceptable surrogate for the judgment of real users. Much current research focuses on automatically assigning keywords to images for enhancing retrieval effectiveness. However, evaluation methods are usually based on system-level assessment, e.g. classification accuracy based on some chosen ground truth dataset. In this paper, we present a qualitative evaluation methodology for automatic image indexing systems. The automatic indexing task is formulated as one of image annotation, or automatic metadata generation for images. The evaluation is composed of two individual methods. First, the automatic indexing annotation results are assessed by human subjects. Second, the subjects are asked to annotate some chosen images as the test set whose annotations are used as ground truth. Then, the system is tested by the test set whose annotation results are judged against the ground truth. Only one of these methods is reported for most systems on which user-centred evaluation are conducted. We believe that both methods need to be considered for full evaluation. We also provide an example evaluation of our system based on this methodology. According to this study, our proposed evaluation methodology is able to provide deeper understanding of the system's performance.

Theme

Automatisches Indexieren

Form

Bilder

Similar documents (author)

1. Tait, J.: CALS and its implications for the library and information retrieval communities (1994) 1.21

   1.2102089 = sum of:
     1.2102089 = product of:
       3.6306264 = sum of:
         3.6306264 = weight(author_txt:tait in 1729) [ClassicSimilarity], result of:
           3.6306264 = score(doc=1729,freq=1.0), product of:
             0.6180524 = queryWeight, product of:
               1.073296 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.061267417 = queryNorm
             5.874302 = fieldWeight in 1729, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.625 = fieldNorm(doc=1729)
       0.33333334 = coord(1/3)
   

2. Tait, J.A.: Authors and titles : an analytical study of the author concept in codes of cataloguing rules in the English language, from that of the British Museum in 1841 to the Anglo-American Cataloguing Rules 1967 (1969) 1.21

   1.2102089 = sum of:
     1.2102089 = product of:
       3.6306264 = sum of:
         3.6306264 = weight(author_txt:tait in 3161) [ClassicSimilarity], result of:
           3.6306264 = score(doc=3161,freq=1.0), product of:
             0.6180524 = queryWeight, product of:
               1.073296 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.061267417 = queryNorm
             5.874302 = fieldWeight in 3161, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.625 = fieldNorm(doc=3161)
       0.33333334 = coord(1/3)
   

3. McGarry, D.: Report from the section on classification and indexing : 1989-90 (1990) 0.98

   0.9788178 = sum of:
     0.9788178 = product of:
       2.9364533 = sum of:
         2.9364533 = weight(author_txt:mcgarry in 474) [ClassicSimilarity], result of:
           2.9364533 = score(doc=474,freq=1.0), product of:
             0.53652054 = queryWeight, product of:
               8.757029 = idf(docFreq=18, maxDocs=44421)
               0.061267417 = queryNorm
             5.4731426 = fieldWeight in 474, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.757029 = idf(docFreq=18, maxDocs=44421)
               0.625 = fieldNorm(doc=474)
       0.33333334 = coord(1/3)
   

4. McGarry, D.: Displays of bibliographic records in call number order : functions of the displays and data elements needed (1992) 0.98

   0.9788178 = sum of:
     0.9788178 = product of:
       2.9364533 = sum of:
         2.9364533 = weight(author_txt:mcgarry in 2383) [ClassicSimilarity], result of:
           2.9364533 = score(doc=2383,freq=1.0), product of:
             0.53652054 = queryWeight, product of:
               8.757029 = idf(docFreq=18, maxDocs=44421)
               0.061267417 = queryNorm
             5.4731426 = fieldWeight in 2383, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.757029 = idf(docFreq=18, maxDocs=44421)
               0.625 = fieldNorm(doc=2383)
       0.33333334 = coord(1/3)
   

5. McGarry, D.: Report from the section on classification and indexing : 1991-1992 (1992) 0.98

   0.9788178 = sum of:
     0.9788178 = product of:
       2.9364533 = sum of:
         2.9364533 = weight(author_txt:mcgarry in 2607) [ClassicSimilarity], result of:
           2.9364533 = score(doc=2607,freq=1.0), product of:
             0.53652054 = queryWeight, product of:
               8.757029 = idf(docFreq=18, maxDocs=44421)
               0.061267417 = queryNorm
             5.4731426 = fieldWeight in 2607, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.757029 = idf(docFreq=18, maxDocs=44421)
               0.625 = fieldNorm(doc=2607)
       0.33333334 = coord(1/3)
   

Similar documents (content)

1. Ostermann, F.O.; Tomko, M.; Purves, R.: User evaluation of automatically generated keywords and toponyms for geo-referenced images (2013) 0.34

   0.33595917 = sum of:
     0.33595917 = product of:
       0.83989793 = sum of:
         0.019456914 = weight(abstract_txt:results in 1663) [ClassicSimilarity], result of:
           0.019456914 = score(doc=1663,freq=2.0), product of:
             0.06328034 = queryWeight, product of:
               1.0340745 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017591657 = queryNorm
             0.30747172 = fieldWeight in 1663, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.05152231 = weight(abstract_txt:qualitative in 1663) [ClassicSimilarity], result of:
           0.05152231 = score(doc=1663,freq=2.0), product of:
             0.10580818 = queryWeight, product of:
               1.0917708 = boost
               5.509105 = idf(docFreq=488, maxDocs=44421)
               0.017591657 = queryNorm
             0.4869407 = fieldWeight in 1663, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.509105 = idf(docFreq=488, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.016269187 = weight(abstract_txt:some in 1663) [ClassicSimilarity], result of:
           0.016269187 = score(doc=1663,freq=1.0), product of:
             0.070763074 = queryWeight, product of:
               1.093505 = boost
               3.6785707 = idf(docFreq=3049, maxDocs=44421)
               0.017591657 = queryNorm
             0.22991067 = fieldWeight in 1663, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6785707 = idf(docFreq=3049, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.09433446 = weight(abstract_txt:keywords in 1663) [ClassicSimilarity], result of:
           0.09433446 = score(doc=1663,freq=4.0), product of:
             0.12568766 = queryWeight, product of:
               1.189921 = boost
               6.004374 = idf(docFreq=297, maxDocs=44421)
               0.017591657 = queryNorm
             0.75054675 = fieldWeight in 1663, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.004374 = idf(docFreq=297, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.087922804 = weight(abstract_txt:image in 1663) [ClassicSimilarity], result of:
           0.087922804 = score(doc=1663,freq=3.0), product of:
             0.15109731 = queryWeight, product of:
               1.5978861 = boost
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.017591657 = queryNorm
             0.58189523 = fieldWeight in 1663, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.04384051 = weight(abstract_txt:methods in 1663) [ClassicSimilarity], result of:
           0.04384051 = score(doc=1663,freq=2.0), product of:
             0.11970606 = queryWeight, product of:
               1.6422713 = boost
               4.1434727 = idf(docFreq=1915, maxDocs=44421)
               0.017591657 = queryNorm
             0.3662347 = fieldWeight in 1663, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1434727 = idf(docFreq=1915, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.11303595 = weight(abstract_txt:annotation in 1663) [ClassicSimilarity], result of:
           0.11303595 = score(doc=1663,freq=1.0), product of:
             0.25765616 = queryWeight, product of:
               2.086593 = boost
               7.019336 = idf(docFreq=107, maxDocs=44421)
               0.017591657 = queryNorm
             0.4387085 = fieldWeight in 1663, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.019336 = idf(docFreq=107, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.18444951 = weight(abstract_txt:images in 1663) [ClassicSimilarity], result of:
           0.18444951 = score(doc=1663,freq=7.0), product of:
             0.20547614 = queryWeight, product of:
               2.1516297 = boost
               5.428591 = idf(docFreq=529, maxDocs=44421)
               0.017591657 = queryNorm
             0.8976688 = fieldWeight in 1663, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.428591 = idf(docFreq=529, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.07640209 = weight(abstract_txt:automatic in 1663) [ClassicSimilarity], result of:
           0.07640209 = score(doc=1663,freq=1.0), product of:
             0.23527867 = queryWeight, product of:
               2.5741456 = boost
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.017591657 = queryNorm
             0.32473022 = fieldWeight in 1663, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
         0.15266423 = weight(abstract_txt:evaluation in 1663) [ClassicSimilarity], result of:
           0.15266423 = score(doc=1663,freq=3.0), product of:
             0.3148142 = queryWeight, product of:
               3.994894 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.017591657 = queryNorm
             0.48493436 = fieldWeight in 1663, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=1663)
       0.4 = coord(10/25)
   

2. Hu, X.; Lee, J.H.; Bainbridge, D.; Choi, K.; Organisciak, P.; Downie, J.S.: ¬The MIREX grand challenge : a framework of holistic user-experience evaluation in music information retrieval (2017) 0.26

   0.25806823 = sum of:
     0.25806823 = product of:
       0.9216722 = sum of:
         0.022167444 = weight(abstract_txt:system in 4321) [ClassicSimilarity], result of:
           0.022167444 = score(doc=4321,freq=2.0), product of:
             0.059487157 = queryWeight, product of:
               1.0026032 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017591657 = queryNorm
             0.37264252 = fieldWeight in 4321, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
         0.03626064 = weight(abstract_txt:systems in 4321) [ClassicSimilarity], result of:
           0.03626064 = score(doc=4321,freq=5.0), product of:
             0.060849417 = queryWeight, product of:
               1.014018 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.017591657 = queryNorm
             0.59590775 = fieldWeight in 4321, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
         0.017197646 = weight(abstract_txt:results in 4321) [ClassicSimilarity], result of:
           0.017197646 = score(doc=4321,freq=1.0), product of:
             0.06328034 = queryWeight, product of:
               1.0340745 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017591657 = queryNorm
             0.2717692 = fieldWeight in 4321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
         0.053796172 = weight(abstract_txt:against in 4321) [ClassicSimilarity], result of:
           0.053796172 = score(doc=4321,freq=1.0), product of:
             0.118238784 = queryWeight, product of:
               1.1541222 = boost
               5.823732 = idf(docFreq=356, maxDocs=44421)
               0.017591657 = queryNorm
             0.45497906 = fieldWeight in 4321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.823732 = idf(docFreq=356, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
         0.23096594 = weight(abstract_txt:ground in 4321) [ClassicSimilarity], result of:
           0.23096594 = score(doc=4321,freq=1.0), product of:
             0.4239074 = queryWeight, product of:
               3.455233 = boost
               6.9740796 = idf(docFreq=112, maxDocs=44421)
               0.017591657 = queryNorm
             0.54485 = fieldWeight in 4321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9740796 = idf(docFreq=112, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
         0.26978627 = weight(abstract_txt:truth in 4321) [ClassicSimilarity], result of:
           0.26978627 = score(doc=4321,freq=1.0), product of:
             0.47016713 = queryWeight, product of:
               3.638882 = boost
               7.344759 = idf(docFreq=77, maxDocs=44421)
               0.017591657 = queryNorm
             0.57380927 = fieldWeight in 4321, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.344759 = idf(docFreq=77, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
         0.29149806 = weight(abstract_txt:evaluation in 4321) [ClassicSimilarity], result of:
           0.29149806 = score(doc=4321,freq=7.0), product of:
             0.3148142 = queryWeight, product of:
               3.994894 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.017591657 = queryNorm
             0.9259368 = fieldWeight in 4321, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=4321)
       0.28 = coord(7/25)
   

3. Névéol, A.; Deserno, T.M.; Darmoni, S.J.; Güld, M.O.; Aronson, A.R.: Natural language processing versus content-based image analysis for medical document retrieval (2009) 0.20

   0.1967533 = sum of:
     0.1967533 = product of:
       0.5465369 = sum of:
         0.027995354 = weight(abstract_txt:test in 3702) [ClassicSimilarity], result of:
           0.027995354 = score(doc=3702,freq=1.0), product of:
             0.08876798 = queryWeight, product of:
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.017591657 = queryNorm
             0.3153767 = fieldWeight in 3702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.0125398 = weight(abstract_txt:system in 3702) [ClassicSimilarity], result of:
           0.0125398 = score(doc=3702,freq=1.0), product of:
             0.059487157 = queryWeight, product of:
               1.0026032 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017591657 = queryNorm
             0.21079844 = fieldWeight in 3702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.012973 = weight(abstract_txt:systems in 3702) [ClassicSimilarity], result of:
           0.012973 = score(doc=3702,freq=1.0), product of:
             0.060849417 = queryWeight, product of:
               1.014018 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.017591657 = queryNorm
             0.21319844 = fieldWeight in 3702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.046606824 = weight(abstract_txt:indexing in 3702) [ClassicSimilarity], result of:
           0.046606824 = score(doc=3702,freq=3.0), product of:
             0.09896661 = queryWeight, product of:
               1.2931885 = boost
               4.3503094 = idf(docFreq=1557, maxDocs=44421)
               0.017591657 = queryNorm
             0.4709348 = fieldWeight in 3702, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.3503094 = idf(docFreq=1557, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.15228677 = weight(abstract_txt:image in 3702) [ClassicSimilarity], result of:
           0.15228677 = score(doc=3702,freq=9.0), product of:
             0.15109731 = queryWeight, product of:
               1.5978861 = boost
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.017591657 = queryNorm
             1.0078721 = fieldWeight in 3702, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.030999923 = weight(abstract_txt:methods in 3702) [ClassicSimilarity], result of:
           0.030999923 = score(doc=3702,freq=1.0), product of:
             0.11970606 = queryWeight, product of:
               1.6422713 = boost
               4.1434727 = idf(docFreq=1915, maxDocs=44421)
               0.017591657 = queryNorm
             0.25896704 = fieldWeight in 3702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1434727 = idf(docFreq=1915, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.098592415 = weight(abstract_txt:images in 3702) [ClassicSimilarity], result of:
           0.098592415 = score(doc=3702,freq=2.0), product of:
             0.20547614 = queryWeight, product of:
               2.1516297 = boost
               5.428591 = idf(docFreq=529, maxDocs=44421)
               0.017591657 = queryNorm
             0.47982416 = fieldWeight in 3702, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.428591 = idf(docFreq=529, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.07640209 = weight(abstract_txt:automatic in 3702) [ClassicSimilarity], result of:
           0.07640209 = score(doc=3702,freq=1.0), product of:
             0.23527867 = queryWeight, product of:
               2.5741456 = boost
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.017591657 = queryNorm
             0.32473022 = fieldWeight in 3702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
         0.08814073 = weight(abstract_txt:evaluation in 3702) [ClassicSimilarity], result of:
           0.08814073 = score(doc=3702,freq=1.0), product of:
             0.3148142 = queryWeight, product of:
               3.994894 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.017591657 = queryNorm
             0.279977 = fieldWeight in 3702, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=3702)
       0.36 = coord(9/25)
   

4. Rasmussen, E.M.: Indexing images (1997) 0.19

   0.18911876 = sum of:
     0.18911876 = product of:
       0.67542416 = sum of:
         0.032432504 = weight(abstract_txt:systems in 3215) [ClassicSimilarity], result of:
           0.032432504 = score(doc=3215,freq=4.0), product of:
             0.060849417 = queryWeight, product of:
               1.014018 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.017591657 = queryNorm
             0.5329961 = fieldWeight in 3215, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
         0.082389995 = weight(abstract_txt:indexing in 3215) [ClassicSimilarity], result of:
           0.082389995 = score(doc=3215,freq=6.0), product of:
             0.09896661 = queryWeight, product of:
               1.2931885 = boost
               4.3503094 = idf(docFreq=1557, maxDocs=44421)
               0.017591657 = queryNorm
             0.83250296 = fieldWeight in 3215, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.3503094 = idf(docFreq=1557, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
         0.14188483 = weight(abstract_txt:image in 3215) [ClassicSimilarity], result of:
           0.14188483 = score(doc=3215,freq=5.0), product of:
             0.15109731 = queryWeight, product of:
               1.5978861 = boost
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.017591657 = queryNorm
             0.93902946 = fieldWeight in 3215, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.375318 = idf(docFreq=558, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
         0.038749903 = weight(abstract_txt:methods in 3215) [ClassicSimilarity], result of:
           0.038749903 = score(doc=3215,freq=1.0), product of:
             0.11970606 = queryWeight, product of:
               1.6422713 = boost
               4.1434727 = idf(docFreq=1915, maxDocs=44421)
               0.017591657 = queryNorm
             0.3237088 = fieldWeight in 3215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1434727 = idf(docFreq=1915, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
         0.1742884 = weight(abstract_txt:images in 3215) [ClassicSimilarity], result of:
           0.1742884 = score(doc=3215,freq=4.0), product of:
             0.20547614 = queryWeight, product of:
               2.1516297 = boost
               5.428591 = idf(docFreq=529, maxDocs=44421)
               0.017591657 = queryNorm
             0.8482173 = fieldWeight in 3215, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.428591 = idf(docFreq=529, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
         0.095502615 = weight(abstract_txt:automatic in 3215) [ClassicSimilarity], result of:
           0.095502615 = score(doc=3215,freq=1.0), product of:
             0.23527867 = queryWeight, product of:
               2.5741456 = boost
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.017591657 = queryNorm
             0.40591276 = fieldWeight in 3215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
         0.110175915 = weight(abstract_txt:evaluation in 3215) [ClassicSimilarity], result of:
           0.110175915 = score(doc=3215,freq=1.0), product of:
             0.3148142 = queryWeight, product of:
               3.994894 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.017591657 = queryNorm
             0.34997123 = fieldWeight in 3215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=3215)
       0.28 = coord(7/25)
   

5. Gil-Leiva, I.: SISA-automatic indexing system for scientific articles : experiments with location heuristics rules versus TF-IDF rules (2017) 0.18

   0.17511879 = sum of:
     0.17511879 = product of:
       0.5472462 = sum of:
         0.015674751 = weight(abstract_txt:system in 4622) [ClassicSimilarity], result of:
           0.015674751 = score(doc=4622,freq=1.0), product of:
             0.059487157 = queryWeight, product of:
               1.0026032 = boost
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.017591657 = queryNorm
             0.26349807 = fieldWeight in 4622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.372775 = idf(docFreq=4140, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.016216252 = weight(abstract_txt:systems in 4622) [ClassicSimilarity], result of:
           0.016216252 = score(doc=4622,freq=1.0), product of:
             0.060849417 = queryWeight, product of:
               1.014018 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.017591657 = queryNorm
             0.26649806 = fieldWeight in 4622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.017197646 = weight(abstract_txt:results in 4622) [ClassicSimilarity], result of:
           0.017197646 = score(doc=4622,freq=1.0), product of:
             0.06328034 = queryWeight, product of:
               1.0340745 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.017591657 = queryNorm
             0.2717692 = fieldWeight in 4622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.02033648 = weight(abstract_txt:some in 4622) [ClassicSimilarity], result of:
           0.02033648 = score(doc=4622,freq=1.0), product of:
             0.070763074 = queryWeight, product of:
               1.093505 = boost
               3.6785707 = idf(docFreq=3049, maxDocs=44421)
               0.017591657 = queryNorm
             0.28738832 = fieldWeight in 4622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6785707 = idf(docFreq=3049, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.05895904 = weight(abstract_txt:keywords in 4622) [ClassicSimilarity], result of:
           0.05895904 = score(doc=4622,freq=1.0), product of:
             0.12568766 = queryWeight, product of:
               1.189921 = boost
               6.004374 = idf(docFreq=297, maxDocs=44421)
               0.017591657 = queryNorm
             0.4690917 = fieldWeight in 4622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.004374 = idf(docFreq=297, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.09513578 = weight(abstract_txt:indexing in 4622) [ClassicSimilarity], result of:
           0.09513578 = score(doc=4622,freq=8.0), product of:
             0.09896661 = queryWeight, product of:
               1.2931885 = boost
               4.3503094 = idf(docFreq=1557, maxDocs=44421)
               0.017591657 = queryNorm
             0.9612917 = fieldWeight in 4622, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.3503094 = idf(docFreq=1557, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.21355034 = weight(abstract_txt:automatic in 4622) [ClassicSimilarity], result of:
           0.21355034 = score(doc=4622,freq=5.0), product of:
             0.23527867 = queryWeight, product of:
               2.5741456 = boost
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.017591657 = queryNorm
             0.90764856 = fieldWeight in 4622, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.1956835 = idf(docFreq=668, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
         0.110175915 = weight(abstract_txt:evaluation in 4622) [ClassicSimilarity], result of:
           0.110175915 = score(doc=4622,freq=1.0), product of:
             0.3148142 = queryWeight, product of:
               3.994894 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.017591657 = queryNorm
             0.34997123 = fieldWeight in 4622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=4622)
       0.32 = coord(8/25)