Document (#41365)

Author

Aledo, J.A.
Gámez, J.A.
Molina, D.
Rosete, A.

Title

Consensus-based journal rankings : a complementary tool for bibliometric evaluation

Source

Journal of the Association for Information Science and Technology. 69(2018) no.7, S.936-948

Year

2018

Abstract

Annual journal rankings are usually considered a tool for the evaluation of research and researchers. Although they are an objective resource for such evaluation, they also present drawbacks: (a) the uncertainty about the definite position of a target journal in the corresponding annual ranking when selecting a journal, and (b) in spite of the nonsignificant difference in score (for instance, impact factor) between consecutive journals in the ranking, the journals are strictly ranked and eventually placed in different terciles/quartiles, which may have a significant influence in the subsequent evaluation. In this article we present several proposals to obtain an aggregated consensus ranking as an alternative/complementary tool to standardize annual rankings. To illustrate the proposed methodology we use as a case study the Journal Citation Reports, and in particular the category of Computer Science: Artificial Intelligence (CS:AI). In the context of the consensus rankings obtained by the different methods, we discuss the convenience of using one or the other procedure according to the corresponding framework. In particular, our proposals allow us to obtain consensus rankings that avoid crisp frontiers between similarly ranked journals and consider the longitudinal/temporal evolution of the journals.

Content

Vgl.: https://onlinelibrary.wiley.com/doi/abs/10.1002/asi.24040.

Theme

Informetrie

Similar documents (author)

1. Molina, M.P.: Interdisciplinary approaches to the concept and practice of written documentary content analysis (WTDCA) (1994) 5.54

   5.5397964 = sum of:
     5.5397964 = weight(author_txt:molina in 6147) [ClassicSimilarity], result of:
       5.5397964 = fieldWeight in 6147, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.863674 = idf(docFreq=16, maxDocs=44218)
         0.625 = fieldNorm(doc=6147)
   

2. Molina, M.P.: Documentary abstracting : toward a methodological approach (1995) 5.54

   5.5397964 = sum of:
     5.5397964 = weight(author_txt:molina in 1790) [ClassicSimilarity], result of:
       5.5397964 = fieldWeight in 1790, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.863674 = idf(docFreq=16, maxDocs=44218)
         0.625 = fieldNorm(doc=1790)
   

3. Molina, C. Navarro- -> Navarro-Molina, C.: 4.70

   4.700673 = sum of:
     4.700673 = weight(author_txt:molina in 946) [ClassicSimilarity], result of:
       4.700673 = fieldWeight in 946, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         8.863674 = idf(docFreq=16, maxDocs=44218)
         0.375 = fieldNorm(doc=946)
   

4. Molina, E. Perez- => Perez-Molina, E.: 4.70

   4.700673 = sum of:
     4.700673 = weight(author_txt:molina in 5458) [ClassicSimilarity], result of:
       4.700673 = fieldWeight in 5458, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         8.863674 = idf(docFreq=16, maxDocs=44218)
         0.375 = fieldNorm(doc=5458)
   

5. Jansen, B.J.; Molina, P.R.: ¬The effectiveness of Web search engines for retrieving relevant ecommerce links (2006) 4.43

   4.431837 = sum of:
     4.431837 = weight(author_txt:molina in 983) [ClassicSimilarity], result of:
       4.431837 = fieldWeight in 983, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.863674 = idf(docFreq=16, maxDocs=44218)
         0.5 = fieldNorm(doc=983)
   

Similar documents (content)

1. Song, M.; Kim, S.Y.; Lee, K.: Ensemble analysis of topical journal ranking in bioinformatics (2017) 0.21

   0.20577699 = sum of:
     0.20577699 = product of:
       0.8574041 = sum of:
         0.065625064 = weight(abstract_txt:ranked in 3650) [ClassicSimilarity], result of:
           0.065625064 = score(doc=3650,freq=1.0), product of:
             0.16345258 = queryWeight, product of:
               1.662799 = boost
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.015302208 = queryNorm
             0.40149298 = fieldWeight in 3650, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.0625 = fieldNorm(doc=3650)
         0.092165865 = weight(abstract_txt:ranking in 3650) [ClassicSimilarity], result of:
           0.092165865 = score(doc=3650,freq=2.0), product of:
             0.18624264 = queryWeight, product of:
               2.1738474 = boost
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.015302208 = queryNorm
             0.49486983 = fieldWeight in 3650, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.0625 = fieldNorm(doc=3650)
         0.04469961 = weight(abstract_txt:evaluation in 3650) [ClassicSimilarity], result of:
           0.04469961 = score(doc=3650,freq=1.0), product of:
             0.1594255 = queryWeight, product of:
               2.322404 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015302208 = queryNorm
             0.2803793 = fieldWeight in 3650, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=3650)
         0.115885906 = weight(abstract_txt:journals in 3650) [ClassicSimilarity], result of:
           0.115885906 = score(doc=3650,freq=3.0), product of:
             0.20861009 = queryWeight, product of:
               2.6566021 = boost
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.015302208 = queryNorm
             0.5555144 = fieldWeight in 3650, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.0625 = fieldNorm(doc=3650)
         0.25340897 = weight(abstract_txt:journal in 3650) [ClassicSimilarity], result of:
           0.25340897 = score(doc=3650,freq=9.0), product of:
             0.26249787 = queryWeight, product of:
               3.3317833 = boost
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.015302208 = queryNorm
             0.9653753 = fieldWeight in 3650, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.0625 = fieldNorm(doc=3650)
         0.28561863 = weight(abstract_txt:rankings in 3650) [ClassicSimilarity], result of:
           0.28561863 = score(doc=3650,freq=2.0), product of:
             0.4693603 = queryWeight, product of:
               4.4551983 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.015302208 = queryNorm
             0.6085275 = fieldWeight in 3650, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.0625 = fieldNorm(doc=3650)
       0.24 = coord(6/25)
   

2. Ferrara, E.; Romero, A.E.: Scientific impact evaluation and the effect of self-citations : mitigating the bias by discounting the h-index (2013) 0.17

   0.16770546 = sum of:
     0.16770546 = product of:
       0.59894806 = sum of:
         0.021133827 = weight(abstract_txt:particular in 1112) [ClassicSimilarity], result of:
           0.021133827 = score(doc=1112,freq=1.0), product of:
             0.07679461 = queryWeight, product of:
               1.139749 = boost
               4.4031897 = idf(docFreq=1470, maxDocs=44218)
               0.015302208 = queryNorm
             0.27519935 = fieldWeight in 1112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4031897 = idf(docFreq=1470, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
         0.065625064 = weight(abstract_txt:ranked in 1112) [ClassicSimilarity], result of:
           0.065625064 = score(doc=1112,freq=1.0), product of:
             0.16345258 = queryWeight, product of:
               1.662799 = boost
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.015302208 = queryNorm
             0.40149298 = fieldWeight in 1112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
         0.06517111 = weight(abstract_txt:ranking in 1112) [ClassicSimilarity], result of:
           0.06517111 = score(doc=1112,freq=1.0), product of:
             0.18624264 = queryWeight, product of:
               2.1738474 = boost
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.015302208 = queryNorm
             0.34992582 = fieldWeight in 1112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
         0.04469961 = weight(abstract_txt:evaluation in 1112) [ClassicSimilarity], result of:
           0.04469961 = score(doc=1112,freq=1.0), product of:
             0.1594255 = queryWeight, product of:
               2.322404 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015302208 = queryNorm
             0.2803793 = fieldWeight in 1112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
         0.115885906 = weight(abstract_txt:journals in 1112) [ClassicSimilarity], result of:
           0.115885906 = score(doc=1112,freq=3.0), product of:
             0.20861009 = queryWeight, product of:
               2.6566021 = boost
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.015302208 = queryNorm
             0.5555144 = fieldWeight in 1112, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
         0.08446965 = weight(abstract_txt:journal in 1112) [ClassicSimilarity], result of:
           0.08446965 = score(doc=1112,freq=1.0), product of:
             0.26249787 = queryWeight, product of:
               3.3317833 = boost
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.015302208 = queryNorm
             0.32179177 = fieldWeight in 1112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
         0.20196287 = weight(abstract_txt:rankings in 1112) [ClassicSimilarity], result of:
           0.20196287 = score(doc=1112,freq=1.0), product of:
             0.4693603 = queryWeight, product of:
               4.4551983 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.015302208 = queryNorm
             0.43029392 = fieldWeight in 1112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.0625 = fieldNorm(doc=1112)
       0.28 = coord(7/25)
   

3. Davis, P.M.: Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? (2008) 0.16

   0.16479261 = sum of:
     0.16479261 = product of:
       0.82396305 = sum of:
         0.078896075 = weight(abstract_txt:tool in 2377) [ClassicSimilarity], result of:
           0.078896075 = score(doc=2377,freq=1.0), product of:
             0.14567576 = queryWeight, product of:
               1.9225743 = boost
               4.951651 = idf(docFreq=849, maxDocs=44218)
               0.015302208 = queryNorm
             0.5415868 = fieldWeight in 2377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.951651 = idf(docFreq=849, maxDocs=44218)
               0.109375 = fieldNorm(doc=2377)
         0.078224316 = weight(abstract_txt:evaluation in 2377) [ClassicSimilarity], result of:
           0.078224316 = score(doc=2377,freq=1.0), product of:
             0.1594255 = queryWeight, product of:
               2.322404 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015302208 = queryNorm
             0.49066377 = fieldWeight in 2377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.109375 = fieldNorm(doc=2377)
         0.16558579 = weight(abstract_txt:journals in 2377) [ClassicSimilarity], result of:
           0.16558579 = score(doc=2377,freq=2.0), product of:
             0.20861009 = queryWeight, product of:
               2.6566021 = boost
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.015302208 = queryNorm
             0.7937573 = fieldWeight in 2377, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.109375 = fieldNorm(doc=2377)
         0.1478219 = weight(abstract_txt:journal in 2377) [ClassicSimilarity], result of:
           0.1478219 = score(doc=2377,freq=1.0), product of:
             0.26249787 = queryWeight, product of:
               3.3317833 = boost
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.015302208 = queryNorm
             0.5631356 = fieldWeight in 2377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.109375 = fieldNorm(doc=2377)
         0.353435 = weight(abstract_txt:rankings in 2377) [ClassicSimilarity], result of:
           0.353435 = score(doc=2377,freq=1.0), product of:
             0.4693603 = queryWeight, product of:
               4.4551983 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.015302208 = queryNorm
             0.7530143 = fieldWeight in 2377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.109375 = fieldNorm(doc=2377)
       0.2 = coord(5/25)
   

4. Holsapple, C.W.; O'Leary, D.: How much and where? : private versus public universities' publication patterns in the information systems discipline (2009) 0.14

   0.1417278 = sum of:
     0.1417278 = product of:
       0.708639 = sum of:
         0.14572704 = weight(abstract_txt:ranking in 2719) [ClassicSimilarity], result of:
           0.14572704 = score(doc=2719,freq=5.0), product of:
             0.18624264 = queryWeight, product of:
               2.1738474 = boost
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.015302208 = queryNorm
             0.78245795 = fieldWeight in 2719, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.0625 = fieldNorm(doc=2719)
         0.063214794 = weight(abstract_txt:evaluation in 2719) [ClassicSimilarity], result of:
           0.063214794 = score(doc=2719,freq=2.0), product of:
             0.1594255 = queryWeight, product of:
               2.322404 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015302208 = queryNorm
             0.3965162 = fieldWeight in 2719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=2719)
         0.09462045 = weight(abstract_txt:journals in 2719) [ClassicSimilarity], result of:
           0.09462045 = score(doc=2719,freq=2.0), product of:
             0.20861009 = queryWeight, product of:
               2.6566021 = boost
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.015302208 = queryNorm
             0.4535756 = fieldWeight in 2719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.0625 = fieldNorm(doc=2719)
         0.119458124 = weight(abstract_txt:journal in 2719) [ClassicSimilarity], result of:
           0.119458124 = score(doc=2719,freq=2.0), product of:
             0.26249787 = queryWeight, product of:
               3.3317833 = boost
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.015302208 = queryNorm
             0.45508227 = fieldWeight in 2719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.0625 = fieldNorm(doc=2719)
         0.28561863 = weight(abstract_txt:rankings in 2719) [ClassicSimilarity], result of:
           0.28561863 = score(doc=2719,freq=2.0), product of:
             0.4693603 = queryWeight, product of:
               4.4551983 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.015302208 = queryNorm
             0.6085275 = fieldWeight in 2719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.0625 = fieldNorm(doc=2719)
       0.2 = coord(5/25)
   

5. Bohlin, L.; Esquivel, A.V.; Lancichinetti, A.; Rosvall, M.: Robustness of journal rankings by network flows with different amounts of memory (2016) 0.14

   0.13562676 = sum of:
     0.13562676 = product of:
       0.8476673 = sum of:
         0.0632378 = weight(abstract_txt:corresponding in 3125) [ClassicSimilarity], result of:
           0.0632378 = score(doc=3125,freq=1.0), product of:
             0.15946418 = queryWeight, product of:
               1.6423867 = boost
               6.345029 = idf(docFreq=210, maxDocs=44218)
               0.015302208 = queryNorm
             0.3965643 = fieldWeight in 3125, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.345029 = idf(docFreq=210, maxDocs=44218)
               0.0625 = fieldNorm(doc=3125)
         0.16388743 = weight(abstract_txt:journals in 3125) [ClassicSimilarity], result of:
           0.16388743 = score(doc=3125,freq=6.0), product of:
             0.20861009 = queryWeight, product of:
               2.6566021 = boost
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.015302208 = queryNorm
             0.78561604 = fieldWeight in 3125, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.1316223 = idf(docFreq=709, maxDocs=44218)
               0.0625 = fieldNorm(doc=3125)
         0.1689393 = weight(abstract_txt:journal in 3125) [ClassicSimilarity], result of:
           0.1689393 = score(doc=3125,freq=4.0), product of:
             0.26249787 = queryWeight, product of:
               3.3317833 = boost
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.015302208 = queryNorm
             0.64358354 = fieldWeight in 3125, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.1486683 = idf(docFreq=697, maxDocs=44218)
               0.0625 = fieldNorm(doc=3125)
         0.45160276 = weight(abstract_txt:rankings in 3125) [ClassicSimilarity], result of:
           0.45160276 = score(doc=3125,freq=5.0), product of:
             0.4693603 = queryWeight, product of:
               4.4551983 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.015302208 = queryNorm
             0.9621665 = fieldWeight in 3125, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.0625 = fieldNorm(doc=3125)
       0.16 = coord(4/25)