Document (#41365)

Author

Aledo, J.A.
Gámez, J.A.
Molina, D.
Rosete, A.

Title

Consensus-based journal rankings : a complementary tool for bibliometric evaluation

Source

Journal of the Association for Information Science and Technology. 69(2018) no.7, S.936-948

Year

2018

Abstract

Annual journal rankings are usually considered a tool for the evaluation of research and researchers. Although they are an objective resource for such evaluation, they also present drawbacks: (a) the uncertainty about the definite position of a target journal in the corresponding annual ranking when selecting a journal, and (b) in spite of the nonsignificant difference in score (for instance, impact factor) between consecutive journals in the ranking, the journals are strictly ranked and eventually placed in different terciles/quartiles, which may have a significant influence in the subsequent evaluation. In this article we present several proposals to obtain an aggregated consensus ranking as an alternative/complementary tool to standardize annual rankings. To illustrate the proposed methodology we use as a case study the Journal Citation Reports, and in particular the category of Computer Science: Artificial Intelligence (CS:AI). In the context of the consensus rankings obtained by the different methods, we discuss the convenience of using one or the other procedure according to the corresponding framework. In particular, our proposals allow us to obtain consensus rankings that avoid crisp frontiers between similarly ranked journals and consider the longitudinal/temporal evolution of the journals.

Content

Vgl.: https://onlinelibrary.wiley.com/doi/abs/10.1002/asi.24040.

Theme

Informetrie

Similar documents (author)

1. Molina, M.P.: Interdisciplinary approaches to the concept and practice of written documentary content analysis (WTDCA) (1994) 5.54

   5.5426593 = sum of:
     5.5426593 = weight(author_txt:molina in 6146) [ClassicSimilarity], result of:
       5.5426593 = fieldWeight in 6146, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.868255 = idf(docFreq=16, maxDocs=44421)
         0.625 = fieldNorm(doc=6146)
   

2. Molina, M.P.: Documentary abstracting : toward a methodological approach (1995) 5.54

   5.5426593 = sum of:
     5.5426593 = weight(author_txt:molina in 1858) [ClassicSimilarity], result of:
       5.5426593 = fieldWeight in 1858, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.868255 = idf(docFreq=16, maxDocs=44421)
         0.625 = fieldNorm(doc=1858)
   

3. Molina, C. Navarro- -> Navarro-Molina, C.: 4.70

   4.703102 = sum of:
     4.703102 = weight(author_txt:molina in 945) [ClassicSimilarity], result of:
       4.703102 = fieldWeight in 945, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         8.868255 = idf(docFreq=16, maxDocs=44421)
         0.375 = fieldNorm(doc=945)
   

4. Molina, E. Perez- => Perez-Molina, E.: 4.70

   4.703102 = sum of:
     4.703102 = weight(author_txt:molina in 5526) [ClassicSimilarity], result of:
       4.703102 = fieldWeight in 5526, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         8.868255 = idf(docFreq=16, maxDocs=44421)
         0.375 = fieldNorm(doc=5526)
   

5. Jansen, B.J.; Molina, P.R.: ¬The effectiveness of Web search engines for retrieving relevant ecommerce links (2006) 4.43

   4.4341273 = sum of:
     4.4341273 = weight(author_txt:molina in 1983) [ClassicSimilarity], result of:
       4.4341273 = fieldWeight in 1983, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.868255 = idf(docFreq=16, maxDocs=44421)
         0.5 = fieldNorm(doc=1983)
   

Similar documents (content)

1. Song, M.; Kim, S.Y.; Lee, K.: Ensemble analysis of topical journal ranking in bioinformatics (2017) 0.21

   0.20561427 = sum of:
     0.20561427 = product of:
       0.85672617 = sum of:
         0.065917134 = weight(abstract_txt:ranked in 4650) [ClassicSimilarity], result of:
           0.065917134 = score(doc=4650,freq=1.0), product of:
             0.16406305 = queryWeight, product of:
               1.6629988 = boost
               6.428468 = idf(docFreq=194, maxDocs=44421)
               0.015346571 = queryNorm
             0.40177926 = fieldWeight in 4650, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.428468 = idf(docFreq=194, maxDocs=44421)
               0.0625 = fieldNorm(doc=4650)
         0.09227248 = weight(abstract_txt:ranking in 4650) [ClassicSimilarity], result of:
           0.09227248 = score(doc=4650,freq=2.0), product of:
             0.18652934 = queryWeight, product of:
               2.1717296 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.015346571 = queryNorm
             0.4946808 = fieldWeight in 4650, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0625 = fieldNorm(doc=4650)
         0.04461006 = weight(abstract_txt:evaluation in 4650) [ClassicSimilarity], result of:
           0.04461006 = score(doc=4650,freq=1.0), product of:
             0.15933473 = queryWeight, product of:
               2.3176975 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015346571 = queryNorm
             0.279977 = fieldWeight in 4650, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=4650)
         0.11589282 = weight(abstract_txt:journals in 4650) [ClassicSimilarity], result of:
           0.11589282 = score(doc=4650,freq=3.0), product of:
             0.20877853 = queryWeight, product of:
               2.653044 = boost
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.015346571 = queryNorm
             0.5550993 = fieldWeight in 4650, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.0625 = fieldNorm(doc=4650)
         0.25319496 = weight(abstract_txt:journal in 4650) [ClassicSimilarity], result of:
           0.25319496 = score(doc=4650,freq=9.0), product of:
             0.2625515 = queryWeight, product of:
               3.326318 = boost
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.015346571 = queryNorm
             0.9643631 = fieldWeight in 4650, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.0625 = fieldNorm(doc=4650)
         0.28483868 = weight(abstract_txt:rankings in 4650) [ClassicSimilarity], result of:
           0.28483868 = score(doc=4650,freq=2.0), product of:
             0.4688651 = queryWeight, product of:
               4.4450893 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.015346571 = queryNorm
             0.6075067 = fieldWeight in 4650, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0625 = fieldNorm(doc=4650)
       0.24 = coord(6/25)
   

2. Ferrara, E.; Romero, A.E.: Scientific impact evaluation and the effect of self-citations : mitigating the bias by discounting the h-index (2013) 0.17

   0.16761559 = sum of:
     0.16761559 = product of:
       0.5986271 = sum of:
         0.021150876 = weight(abstract_txt:particular in 2112) [ClassicSimilarity], result of:
           0.021150876 = score(doc=2112,freq=1.0), product of:
             0.076894894 = queryWeight, product of:
               1.1385059 = boost
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.015346571 = queryNorm
             0.27506217 = fieldWeight in 2112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
         0.065917134 = weight(abstract_txt:ranked in 2112) [ClassicSimilarity], result of:
           0.065917134 = score(doc=2112,freq=1.0), product of:
             0.16406305 = queryWeight, product of:
               1.6629988 = boost
               6.428468 = idf(docFreq=194, maxDocs=44421)
               0.015346571 = queryNorm
             0.40177926 = fieldWeight in 2112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.428468 = idf(docFreq=194, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
         0.0652465 = weight(abstract_txt:ranking in 2112) [ClassicSimilarity], result of:
           0.0652465 = score(doc=2112,freq=1.0), product of:
             0.18652934 = queryWeight, product of:
               2.1717296 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.015346571 = queryNorm
             0.34979215 = fieldWeight in 2112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
         0.04461006 = weight(abstract_txt:evaluation in 2112) [ClassicSimilarity], result of:
           0.04461006 = score(doc=2112,freq=1.0), product of:
             0.15933473 = queryWeight, product of:
               2.3176975 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015346571 = queryNorm
             0.279977 = fieldWeight in 2112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
         0.11589282 = weight(abstract_txt:journals in 2112) [ClassicSimilarity], result of:
           0.11589282 = score(doc=2112,freq=3.0), product of:
             0.20877853 = queryWeight, product of:
               2.653044 = boost
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.015346571 = queryNorm
             0.5550993 = fieldWeight in 2112, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
         0.08439832 = weight(abstract_txt:journal in 2112) [ClassicSimilarity], result of:
           0.08439832 = score(doc=2112,freq=1.0), product of:
             0.2625515 = queryWeight, product of:
               3.326318 = boost
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.015346571 = queryNorm
             0.32145438 = fieldWeight in 2112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
         0.20141137 = weight(abstract_txt:rankings in 2112) [ClassicSimilarity], result of:
           0.20141137 = score(doc=2112,freq=1.0), product of:
             0.4688651 = queryWeight, product of:
               4.4450893 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.015346571 = queryNorm
             0.4295721 = fieldWeight in 2112, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0625 = fieldNorm(doc=2112)
       0.28 = coord(7/25)
   

3. Davis, P.M.: Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? (2008) 0.16

   0.16455813 = sum of:
     0.16455813 = product of:
       0.8227906 = sum of:
         0.07896042 = weight(abstract_txt:tool in 3377) [ClassicSimilarity], result of:
           0.07896042 = score(doc=3377,freq=1.0), product of:
             0.14586684 = queryWeight, product of:
               1.920483 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.015346571 = queryNorm
             0.5413185 = fieldWeight in 3377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.109375 = fieldNorm(doc=3377)
         0.07806761 = weight(abstract_txt:evaluation in 3377) [ClassicSimilarity], result of:
           0.07806761 = score(doc=3377,freq=1.0), product of:
             0.15933473 = queryWeight, product of:
               2.3176975 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015346571 = queryNorm
             0.48995975 = fieldWeight in 3377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.109375 = fieldNorm(doc=3377)
         0.16559565 = weight(abstract_txt:journals in 3377) [ClassicSimilarity], result of:
           0.16559565 = score(doc=3377,freq=2.0), product of:
             0.20877853 = queryWeight, product of:
               2.653044 = boost
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.015346571 = queryNorm
             0.7931642 = fieldWeight in 3377, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.109375 = fieldNorm(doc=3377)
         0.14769708 = weight(abstract_txt:journal in 3377) [ClassicSimilarity], result of:
           0.14769708 = score(doc=3377,freq=1.0), product of:
             0.2625515 = queryWeight, product of:
               3.326318 = boost
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.015346571 = queryNorm
             0.5625452 = fieldWeight in 3377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.109375 = fieldNorm(doc=3377)
         0.3524699 = weight(abstract_txt:rankings in 3377) [ClassicSimilarity], result of:
           0.3524699 = score(doc=3377,freq=1.0), product of:
             0.4688651 = queryWeight, product of:
               4.4450893 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.015346571 = queryNorm
             0.7517512 = fieldWeight in 3377, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.109375 = fieldNorm(doc=3377)
       0.2 = coord(5/25)
   

4. Holsapple, C.W.; O'Leary, D.: How much and where? : private versus public universities' publication patterns in the information systems discipline (2009) 0.14

   0.14156115 = sum of:
     0.14156115 = product of:
       0.70780575 = sum of:
         0.1458956 = weight(abstract_txt:ranking in 3719) [ClassicSimilarity], result of:
           0.1458956 = score(doc=3719,freq=5.0), product of:
             0.18652934 = queryWeight, product of:
               2.1717296 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.015346571 = queryNorm
             0.78215903 = fieldWeight in 3719, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0625 = fieldNorm(doc=3719)
         0.06308815 = weight(abstract_txt:evaluation in 3719) [ClassicSimilarity], result of:
           0.06308815 = score(doc=3719,freq=2.0), product of:
             0.15933473 = queryWeight, product of:
               2.3176975 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015346571 = queryNorm
             0.39594725 = fieldWeight in 3719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=3719)
         0.094626084 = weight(abstract_txt:journals in 3719) [ClassicSimilarity], result of:
           0.094626084 = score(doc=3719,freq=2.0), product of:
             0.20877853 = queryWeight, product of:
               2.653044 = boost
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.015346571 = queryNorm
             0.45323667 = fieldWeight in 3719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.0625 = fieldNorm(doc=3719)
         0.11935725 = weight(abstract_txt:journal in 3719) [ClassicSimilarity], result of:
           0.11935725 = score(doc=3719,freq=2.0), product of:
             0.2625515 = queryWeight, product of:
               3.326318 = boost
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.015346571 = queryNorm
             0.45460513 = fieldWeight in 3719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.0625 = fieldNorm(doc=3719)
         0.28483868 = weight(abstract_txt:rankings in 3719) [ClassicSimilarity], result of:
           0.28483868 = score(doc=3719,freq=2.0), product of:
             0.4688651 = queryWeight, product of:
               4.4450893 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.015346571 = queryNorm
             0.6075067 = fieldWeight in 3719, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0625 = fieldNorm(doc=3719)
       0.2 = coord(5/25)
   

5. Bohlin, L.; Esquivel, A.V.; Lancichinetti, A.; Rosvall, M.: Robustness of journal rankings by network flows with different amounts of memory (2016) 0.14

   0.13538584 = sum of:
     0.13538584 = product of:
       0.8461615 = sum of:
         0.06309817 = weight(abstract_txt:corresponding in 4125) [ClassicSimilarity], result of:
           0.06309817 = score(doc=4125,freq=1.0), product of:
             0.1593516 = queryWeight, product of:
               1.6389464 = boost
               6.3354917 = idf(docFreq=213, maxDocs=44421)
               0.015346571 = queryNorm
             0.39596823 = fieldWeight in 4125, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.3354917 = idf(docFreq=213, maxDocs=44421)
               0.0625 = fieldNorm(doc=4125)
         0.1638972 = weight(abstract_txt:journals in 4125) [ClassicSimilarity], result of:
           0.1638972 = score(doc=4125,freq=6.0), product of:
             0.20877853 = queryWeight, product of:
               2.653044 = boost
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.015346571 = queryNorm
             0.785029 = fieldWeight in 4125, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.1277876 = idf(docFreq=715, maxDocs=44421)
               0.0625 = fieldNorm(doc=4125)
         0.16879664 = weight(abstract_txt:journal in 4125) [ClassicSimilarity], result of:
           0.16879664 = score(doc=4125,freq=4.0), product of:
             0.2625515 = queryWeight, product of:
               3.326318 = boost
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.015346571 = queryNorm
             0.64290875 = fieldWeight in 4125, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.14327 = idf(docFreq=704, maxDocs=44421)
               0.0625 = fieldNorm(doc=4125)
         0.4503695 = weight(abstract_txt:rankings in 4125) [ClassicSimilarity], result of:
           0.4503695 = score(doc=4125,freq=5.0), product of:
             0.4688651 = queryWeight, product of:
               4.4450893 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.015346571 = queryNorm
             0.96055245 = fieldWeight in 4125, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0625 = fieldNorm(doc=4125)
       0.16 = coord(4/25)