Document (#43638)

Author

Seeber, M.
Vlegels, J.
Cattaneo, M.

Title

Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation

Source

Journal of the Association for Information Science and Technology. 73(2022) no.8, S.1106-1126

Year

2022

Abstract

Despite interdisciplinary research playing a pivotal role in modern science, interdisciplinary research proposals appear to have a lower chance of being funded. Scholars suggested that interdisciplinary research may be disadvantaged in the evaluation and should be earmarked specific resources and evaluated by specific panels. However, empirical evidence is limited regarding the conditions under which interdisciplinary proposals are disadvantaged. We explore this issue in the context of the European Cooperation in Science and Technology (COST) research funding framework, which, contrary to the common practice, does not organize the evaluation process around disciplinary panels, but with a panel-free system. The sample includes data from five calls, from March 2015 to September 2017, for a total of 1,928 proposals, and 5,330 evaluations conducted by 3,050 reviewers. We find that the effect of a proposal's degree of interdisciplinarity is negligible and not significant. We found no variation in this result across scientific fields and disciplinary expertise of reviewers, and no evidence of disciplinary "turf wars." These results suggest that factors assumed to disadvantage interdisciplinary proposals, such as being inherently more challenging to be evaluated and being riskier, are less problematic when the evaluation is not organized around disciplinary panels but rather with a panel-free system.

Content

https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24617.

Similar documents (content)

1. Zou, J.; Thoma, G.; Antani, S.: Unified deep neural network for segmentation and labeling of multipanel biomedical figures (2020) 0.14

   0.14363253 = sum of:
     0.14363253 = product of:
       0.71816266 = sum of:
         0.005626099 = weight(abstract_txt:that in 1011) [ClassicSimilarity], result of:
           0.005626099 = score(doc=1011,freq=1.0), product of:
             0.038063448 = queryWeight, product of:
               1.2283511 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013102869 = queryNorm
             0.14780845 = fieldWeight in 1011, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=1011)
         0.056216042 = weight(abstract_txt:evaluated in 1011) [ClassicSimilarity], result of:
           0.056216042 = score(doc=1011,freq=2.0), product of:
             0.11123891 = queryWeight, product of:
               1.4848474 = boost
               5.717531 = idf(docFreq=396, maxDocs=44421)
               0.013102869 = queryNorm
             0.5053631 = fieldWeight in 1011, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.717531 = idf(docFreq=396, maxDocs=44421)
               0.0625 = fieldNorm(doc=1011)
         0.24621516 = weight(abstract_txt:panel in 1011) [ClassicSimilarity], result of:
           0.24621516 = score(doc=1011,freq=6.0), product of:
             0.20646794 = queryWeight, product of:
               2.022925 = boost
               7.7894444 = idf(docFreq=49, maxDocs=44421)
               0.013102869 = queryNorm
             1.1925104 = fieldWeight in 1011, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.7894444 = idf(docFreq=49, maxDocs=44421)
               0.0625 = fieldNorm(doc=1011)
         0.016770598 = weight(abstract_txt:research in 1011) [ClassicSimilarity], result of:
           0.016770598 = score(doc=1011,freq=1.0), product of:
             0.08492566 = queryWeight, product of:
               2.051365 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013102869 = queryNorm
             0.19747387 = fieldWeight in 1011, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=1011)
         0.39333478 = weight(abstract_txt:panels in 1011) [ClassicSimilarity], result of:
           0.39333478 = score(doc=1011,freq=3.0), product of:
             0.40693477 = queryWeight, product of:
               3.4782555 = boost
               8.928879 = idf(docFreq=15, maxDocs=44421)
               0.013102869 = queryNorm
             0.96657944 = fieldWeight in 1011, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.928879 = idf(docFreq=15, maxDocs=44421)
               0.0625 = fieldNorm(doc=1011)
       0.2 = coord(5/25)
   

2. Mardis, M.A.; Hoffman, E.S.; McMartin, F.P.: Toward broader impacts : making sense of NSF's merit review criteria in the context of the National Science Digital Library (2012) 0.11

   0.108710214 = sum of:
     0.108710214 = product of:
       0.38825077 = sum of:
         0.021031026 = weight(abstract_txt:science in 1384) [ClassicSimilarity], result of:
           0.021031026 = score(doc=1384,freq=3.0), product of:
             0.050453708 = queryWeight, product of:
               3.850585 = idf(docFreq=2567, maxDocs=44421)
               0.013102869 = queryNorm
             0.41683805 = fieldWeight in 1384, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.850585 = idf(docFreq=2567, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
         0.016977156 = weight(abstract_txt:specific in 1384) [ClassicSimilarity], result of:
           0.016977156 = score(doc=1384,freq=1.0), product of:
             0.06308651 = queryWeight, product of:
               1.1182057 = boost
               4.305746 = idf(docFreq=1628, maxDocs=44421)
               0.013102869 = queryNorm
             0.26910913 = fieldWeight in 1384, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.305746 = idf(docFreq=1628, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
         0.011252198 = weight(abstract_txt:that in 1384) [ClassicSimilarity], result of:
           0.011252198 = score(doc=1384,freq=4.0), product of:
             0.038063448 = queryWeight, product of:
               1.2283511 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013102869 = queryNorm
             0.2956169 = fieldWeight in 1384, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
         0.029235926 = weight(abstract_txt:being in 1384) [ClassicSimilarity], result of:
           0.029235926 = score(doc=1384,freq=1.0), product of:
             0.10375321 = queryWeight, product of:
               1.7563046 = boost
               4.5085335 = idf(docFreq=1329, maxDocs=44421)
               0.013102869 = queryNorm
             0.28178334 = fieldWeight in 1384, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5085335 = idf(docFreq=1329, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
         0.023717208 = weight(abstract_txt:research in 1384) [ClassicSimilarity], result of:
           0.023717208 = score(doc=1384,freq=2.0), product of:
             0.08492566 = queryWeight, product of:
               2.051365 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013102869 = queryNorm
             0.27927023 = fieldWeight in 1384, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
         0.11747598 = weight(abstract_txt:reviewers in 1384) [ClassicSimilarity], result of:
           0.11747598 = score(doc=1384,freq=1.0), product of:
             0.22908285 = queryWeight, product of:
               2.1308346 = boost
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.013102869 = queryNorm
             0.51281 = fieldWeight in 1384, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
         0.16856128 = weight(abstract_txt:proposals in 1384) [ClassicSimilarity], result of:
           0.16856128 = score(doc=1384,freq=1.0), product of:
             0.3955291 = queryWeight, product of:
               4.427032 = boost
               6.8186655 = idf(docFreq=131, maxDocs=44421)
               0.013102869 = queryNorm
             0.4261666 = fieldWeight in 1384, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8186655 = idf(docFreq=131, maxDocs=44421)
               0.0625 = fieldNorm(doc=1384)
       0.28 = coord(7/25)
   

3. Schwarz, C.: THESYS: Thesaurus Syntax System : a fully automatic thesaurus building aid (1988) 0.11

   0.10636058 = sum of:
     0.10636058 = product of:
       0.5318029 = sum of:
         0.056126032 = weight(abstract_txt:free in 1429) [ClassicSimilarity], result of:
           0.056126032 = score(doc=1429,freq=1.0), product of:
             0.10684198 = queryWeight, product of:
               1.4552058 = boost
               5.6033936 = idf(docFreq=444, maxDocs=44421)
               0.013102869 = queryNorm
             0.52531815 = fieldWeight in 1429, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6033936 = idf(docFreq=444, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
         0.08432406 = weight(abstract_txt:evaluated in 1429) [ClassicSimilarity], result of:
           0.08432406 = score(doc=1429,freq=2.0), product of:
             0.11123891 = queryWeight, product of:
               1.4848474 = boost
               5.717531 = idf(docFreq=396, maxDocs=44421)
               0.013102869 = queryNorm
             0.75804466 = fieldWeight in 1429, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.717531 = idf(docFreq=396, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
         0.076492056 = weight(abstract_txt:conditions in 1429) [ClassicSimilarity], result of:
           0.076492056 = score(doc=1429,freq=1.0), product of:
             0.13133383 = queryWeight, product of:
               1.6133989 = boost
               6.2125297 = idf(docFreq=241, maxDocs=44421)
               0.013102869 = queryNorm
             0.58242464 = fieldWeight in 1429, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2125297 = idf(docFreq=241, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
         0.06201876 = weight(abstract_txt:being in 1429) [ClassicSimilarity], result of:
           0.06201876 = score(doc=1429,freq=2.0), product of:
             0.10375321 = queryWeight, product of:
               1.7563046 = boost
               4.5085335 = idf(docFreq=1329, maxDocs=44421)
               0.013102869 = queryNorm
             0.5977527 = fieldWeight in 1429, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.5085335 = idf(docFreq=1329, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
         0.25284195 = weight(abstract_txt:proposals in 1429) [ClassicSimilarity], result of:
           0.25284195 = score(doc=1429,freq=1.0), product of:
             0.3955291 = queryWeight, product of:
               4.427032 = boost
               6.8186655 = idf(docFreq=131, maxDocs=44421)
               0.013102869 = queryNorm
             0.6392499 = fieldWeight in 1429, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8186655 = idf(docFreq=131, maxDocs=44421)
               0.09375 = fieldNorm(doc=1429)
       0.2 = coord(5/25)
   

4. Levitt, J.M.; Thelwall, M.; Oppenheim, C.: Variations between subjects in the extent to which the social sciences have become more interdisciplinary (2011) 0.11

   0.10627577 = sum of:
     0.10627577 = product of:
       0.44281572 = sum of:
         0.021031026 = weight(abstract_txt:science in 465) [ClassicSimilarity], result of:
           0.021031026 = score(doc=465,freq=3.0), product of:
             0.050453708 = queryWeight, product of:
               3.850585 = idf(docFreq=2567, maxDocs=44421)
               0.013102869 = queryNorm
             0.41683805 = fieldWeight in 465, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.850585 = idf(docFreq=2567, maxDocs=44421)
               0.0625 = fieldNorm(doc=465)
         0.009744689 = weight(abstract_txt:that in 465) [ClassicSimilarity], result of:
           0.009744689 = score(doc=465,freq=3.0), product of:
             0.038063448 = queryWeight, product of:
               1.2283511 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013102869 = queryNorm
             0.25601172 = fieldWeight in 465, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=465)
         0.032867912 = weight(abstract_txt:evidence in 465) [ClassicSimilarity], result of:
           0.032867912 = score(doc=465,freq=1.0), product of:
             0.09799591 = queryWeight, product of:
               1.3936619 = boost
               5.3664136 = idf(docFreq=563, maxDocs=44421)
               0.013102869 = queryNorm
             0.33540085 = fieldWeight in 465, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3664136 = idf(docFreq=563, maxDocs=44421)
               0.0625 = fieldNorm(doc=465)
         0.023717208 = weight(abstract_txt:research in 465) [ClassicSimilarity], result of:
           0.023717208 = score(doc=465,freq=2.0), product of:
             0.08492566 = queryWeight, product of:
               2.051365 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013102869 = queryNorm
             0.27927023 = fieldWeight in 465, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=465)
         0.15434058 = weight(abstract_txt:disciplinary in 465) [ClassicSimilarity], result of:
           0.15434058 = score(doc=465,freq=2.0), product of:
             0.27479783 = queryWeight, product of:
               3.3004649 = boost
               6.35436 = idf(docFreq=209, maxDocs=44421)
               0.013102869 = queryNorm
             0.5616514 = fieldWeight in 465, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.35436 = idf(docFreq=209, maxDocs=44421)
               0.0625 = fieldNorm(doc=465)
         0.20111428 = weight(abstract_txt:interdisciplinary in 465) [ClassicSimilarity], result of:
           0.20111428 = score(doc=465,freq=2.0), product of:
             0.37527725 = queryWeight, product of:
               4.723786 = boost
               6.0631127 = idf(docFreq=280, maxDocs=44421)
               0.013102869 = queryNorm
             0.5359085 = fieldWeight in 465, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.0631127 = idf(docFreq=280, maxDocs=44421)
               0.0625 = fieldNorm(doc=465)
       0.24 = coord(6/25)
   

5. Scimeca, R.; Labaree, R.V.: Synoptic reference : introducing a polymathic approach to reference services (2015) 0.10

   0.10176998 = sum of:
     0.10176998 = product of:
       0.50884986 = sum of:
         0.011252198 = weight(abstract_txt:that in 532) [ClassicSimilarity], result of:
           0.011252198 = score(doc=532,freq=4.0), product of:
             0.038063448 = queryWeight, product of:
               1.2283511 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013102869 = queryNorm
             0.2956169 = fieldWeight in 532, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=532)
         0.042067926 = weight(abstract_txt:around in 532) [ClassicSimilarity], result of:
           0.042067926 = score(doc=532,freq=1.0), product of:
             0.11552091 = queryWeight, product of:
               1.5131562 = boost
               5.8265367 = idf(docFreq=355, maxDocs=44421)
               0.013102869 = queryNorm
             0.36415854 = fieldWeight in 532, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8265367 = idf(docFreq=355, maxDocs=44421)
               0.0625 = fieldNorm(doc=532)
         0.016770598 = weight(abstract_txt:research in 532) [ClassicSimilarity], result of:
           0.016770598 = score(doc=532,freq=1.0), product of:
             0.08492566 = queryWeight, product of:
               2.051365 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013102869 = queryNorm
             0.19747387 = fieldWeight in 532, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=532)
         0.15434058 = weight(abstract_txt:disciplinary in 532) [ClassicSimilarity], result of:
           0.15434058 = score(doc=532,freq=2.0), product of:
             0.27479783 = queryWeight, product of:
               3.3004649 = boost
               6.35436 = idf(docFreq=209, maxDocs=44421)
               0.013102869 = queryNorm
             0.5616514 = fieldWeight in 532, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.35436 = idf(docFreq=209, maxDocs=44421)
               0.0625 = fieldNorm(doc=532)
         0.28441852 = weight(abstract_txt:interdisciplinary in 532) [ClassicSimilarity], result of:
           0.28441852 = score(doc=532,freq=4.0), product of:
             0.37527725 = queryWeight, product of:
               4.723786 = boost
               6.0631127 = idf(docFreq=280, maxDocs=44421)
               0.013102869 = queryNorm
             0.7578891 = fieldWeight in 532, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.0631127 = idf(docFreq=280, maxDocs=44421)
               0.0625 = fieldNorm(doc=532)
       0.2 = coord(5/25)