Document (#39271)

Author

Mutz, R.
Bornmann, L.
Daniel, H.-D.

Title

Testing for the fairness and predictive validity of research funding decisions : a multilevel multiple imputation for missing data approach using ex-ante and ex-post peer evaluation data from the Austrian science fund

Source

Journal of the Association for Information Science and Technology. 66(2015) no.11, S.2321-2339

Year

2015

Abstract

It is essential for research funding organizations to ensure both the validity and fairness of the grant approval procedure. The ex-ante peer evaluation (EXANTE) of N?=?8,496 grant applications submitted to the Austrian Science Fund from 1999 to 2009 was statistically analyzed. For 1,689 funded research projects an ex-post peer evaluation (EXPOST) was also available; for the rest of the grant applications a multilevel missing data imputation approach was used to consider verification bias for the first time in peer-review research. Without imputation, the predictive validity of EXANTE was low (r?=?.26) but underestimated due to verification bias, and with imputation it was r?=?.49. That is, the decision-making procedure is capable of selecting the best research proposals for funding. In the EXANTE there were several potential biases (e.g., gender). With respect to the EXPOST there was only one real bias (discipline-specific and year-specific differential prediction). The novelty of this contribution is, first, the combining of theoretical concepts of validity and fairness with a missing data imputation approach to correct for verification bias and, second, multilevel modeling to test peer review-based funding decisions for both validity and fairness in terms of potential and real biases.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23315/abstract.

Similar documents (author)

1. Bornmann, L.; Mutz, R.; Daniel, H.-D.: Are there better indices for evaluation purposes than the h index? : a comparison of nine different variants of the h index using data from biomedicine (2008) 5.43

   5.4306335 = sum of:
     5.4306335 = sum of:
       1.3167337 = weight(author_txt:bornmann in 2608) [ClassicSimilarity], result of:
         1.3167337 = score(doc=2608,freq=1.0), product of:
           0.46357745 = queryWeight, product of:
             7.574333 = idf(docFreq=61, maxDocs=44421)
             0.06120373 = queryNorm
           2.840375 = fieldWeight in 2608, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.574333 = idf(docFreq=61, maxDocs=44421)
             0.375 = fieldNorm(doc=2608)
       1.6737636 = weight(author_txt:daniel in 2608) [ClassicSimilarity], result of:
         1.6737636 = score(doc=2608,freq=1.0), product of:
           0.54398435 = queryWeight, product of:
             1.0832584 = boost
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.06120373 = queryNorm
           3.07686 = fieldWeight in 2608, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.375 = fieldNorm(doc=2608)
       2.440136 = weight(author_txt:mutz in 2608) [ClassicSimilarity], result of:
         2.440136 = score(doc=2608,freq=1.0), product of:
           0.69941187 = queryWeight, product of:
             1.2283025 = boost
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.06120373 = queryNorm
           3.4888396 = fieldWeight in 2608, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.375 = fieldNorm(doc=2608)
   

2. Bornmann, L.; Mutz, R.; Daniel, H.D.: Do we need the h index and its variants in addition to standard bibliometric measures? (2009) 5.43

   5.4306335 = sum of:
     5.4306335 = sum of:
       1.3167337 = weight(author_txt:bornmann in 3861) [ClassicSimilarity], result of:
         1.3167337 = score(doc=3861,freq=1.0), product of:
           0.46357745 = queryWeight, product of:
             7.574333 = idf(docFreq=61, maxDocs=44421)
             0.06120373 = queryNorm
           2.840375 = fieldWeight in 3861, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.574333 = idf(docFreq=61, maxDocs=44421)
             0.375 = fieldNorm(doc=3861)
       1.6737636 = weight(author_txt:daniel in 3861) [ClassicSimilarity], result of:
         1.6737636 = score(doc=3861,freq=1.0), product of:
           0.54398435 = queryWeight, product of:
             1.0832584 = boost
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.06120373 = queryNorm
           3.07686 = fieldWeight in 3861, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.375 = fieldNorm(doc=3861)
       2.440136 = weight(author_txt:mutz in 3861) [ClassicSimilarity], result of:
         2.440136 = score(doc=3861,freq=1.0), product of:
           0.69941187 = queryWeight, product of:
             1.2283025 = boost
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.06120373 = queryNorm
           3.4888396 = fieldWeight in 3861, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.375 = fieldNorm(doc=3861)
   

3. Bornmann, L.; Mutz, R.; Daniel, H.-D.: Multilevel-statistical reformulation of citation-based university rankings : the Leiden ranking 2011/2012 (2013) 5.43

   5.4306335 = sum of:
     5.4306335 = sum of:
       1.3167337 = weight(author_txt:bornmann in 2007) [ClassicSimilarity], result of:
         1.3167337 = score(doc=2007,freq=1.0), product of:
           0.46357745 = queryWeight, product of:
             7.574333 = idf(docFreq=61, maxDocs=44421)
             0.06120373 = queryNorm
           2.840375 = fieldWeight in 2007, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.574333 = idf(docFreq=61, maxDocs=44421)
             0.375 = fieldNorm(doc=2007)
       1.6737636 = weight(author_txt:daniel in 2007) [ClassicSimilarity], result of:
         1.6737636 = score(doc=2007,freq=1.0), product of:
           0.54398435 = queryWeight, product of:
             1.0832584 = boost
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.06120373 = queryNorm
           3.07686 = fieldWeight in 2007, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.20496 = idf(docFreq=32, maxDocs=44421)
             0.375 = fieldNorm(doc=2007)
       2.440136 = weight(author_txt:mutz in 2007) [ClassicSimilarity], result of:
         2.440136 = score(doc=2007,freq=1.0), product of:
           0.69941187 = queryWeight, product of:
             1.2283025 = boost
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.06120373 = queryNorm
           3.4888396 = fieldWeight in 2007, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.375 = fieldNorm(doc=2007)
   

4. Bornmann, L.; Mutz, R.: From P100 to P100' : a new citation-rank approach (2014) 3.34

   3.3394399 = sum of:
     3.3394399 = product of:
       5.0091596 = sum of:
         1.755645 = weight(author_txt:bornmann in 2431) [ClassicSimilarity], result of:
           1.755645 = score(doc=2431,freq=1.0), product of:
             0.46357745 = queryWeight, product of:
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.06120373 = queryNorm
             3.7871666 = fieldWeight in 2431, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.5 = fieldNorm(doc=2431)
         3.2535145 = weight(author_txt:mutz in 2431) [ClassicSimilarity], result of:
           3.2535145 = score(doc=2431,freq=1.0), product of:
             0.69941187 = queryWeight, product of:
               1.2283025 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.06120373 = queryNorm
             4.6517863 = fieldWeight in 2431, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.5 = fieldNorm(doc=2431)
       0.6666667 = coord(2/3)
   

5. Bornmann, L.; Mutz, R.: Growth rates of modern science : a bibliometric analysis based on the number of publications and cited references (2015) 3.34

   3.3394399 = sum of:
     3.3394399 = product of:
       5.0091596 = sum of:
         1.755645 = weight(author_txt:bornmann in 3261) [ClassicSimilarity], result of:
           1.755645 = score(doc=3261,freq=1.0), product of:
             0.46357745 = queryWeight, product of:
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.06120373 = queryNorm
             3.7871666 = fieldWeight in 3261, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.5 = fieldNorm(doc=3261)
         3.2535145 = weight(author_txt:mutz in 3261) [ClassicSimilarity], result of:
           3.2535145 = score(doc=3261,freq=1.0), product of:
             0.69941187 = queryWeight, product of:
               1.2283025 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.06120373 = queryNorm
             4.6517863 = fieldWeight in 3261, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.5 = fieldNorm(doc=3261)
       0.6666667 = coord(2/3)
   

Similar documents (content)

1. Daniel, H.-D.: Guardians of science : fairness and reliabiblity of peer review (1993) 0.21

   0.21452318 = sum of:
     0.21452318 = product of:
       0.89384663 = sum of:
         0.049181115 = weight(abstract_txt:review in 2713) [ClassicSimilarity], result of:
           0.049181115 = score(doc=2713,freq=4.0), product of:
             0.06506428 = queryWeight, product of:
               1.0483093 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.012829729 = queryNorm
             0.755885 = fieldWeight in 2713, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.078125 = fieldNorm(doc=2713)
         0.04223362 = weight(abstract_txt:decisions in 2713) [ClassicSimilarity], result of:
           0.04223362 = score(doc=2713,freq=1.0), product of:
             0.09331166 = queryWeight, product of:
               1.2554114 = boost
               5.7933846 = idf(docFreq=367, maxDocs=44421)
               0.012829729 = queryNorm
             0.45260817 = fieldWeight in 2713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7933846 = idf(docFreq=367, maxDocs=44421)
               0.078125 = fieldNorm(doc=2713)
         0.02928731 = weight(abstract_txt:evaluation in 2713) [ClassicSimilarity], result of:
           0.02928731 = score(doc=2713,freq=1.0), product of:
             0.08368491 = queryWeight, product of:
               1.456087 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.012829729 = queryNorm
             0.34997123 = fieldWeight in 2713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=2713)
         0.29565132 = weight(abstract_txt:peer in 2713) [ClassicSimilarity], result of:
           0.29565132 = score(doc=2713,freq=4.0), product of:
             0.2919512 = queryWeight, product of:
               3.5111003 = boost
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.012829729 = queryNorm
             1.0126737 = fieldWeight in 2713, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.078125 = fieldNorm(doc=2713)
         0.16826299 = weight(abstract_txt:validity in 2713) [ClassicSimilarity], result of:
           0.16826299 = score(doc=2713,freq=1.0), product of:
             0.31827518 = queryWeight, product of:
               3.665975 = boost
               6.7669935 = idf(docFreq=138, maxDocs=44421)
               0.012829729 = queryNorm
             0.5286714 = fieldWeight in 2713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7669935 = idf(docFreq=138, maxDocs=44421)
               0.078125 = fieldNorm(doc=2713)
         0.3092303 = weight(abstract_txt:fairness in 2713) [ClassicSimilarity], result of:
           0.3092303 = score(doc=2713,freq=1.0), product of:
             0.4432973 = queryWeight, product of:
               3.8697298 = boost
               8.928879 = idf(docFreq=15, maxDocs=44421)
               0.012829729 = queryNorm
             0.69756866 = fieldWeight in 2713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.928879 = idf(docFreq=15, maxDocs=44421)
               0.078125 = fieldNorm(doc=2713)
       0.24 = coord(6/25)
   

2. Lee, C.J.; Sugimoto, C.R.; Zhang, G.; Cronin, B.: Bias in peer review (2013) 0.16

   0.16362257 = sum of:
     0.16362257 = product of:
       0.81811285 = sum of:
         0.0520484 = weight(abstract_txt:review in 1525) [ClassicSimilarity], result of:
           0.0520484 = score(doc=1525,freq=7.0), product of:
             0.06506428 = queryWeight, product of:
               1.0483093 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.012829729 = queryNorm
             0.7999535 = fieldWeight in 1525, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=1525)
         0.03063833 = weight(abstract_txt:research in 1525) [ClassicSimilarity], result of:
           0.03063833 = score(doc=1525,freq=5.0), product of:
             0.069385774 = queryWeight, product of:
               1.711683 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.012829729 = queryNorm
             0.441565 = fieldWeight in 1525, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=1525)
         0.09344533 = weight(abstract_txt:funding in 1525) [ClassicSimilarity], result of:
           0.09344533 = score(doc=1525,freq=1.0), product of:
             0.2316428 = queryWeight, product of:
               2.7973223 = boost
               6.4544435 = idf(docFreq=189, maxDocs=44421)
               0.012829729 = queryNorm
             0.40340272 = fieldWeight in 1525, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4544435 = idf(docFreq=189, maxDocs=44421)
               0.0625 = fieldNorm(doc=1525)
         0.35230285 = weight(abstract_txt:bias in 1525) [ClassicSimilarity], result of:
           0.35230285 = score(doc=1525,freq=9.0), product of:
             0.26975965 = queryWeight, product of:
               3.0187116 = boost
               6.965269 = idf(docFreq=113, maxDocs=44421)
               0.012829729 = queryNorm
             1.305988 = fieldWeight in 1525, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               6.965269 = idf(docFreq=113, maxDocs=44421)
               0.0625 = fieldNorm(doc=1525)
         0.28967795 = weight(abstract_txt:peer in 1525) [ClassicSimilarity], result of:
           0.28967795 = score(doc=1525,freq=6.0), product of:
             0.2919512 = queryWeight, product of:
               3.5111003 = boost
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.012829729 = queryNorm
             0.9922136 = fieldWeight in 1525, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.0625 = fieldNorm(doc=1525)
       0.2 = coord(5/25)
   

3. Zhao, Y.W.; Chi. C.-H.; Heuvel, W.J. van den: Imperfect referees : reducing the impact of multiple biases in peer review (2015) 0.16

   0.15781242 = sum of:
     0.15781242 = product of:
       0.65755177 = sum of:
         0.039344892 = weight(abstract_txt:review in 3271) [ClassicSimilarity], result of:
           0.039344892 = score(doc=3271,freq=4.0), product of:
             0.06506428 = queryWeight, product of:
               1.0483093 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.012829729 = queryNorm
             0.604708 = fieldWeight in 3271, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=3271)
         0.025935454 = weight(abstract_txt:real in 3271) [ClassicSimilarity], result of:
           0.025935454 = score(doc=3271,freq=1.0), product of:
             0.078228734 = queryWeight, product of:
               1.1494796 = boost
               5.304538 = idf(docFreq=599, maxDocs=44421)
               0.012829729 = queryNorm
             0.3315336 = fieldWeight in 3271, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.304538 = idf(docFreq=599, maxDocs=44421)
               0.0625 = fieldNorm(doc=3271)
         0.11114523 = weight(abstract_txt:biases in 3271) [ClassicSimilarity], result of:
           0.11114523 = score(doc=3271,freq=2.0), product of:
             0.16381523 = queryWeight, product of:
               1.6633945 = boost
               7.676116 = idf(docFreq=55, maxDocs=44421)
               0.012829729 = queryNorm
             0.6784792 = fieldWeight in 3271, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.676116 = idf(docFreq=55, maxDocs=44421)
               0.0625 = fieldNorm(doc=3271)
         0.013701878 = weight(abstract_txt:research in 3271) [ClassicSimilarity], result of:
           0.013701878 = score(doc=3271,freq=1.0), product of:
             0.069385774 = queryWeight, product of:
               1.711683 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.012829729 = queryNorm
             0.19747387 = fieldWeight in 3271, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=3271)
         0.26259106 = weight(abstract_txt:bias in 3271) [ClassicSimilarity], result of:
           0.26259106 = score(doc=3271,freq=5.0), product of:
             0.26975965 = queryWeight, product of:
               3.0187116 = boost
               6.965269 = idf(docFreq=113, maxDocs=44421)
               0.012829729 = queryNorm
             0.973426 = fieldWeight in 3271, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.965269 = idf(docFreq=113, maxDocs=44421)
               0.0625 = fieldNorm(doc=3271)
         0.20483325 = weight(abstract_txt:peer in 3271) [ClassicSimilarity], result of:
           0.20483325 = score(doc=3271,freq=3.0), product of:
             0.2919512 = queryWeight, product of:
               3.5111003 = boost
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.012829729 = queryNorm
             0.70160097 = fieldWeight in 3271, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.0625 = fieldNorm(doc=3271)
       0.24 = coord(6/25)
   

4. Abramo, G.; D'Angelo, C.A.: ¬A decision support system for public research organizations participating in national research assessment exercises (2009) 0.15

   0.153326 = sum of:
     0.153326 = product of:
       0.4791438 = sum of:
         0.02782104 = weight(abstract_txt:review in 110) [ClassicSimilarity], result of:
           0.02782104 = score(doc=110,freq=2.0), product of:
             0.06506428 = queryWeight, product of:
               1.0483093 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.012829729 = queryNorm
             0.42759314 = fieldWeight in 110, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.03667827 = weight(abstract_txt:real in 110) [ClassicSimilarity], result of:
           0.03667827 = score(doc=110,freq=2.0), product of:
             0.078228734 = queryWeight, product of:
               1.1494796 = boost
               5.304538 = idf(docFreq=599, maxDocs=44421)
               0.012829729 = queryNorm
             0.46885931 = fieldWeight in 110, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.304538 = idf(docFreq=599, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.019300597 = weight(abstract_txt:approach in 110) [ClassicSimilarity], result of:
           0.019300597 = score(doc=110,freq=2.0), product of:
             0.05836753 = queryWeight, product of:
               1.2160443 = boost
               3.741144 = idf(docFreq=2864, maxDocs=44421)
               0.012829729 = queryNorm
             0.33067352 = fieldWeight in 110, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.741144 = idf(docFreq=2864, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.012835184 = weight(abstract_txt:data in 110) [ClassicSimilarity], result of:
           0.012835184 = score(doc=110,freq=1.0), product of:
             0.06166638 = queryWeight, product of:
               1.4433024 = boost
               3.3302255 = idf(docFreq=4320, maxDocs=44421)
               0.012829729 = queryNorm
             0.20813909 = fieldWeight in 110, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3302255 = idf(docFreq=4320, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.046859697 = weight(abstract_txt:evaluation in 110) [ClassicSimilarity], result of:
           0.046859697 = score(doc=110,freq=4.0), product of:
             0.08368491 = queryWeight, product of:
               1.456087 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.012829729 = queryNorm
             0.559954 = fieldWeight in 110, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.036251757 = weight(abstract_txt:research in 110) [ClassicSimilarity], result of:
           0.036251757 = score(doc=110,freq=7.0), product of:
             0.069385774 = queryWeight, product of:
               1.711683 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.012829729 = queryNorm
             0.5224667 = fieldWeight in 110, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.13215165 = weight(abstract_txt:funding in 110) [ClassicSimilarity], result of:
           0.13215165 = score(doc=110,freq=2.0), product of:
             0.2316428 = queryWeight, product of:
               2.7973223 = boost
               6.4544435 = idf(docFreq=189, maxDocs=44421)
               0.012829729 = queryNorm
             0.5704976 = fieldWeight in 110, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.4544435 = idf(docFreq=189, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
         0.16724564 = weight(abstract_txt:peer in 110) [ClassicSimilarity], result of:
           0.16724564 = score(doc=110,freq=2.0), product of:
             0.2919512 = queryWeight, product of:
               3.5111003 = boost
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.012829729 = queryNorm
             0.57285476 = fieldWeight in 110, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.0625 = fieldNorm(doc=110)
       0.32 = coord(8/25)
   

5. Bornmann, L.; Schier, H.; Marx, W.; Daniel, H.-D.: Is interactive open access publishing able to identify high-impact submissions? : a study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classes (2011) 0.14

   0.1423357 = sum of:
     0.1423357 = product of:
       0.59306544 = sum of:
         0.019672446 = weight(abstract_txt:review in 132) [ClassicSimilarity], result of:
           0.019672446 = score(doc=132,freq=1.0), product of:
             0.06506428 = queryWeight, product of:
               1.0483093 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.012829729 = queryNorm
             0.302354 = fieldWeight in 132, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=132)
         0.047781885 = weight(abstract_txt:decisions in 132) [ClassicSimilarity], result of:
           0.047781885 = score(doc=132,freq=2.0), product of:
             0.09331166 = queryWeight, product of:
               1.2554114 = boost
               5.7933846 = idf(docFreq=367, maxDocs=44421)
               0.012829729 = queryNorm
             0.5120677 = fieldWeight in 132, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7933846 = idf(docFreq=367, maxDocs=44421)
               0.0625 = fieldNorm(doc=132)
         0.01937738 = weight(abstract_txt:research in 132) [ClassicSimilarity], result of:
           0.01937738 = score(doc=132,freq=2.0), product of:
             0.069385774 = queryWeight, product of:
               1.711683 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.012829729 = queryNorm
             0.27927023 = fieldWeight in 132, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=132)
         0.15482117 = weight(abstract_txt:predictive in 132) [ClassicSimilarity], result of:
           0.15482117 = score(doc=132,freq=3.0), product of:
             0.17849123 = queryWeight, product of:
               1.7363071 = boost
               8.0125885 = idf(docFreq=39, maxDocs=44421)
               0.012829729 = queryNorm
             0.8673881 = fieldWeight in 132, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.0125885 = idf(docFreq=39, maxDocs=44421)
               0.0625 = fieldNorm(doc=132)
         0.11826053 = weight(abstract_txt:peer in 132) [ClassicSimilarity], result of:
           0.11826053 = score(doc=132,freq=1.0), product of:
             0.2919512 = queryWeight, product of:
               3.5111003 = boost
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.012829729 = queryNorm
             0.4050695 = fieldWeight in 132, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.0625 = fieldNorm(doc=132)
         0.23315202 = weight(abstract_txt:validity in 132) [ClassicSimilarity], result of:
           0.23315202 = score(doc=132,freq=3.0), product of:
             0.31827518 = queryWeight, product of:
               3.665975 = boost
               6.7669935 = idf(docFreq=138, maxDocs=44421)
               0.012829729 = queryNorm
             0.73254853 = fieldWeight in 132, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.7669935 = idf(docFreq=138, maxDocs=44421)
               0.0625 = fieldNorm(doc=132)
       0.24 = coord(6/25)