Document (#38531)

Author

Zuccala, A.
Someren, M. van
Bellen, M. van

Title

¬A machine-learning approach to coding book reviews as quality indicators : toward a theory of megacitation

Source

Journal of the Association for Information Science and Technology. 65(2014) no.11, S.2248-2260

Year

2014

Abstract

A theory of "megacitation" is introduced and used in an experiment to demonstrate how a qualitative scholarly book review can be converted into a weighted bibliometric indicator. We employ a manual human-coding approach to classify book reviews in the field of history based on reviewers' assessments of a book author's scholarly credibility (SC) and writing style (WS). In total, 100 book reviews were selected from the American Historical Review and coded for their positive/negative valence on these two dimensions. Most were coded as positive (68% for SC and 47% for WS), and there was also a small positive correlation between SC and WS (r = 0.2). We then constructed a classifier, combining both manual design and machine learning, to categorize sentiment-based sentences in history book reviews. The machine classifier produced a matched accuracy (matched to the human coding) of approximately 75% for SC and 64% for WS. WS was found to be more difficult to classify by machine than SC because of the reviewers' use of more subtle language. With further training data, a machine-learning approach could be useful for automatically classifying a large number of history book reviews at once. Weighted megacitations can be especially valuable if they are used in conjunction with regular book/journal citations, and "libcitations" (i.e., library holding counts) for a comprehensive assessment of a book/monograph's scholarly impact.

Theme

Informetrie

Similar documents (author)

1. Zuccala, A.: Modeling the invisible college (2006) 5.76

   5.7603507 = sum of:
     5.7603507 = weight(author_txt:zuccala in 4350) [ClassicSimilarity], result of:
       5.7603507 = fieldWeight in 4350, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.216561 = idf(docFreq=11, maxDocs=44421)
         0.625 = fieldNorm(doc=4350)
   

2. Zuccala, A.: Author cocitation analysis is to intellectual structure as Web colink analysis is to ... ? (2006) 5.76

   5.7603507 = sum of:
     5.7603507 = weight(author_txt:zuccala in 8) [ClassicSimilarity], result of:
       5.7603507 = fieldWeight in 8, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.216561 = idf(docFreq=11, maxDocs=44421)
         0.625 = fieldNorm(doc=8)
   

3. Rousseau, R.; Zuccala, A.: ¬A classification of author co-citations : definitions and search strategies (2004) 4.61

   4.6082807 = sum of:
     4.6082807 = weight(author_txt:zuccala in 3266) [ClassicSimilarity], result of:
       4.6082807 = fieldWeight in 3266, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.216561 = idf(docFreq=11, maxDocs=44421)
         0.5 = fieldNorm(doc=3266)
   

4. Zuccala, A.; Leeuwen, T.van: Book reviews in humanities research evaluations (2011) 4.61

   4.6082807 = sum of:
     4.6082807 = weight(author_txt:zuccala in 771) [ClassicSimilarity], result of:
       4.6082807 = fieldWeight in 771, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.216561 = idf(docFreq=11, maxDocs=44421)
         0.5 = fieldNorm(doc=771)
   

5. White, H.D.; Zuccala, A.A.: Libcitations, worldcat, cultural impact, and fame (2018) 4.61

   4.6082807 = sum of:
     4.6082807 = weight(author_txt:zuccala in 578) [ClassicSimilarity], result of:
       4.6082807 = fieldWeight in 578, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.216561 = idf(docFreq=11, maxDocs=44421)
         0.5 = fieldNorm(doc=578)
   

Similar documents (content)

1. Zuccala, A.; Leeuwen, T.van: Book reviews in humanities research evaluations (2011) 0.20

   0.2030171 = sum of:
     0.2030171 = product of:
       0.8459046 = sum of:
         0.04101935 = weight(abstract_txt:review in 771) [ClassicSimilarity], result of:
           0.04101935 = score(doc=771,freq=2.0), product of:
             0.0959308 = queryWeight, product of:
               1.2854025 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.015427061 = queryNorm
             0.42759314 = fieldWeight in 771, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=771)
         0.047394417 = weight(abstract_txt:history in 771) [ClassicSimilarity], result of:
           0.047394417 = score(doc=771,freq=1.0), product of:
             0.15234356 = queryWeight, product of:
               1.9838909 = boost
               4.9776354 = idf(docFreq=831, maxDocs=44421)
               0.015427061 = queryNorm
             0.3111022 = fieldWeight in 771, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9776354 = idf(docFreq=831, maxDocs=44421)
               0.0625 = fieldNorm(doc=771)
         0.123992205 = weight(abstract_txt:scholarly in 771) [ClassicSimilarity], result of:
           0.123992205 = score(doc=771,freq=4.0), product of:
             0.1822142 = queryWeight, product of:
               2.1696858 = boost
               5.4438 = idf(docFreq=521, maxDocs=44421)
               0.015427061 = queryNorm
             0.680475 = fieldWeight in 771, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.4438 = idf(docFreq=521, maxDocs=44421)
               0.0625 = fieldNorm(doc=771)
         0.07865732 = weight(abstract_txt:positive in 771) [ClassicSimilarity], result of:
           0.07865732 = score(doc=771,freq=1.0), product of:
             0.21354958 = queryWeight, product of:
               2.348849 = boost
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.015427061 = queryNorm
             0.36833283 = fieldWeight in 771, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.0625 = fieldNorm(doc=771)
         0.20542817 = weight(abstract_txt:reviews in 771) [ClassicSimilarity], result of:
           0.20542817 = score(doc=771,freq=7.0), product of:
             0.25101298 = queryWeight, product of:
               3.2875943 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.015427061 = queryNorm
             0.81839657 = fieldWeight in 771, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.0625 = fieldNorm(doc=771)
         0.34941313 = weight(abstract_txt:book in 771) [ClassicSimilarity], result of:
           0.34941313 = score(doc=771,freq=7.0), product of:
             0.43508404 = queryWeight, product of:
               5.807015 = boost
               4.8566523 = idf(docFreq=938, maxDocs=44421)
               0.015427061 = queryNorm
             0.8030934 = fieldWeight in 771, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.8566523 = idf(docFreq=938, maxDocs=44421)
               0.0625 = fieldNorm(doc=771)
       0.24 = coord(6/25)
   

2. Na, J.-C.; Sui, H.; Khoo, C.; Chan, S.; Zhou, Y.: Effectiveness of simple linguistic processing in automatic sentiment classification of product reviews (2004) 0.16

   0.16178887 = sum of:
     0.16178887 = product of:
       0.5778174 = sum of:
         0.10925627 = weight(abstract_txt:sentiment in 3624) [ClassicSimilarity], result of:
           0.10925627 = score(doc=3624,freq=4.0), product of:
             0.11612073 = queryWeight, product of:
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.015427061 = queryNorm
             0.94088507 = fieldWeight in 3624, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.5270805 = idf(docFreq=64, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
         0.029005062 = weight(abstract_txt:review in 3624) [ClassicSimilarity], result of:
           0.029005062 = score(doc=3624,freq=1.0), product of:
             0.0959308 = queryWeight, product of:
               1.2854025 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.015427061 = queryNorm
             0.302354 = fieldWeight in 3624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
         0.028456805 = weight(abstract_txt:approach in 3624) [ClassicSimilarity], result of:
           0.028456805 = score(doc=3624,freq=2.0), product of:
             0.086057104 = queryWeight, product of:
               1.4910737 = boost
               3.741144 = idf(docFreq=2864, maxDocs=44421)
               0.015427061 = queryNorm
             0.33067352 = fieldWeight in 3624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.741144 = idf(docFreq=2864, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
         0.071367815 = weight(abstract_txt:classify in 3624) [ClassicSimilarity], result of:
           0.071367815 = score(doc=3624,freq=1.0), product of:
             0.17484121 = queryWeight, product of:
               1.7353297 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.015427061 = queryNorm
             0.40818647 = fieldWeight in 3624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
         0.11123825 = weight(abstract_txt:positive in 3624) [ClassicSimilarity], result of:
           0.11123825 = score(doc=3624,freq=2.0), product of:
             0.21354958 = queryWeight, product of:
               2.348849 = boost
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.015427061 = queryNorm
             0.52090126 = fieldWeight in 3624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
         0.13448429 = weight(abstract_txt:reviews in 3624) [ClassicSimilarity], result of:
           0.13448429 = score(doc=3624,freq=3.0), product of:
             0.25101298 = queryWeight, product of:
               3.2875943 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.015427061 = queryNorm
             0.5357663 = fieldWeight in 3624, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
         0.094008885 = weight(abstract_txt:machine in 3624) [ClassicSimilarity], result of:
           0.094008885 = score(doc=3624,freq=1.0), product of:
             0.28514656 = queryWeight, product of:
               3.5040007 = boost
               5.274979 = idf(docFreq=617, maxDocs=44421)
               0.015427061 = queryNorm
             0.3296862 = fieldWeight in 3624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.274979 = idf(docFreq=617, maxDocs=44421)
               0.0625 = fieldNorm(doc=3624)
       0.28 = coord(7/25)
   

3. Jia, Y.; Liu, I.L.B.: Do consumers always follow "useful" reviews? : The interaction effect of review valence and review usefulness on consumers' purchase decisions (2018) 0.16

   0.16165099 = sum of:
     0.16165099 = product of:
       0.6735458 = sum of:
         0.029765755 = weight(abstract_txt:theory in 541) [ClassicSimilarity], result of:
           0.029765755 = score(doc=541,freq=1.0), product of:
             0.084109835 = queryWeight, product of:
               1.2036036 = boost
               4.529811 = idf(docFreq=1301, maxDocs=44421)
               0.015427061 = queryNorm
             0.3538915 = fieldWeight in 541, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.529811 = idf(docFreq=1301, maxDocs=44421)
               0.078125 = fieldNorm(doc=541)
         0.23026927 = weight(abstract_txt:valence in 541) [ClassicSimilarity], result of:
           0.23026927 = score(doc=541,freq=3.0), product of:
             0.1810544 = queryWeight, product of:
               1.2486757 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.015427061 = queryNorm
             1.2718236 = fieldWeight in 541, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.078125 = fieldNorm(doc=541)
         0.09592523 = weight(abstract_txt:review in 541) [ClassicSimilarity], result of:
           0.09592523 = score(doc=541,freq=7.0), product of:
             0.0959308 = queryWeight, product of:
               1.2854025 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.015427061 = queryNorm
             0.9999419 = fieldWeight in 541, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.078125 = fieldNorm(doc=541)
         0.0251525 = weight(abstract_txt:approach in 541) [ClassicSimilarity], result of:
           0.0251525 = score(doc=541,freq=1.0), product of:
             0.086057104 = queryWeight, product of:
               1.4910737 = boost
               3.741144 = idf(docFreq=2864, maxDocs=44421)
               0.015427061 = queryNorm
             0.29227686 = fieldWeight in 541, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.741144 = idf(docFreq=2864, maxDocs=44421)
               0.078125 = fieldNorm(doc=541)
         0.098321654 = weight(abstract_txt:positive in 541) [ClassicSimilarity], result of:
           0.098321654 = score(doc=541,freq=1.0), product of:
             0.21354958 = queryWeight, product of:
               2.348849 = boost
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.015427061 = queryNorm
             0.46041605 = fieldWeight in 541, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.078125 = fieldNorm(doc=541)
         0.19411138 = weight(abstract_txt:reviews in 541) [ClassicSimilarity], result of:
           0.19411138 = score(doc=541,freq=4.0), product of:
             0.25101298 = queryWeight, product of:
               3.2875943 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.015427061 = queryNorm
             0.7733121 = fieldWeight in 541, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.078125 = fieldNorm(doc=541)
       0.24 = coord(6/25)
   

4. Nobarany, S.; Booth, K.S.: Use of politeness strategies in signed open peer review (2015) 0.14

   0.14102897 = sum of:
     0.14102897 = product of:
       0.58762074 = sum of:
         0.023812603 = weight(abstract_txt:theory in 2825) [ClassicSimilarity], result of:
           0.023812603 = score(doc=2825,freq=1.0), product of:
             0.084109835 = queryWeight, product of:
               1.2036036 = boost
               4.529811 = idf(docFreq=1301, maxDocs=44421)
               0.015427061 = queryNorm
             0.28311318 = fieldWeight in 2825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.529811 = idf(docFreq=1301, maxDocs=44421)
               0.0625 = fieldNorm(doc=2825)
         0.026282072 = weight(abstract_txt:human in 2825) [ClassicSimilarity], result of:
           0.026282072 = score(doc=2825,freq=1.0), product of:
             0.08982873 = queryWeight, product of:
               1.2438493 = boost
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.015427061 = queryNorm
             0.2925798 = fieldWeight in 2825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.0625 = fieldNorm(doc=2825)
         0.05023824 = weight(abstract_txt:review in 2825) [ClassicSimilarity], result of:
           0.05023824 = score(doc=2825,freq=3.0), product of:
             0.0959308 = queryWeight, product of:
               1.2854025 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.015427061 = queryNorm
             0.5236925 = fieldWeight in 2825, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.0625 = fieldNorm(doc=2825)
         0.061996102 = weight(abstract_txt:scholarly in 2825) [ClassicSimilarity], result of:
           0.061996102 = score(doc=2825,freq=1.0), product of:
             0.1822142 = queryWeight, product of:
               2.1696858 = boost
               5.4438 = idf(docFreq=521, maxDocs=44421)
               0.015427061 = queryNorm
             0.3402375 = fieldWeight in 2825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4438 = idf(docFreq=521, maxDocs=44421)
               0.0625 = fieldNorm(doc=2825)
         0.3466344 = weight(abstract_txt:reviewers in 2825) [ClassicSimilarity], result of:
           0.3466344 = score(doc=2825,freq=6.0), product of:
             0.2759558 = queryWeight, product of:
               2.1801174 = boost
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.015427061 = queryNorm
             1.2561228 = fieldWeight in 2825, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.0625 = fieldNorm(doc=2825)
         0.07865732 = weight(abstract_txt:positive in 2825) [ClassicSimilarity], result of:
           0.07865732 = score(doc=2825,freq=1.0), product of:
             0.21354958 = queryWeight, product of:
               2.348849 = boost
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.015427061 = queryNorm
             0.36833283 = fieldWeight in 2825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.0625 = fieldNorm(doc=2825)
       0.24 = coord(6/25)
   

5. Chua, A.Y.K.; Banerjee, S.: Understanding review helpfulness as a function of reviewer reputation, review rating, and review depth (2015) 0.13

   0.12915869 = sum of:
     0.12915869 = product of:
       0.64579344 = sum of:
         0.035718903 = weight(abstract_txt:theory in 2641) [ClassicSimilarity], result of:
           0.035718903 = score(doc=2641,freq=1.0), product of:
             0.084109835 = queryWeight, product of:
               1.2036036 = boost
               4.529811 = idf(docFreq=1301, maxDocs=44421)
               0.015427061 = queryNorm
             0.42466977 = fieldWeight in 2641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.529811 = idf(docFreq=1301, maxDocs=44421)
               0.09375 = fieldNorm(doc=2641)
         0.11511028 = weight(abstract_txt:review in 2641) [ClassicSimilarity], result of:
           0.11511028 = score(doc=2641,freq=7.0), product of:
             0.0959308 = queryWeight, product of:
               1.2854025 = boost
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.015427061 = queryNorm
             1.1999303 = fieldWeight in 2641, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.837664 = idf(docFreq=956, maxDocs=44421)
               0.09375 = fieldNorm(doc=2641)
         0.21226934 = weight(abstract_txt:reviewers in 2641) [ClassicSimilarity], result of:
           0.21226934 = score(doc=2641,freq=1.0), product of:
             0.2759558 = queryWeight, product of:
               2.1801174 = boost
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.015427061 = queryNorm
             0.769215 = fieldWeight in 2641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.09375 = fieldNorm(doc=2641)
         0.11798598 = weight(abstract_txt:positive in 2641) [ClassicSimilarity], result of:
           0.11798598 = score(doc=2641,freq=1.0), product of:
             0.21354958 = queryWeight, product of:
               2.348849 = boost
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.015427061 = queryNorm
             0.55249923 = fieldWeight in 2641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8933253 = idf(docFreq=332, maxDocs=44421)
               0.09375 = fieldNorm(doc=2641)
         0.16470896 = weight(abstract_txt:reviews in 2641) [ClassicSimilarity], result of:
           0.16470896 = score(doc=2641,freq=2.0), product of:
             0.25101298 = queryWeight, product of:
               3.2875943 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.015427061 = queryNorm
             0.65617704 = fieldWeight in 2641, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.09375 = fieldNorm(doc=2641)
       0.2 = coord(5/25)