Document (#40355)

Author

Hu, X.
Choi, K.
Downie, J.S.

Title

¬A framework for evaluating multimodal music mood classification

Source

Journal of the Association for Information Science and Technology. 68(2017) no.2, S.273-285

Year

2017

Abstract

This research proposes a framework for music mood classification that uses multiple and complementary information sources, namely, music audio, lyric text, and social tags associated with music pieces. This article presents the framework and a thorough evaluation of each of its components. Experimental results on a large data set of 18 mood categories show that combining lyrics and audio significantly outperformed systems using audio-only features. Automatic feature selection techniques were further proved to have reduced feature space. In addition, the examination of learning curves shows that the hybrid systems using lyrics and audio needed fewer training samples and shorter audio clips to achieve the same or better classification accuracies than systems using lyrics or audio singularly. Last but not least, performance comparisons reveal the relative importance of audio and lyric features across mood categories.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23649/full.

Field

Musik

Similar documents (author)

1. Hu, X.; Lee, J.H.; Bainbridge, D.; Choi, K.; Organisciak, P.; Downie, J.S.: ¬The MIREX grand challenge : a framework of holistic user-experience evaluation in music information retrieval (2017) 3.18

   3.1810617 = sum of:
     3.1810617 = sum of:
       1.1949575 = weight(author_txt:choi in 4321) [ClassicSimilarity], result of:
         1.1949575 = score(doc=4321,freq=1.0), product of:
           0.58037704 = queryWeight, product of:
             8.235732 = idf(docFreq=31, maxDocs=44421)
             0.07047061 = queryNorm
           2.058933 = fieldWeight in 4321, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.235732 = idf(docFreq=31, maxDocs=44421)
             0.25 = fieldNorm(doc=4321)
       1.9861042 = weight(author_txt:downie in 4321) [ClassicSimilarity], result of:
         1.9861042 = score(doc=4321,freq=1.0), product of:
           0.81434786 = queryWeight, product of:
             1.1845404 = boost
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.07047061 = queryNorm
           2.4388893 = fieldWeight in 4321, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.755557 = idf(docFreq=6, maxDocs=44421)
             0.25 = fieldNorm(doc=4321)
   

2. Downie, J.S.: ¬The MusiFind Music Information Retrieval project, phase III : evaluation of indexing options (1995) 2.48

   2.4826305 = sum of:
     2.4826305 = product of:
       4.965261 = sum of:
         4.965261 = weight(author_txt:downie in 3557) [ClassicSimilarity], result of:
           4.965261 = score(doc=3557,freq=1.0), product of:
             0.81434786 = queryWeight, product of:
               1.1845404 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.07047061 = queryNorm
             6.0972233 = fieldWeight in 3557, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.625 = fieldNorm(doc=3557)
       0.5 = coord(1/2)
   

3. Downie, J.S.: ¬A sample of music information retrieval approaches (2004) 2.48

   2.4826305 = sum of:
     2.4826305 = product of:
       4.965261 = sum of:
         4.965261 = weight(author_txt:downie in 4056) [ClassicSimilarity], result of:
           4.965261 = score(doc=4056,freq=1.0), product of:
             0.81434786 = queryWeight, product of:
               1.1845404 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.07047061 = queryNorm
             6.0972233 = fieldWeight in 4056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.625 = fieldNorm(doc=4056)
       0.5 = coord(1/2)
   

4. Downie, J.S.: Music information retrieval (2002) 2.48

   2.4826305 = sum of:
     2.4826305 = product of:
       4.965261 = sum of:
         4.965261 = weight(author_txt:downie in 5287) [ClassicSimilarity], result of:
           4.965261 = score(doc=5287,freq=1.0), product of:
             0.81434786 = queryWeight, product of:
               1.1845404 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.07047061 = queryNorm
             6.0972233 = fieldWeight in 5287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.625 = fieldNorm(doc=5287)
       0.5 = coord(1/2)
   

5. Choi, Y.: Effects of contextual factors on image searching on the Web (2010) 1.49

   1.4936968 = sum of:
     1.4936968 = product of:
       2.9873936 = sum of:
         2.9873936 = weight(author_txt:choi in 982) [ClassicSimilarity], result of:
           2.9873936 = score(doc=982,freq=1.0), product of:
             0.58037704 = queryWeight, product of:
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.07047061 = queryNorm
             5.1473327 = fieldWeight in 982, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.625 = fieldNorm(doc=982)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Hu, X.; Yang, Y.-H.: ¬The mood of Chinese Pop music : representation and recognition (2017) 0.32

   0.32148826 = sum of:
     0.32148826 = product of:
       1.1481724 = sum of:
         0.0068599926 = weight(abstract_txt:that in 4755) [ClassicSimilarity], result of:
           0.0068599926 = score(doc=4755,freq=3.0), product of:
             0.026795618 = queryWeight, product of:
               1.0358231 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010938529 = queryNorm
             0.25601172 = fieldWeight in 4755, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
         0.018687878 = weight(abstract_txt:features in 4755) [ClassicSimilarity], result of:
           0.018687878 = score(doc=4755,freq=1.0), product of:
             0.065851346 = queryWeight, product of:
               1.3258379 = boost
               4.5406218 = idf(docFreq=1287, maxDocs=44421)
               0.010938529 = queryNorm
             0.28378886 = fieldWeight in 4755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5406218 = idf(docFreq=1287, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
         0.027712537 = weight(abstract_txt:categories in 4755) [ClassicSimilarity], result of:
           0.027712537 = score(doc=4755,freq=1.0), product of:
             0.0856332 = queryWeight, product of:
               1.5119213 = boost
               5.177905 = idf(docFreq=680, maxDocs=44421)
               0.010938529 = queryNorm
             0.32361907 = fieldWeight in 4755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.177905 = idf(docFreq=680, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
         0.01905149 = weight(abstract_txt:classification in 4755) [ClassicSimilarity], result of:
           0.01905149 = score(doc=4755,freq=1.0), product of:
             0.07635563 = queryWeight, product of:
               1.7485347 = boost
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.010938529 = queryNorm
             0.24950996 = fieldWeight in 4755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
         0.22602452 = weight(abstract_txt:music in 4755) [ClassicSimilarity], result of:
           0.22602452 = score(doc=4755,freq=5.0), product of:
             0.2556515 = queryWeight, product of:
               3.6944284 = boost
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.010938529 = queryNorm
             0.8841118 = fieldWeight in 4755, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
         0.6430182 = weight(abstract_txt:mood in 4755) [ClassicSimilarity], result of:
           0.6430182 = score(doc=4755,freq=4.0), product of:
             0.55292153 = queryWeight, product of:
               5.4331894 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.010938529 = queryNorm
             1.1629466 = fieldWeight in 4755, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
         0.20681772 = weight(abstract_txt:audio in 4755) [ClassicSimilarity], result of:
           0.20681772 = score(doc=4755,freq=1.0), product of:
             0.49652278 = queryWeight, product of:
               6.8110127 = boost
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.010938529 = queryNorm
             0.4165322 = fieldWeight in 4755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.0625 = fieldNorm(doc=4755)
       0.28 = coord(7/25)
   

2. Nagarajan, K.S.: Documentation of compositions in carnatic music : need for and utility of a computerized database (2006) 0.13

   0.13373324 = sum of:
     0.13373324 = product of:
       0.6686662 = sum of:
         0.0049507734 = weight(abstract_txt:that in 2500) [ClassicSimilarity], result of:
           0.0049507734 = score(doc=2500,freq=1.0), product of:
             0.026795618 = queryWeight, product of:
               1.0358231 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010938529 = queryNorm
             0.18476056 = fieldWeight in 2500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=2500)
         0.056041658 = weight(abstract_txt:pieces in 2500) [ClassicSimilarity], result of:
           0.056041658 = score(doc=2500,freq=1.0), product of:
             0.093666 = queryWeight, product of:
               1.1181089 = boost
               7.6584163 = idf(docFreq=56, maxDocs=44421)
               0.010938529 = queryNorm
             0.59831375 = fieldWeight in 2500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.6584163 = idf(docFreq=56, maxDocs=44421)
               0.078125 = fieldNorm(doc=2500)
         0.014856871 = weight(abstract_txt:systems in 2500) [ClassicSimilarity], result of:
           0.014856871 = score(doc=2500,freq=1.0), product of:
             0.055748515 = queryWeight, product of:
               1.494068 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.010938529 = queryNorm
             0.26649806 = fieldWeight in 2500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=2500)
         0.33429474 = weight(abstract_txt:music in 2500) [ClassicSimilarity], result of:
           0.33429474 = score(doc=2500,freq=7.0), product of:
             0.2556515 = queryWeight, product of:
               3.6944284 = boost
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.010938529 = queryNorm
             1.3076189 = fieldWeight in 2500, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.078125 = fieldNorm(doc=2500)
         0.25852215 = weight(abstract_txt:audio in 2500) [ClassicSimilarity], result of:
           0.25852215 = score(doc=2500,freq=1.0), product of:
             0.49652278 = queryWeight, product of:
               6.8110127 = boost
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.010938529 = queryNorm
             0.5206652 = fieldWeight in 2500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.078125 = fieldNorm(doc=2500)
       0.2 = coord(5/25)
   

3. Dubnov, S.; McAdams, S.; Reynolds, R.: Structural and affective aspects of music from statistical audio signal analysis (2006) 0.13

   0.1311258 = sum of:
     0.1311258 = product of:
       0.6556289 = sum of:
         0.007001451 = weight(abstract_txt:that in 11) [ClassicSimilarity], result of:
           0.007001451 = score(doc=11,freq=2.0), product of:
             0.026795618 = queryWeight, product of:
               1.0358231 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010938529 = queryNorm
             0.2612909 = fieldWeight in 11, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=11)
         0.014856871 = weight(abstract_txt:systems in 11) [ClassicSimilarity], result of:
           0.014856871 = score(doc=11,freq=1.0), product of:
             0.055748515 = queryWeight, product of:
               1.494068 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.010938529 = queryNorm
             0.26649806 = fieldWeight in 11, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=11)
         0.015461965 = weight(abstract_txt:using in 11) [ClassicSimilarity], result of:
           0.015461965 = score(doc=11,freq=1.0), product of:
             0.057252117 = queryWeight, product of:
               1.5140823 = boost
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.010938529 = queryNorm
             0.27006802 = fieldWeight in 11, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4568708 = idf(docFreq=3806, maxDocs=44421)
               0.078125 = fieldNorm(doc=11)
         0.25270307 = weight(abstract_txt:music in 11) [ClassicSimilarity], result of:
           0.25270307 = score(doc=11,freq=4.0), product of:
             0.2556515 = queryWeight, product of:
               3.6944284 = boost
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.010938529 = queryNorm
             0.98846704 = fieldWeight in 11, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.078125 = fieldNorm(doc=11)
         0.36560553 = weight(abstract_txt:audio in 11) [ClassicSimilarity], result of:
           0.36560553 = score(doc=11,freq=2.0), product of:
             0.49652278 = queryWeight, product of:
               6.8110127 = boost
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.010938529 = queryNorm
             0.7363318 = fieldWeight in 11, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.078125 = fieldNorm(doc=11)
       0.2 = coord(5/25)
   

4. Tzanetakis, G.; Cook, P.: Music analysis and retrieval systems for audio signals (2004) 0.13

   0.12704329 = sum of:
     0.12704329 = product of:
       0.6352165 = sum of:
         0.0049507734 = weight(abstract_txt:that in 4059) [ClassicSimilarity], result of:
           0.0049507734 = score(doc=4059,freq=1.0), product of:
             0.026795618 = queryWeight, product of:
               1.0358231 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010938529 = queryNorm
             0.18476056 = fieldWeight in 4059, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=4059)
         0.021010788 = weight(abstract_txt:systems in 4059) [ClassicSimilarity], result of:
           0.021010788 = score(doc=4059,freq=2.0), product of:
             0.055748515 = queryWeight, product of:
               1.494068 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.010938529 = queryNorm
             0.37688518 = fieldWeight in 4059, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=4059)
         0.035129897 = weight(abstract_txt:framework in 4059) [ClassicSimilarity], result of:
           0.035129897 = score(doc=4059,freq=1.0), product of:
             0.09894632 = queryWeight, product of:
               1.9904604 = boost
               4.5445113 = idf(docFreq=1282, maxDocs=44421)
               0.010938529 = queryNorm
             0.35503995 = fieldWeight in 4059, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5445113 = idf(docFreq=1282, maxDocs=44421)
               0.078125 = fieldNorm(doc=4059)
         0.12635154 = weight(abstract_txt:music in 4059) [ClassicSimilarity], result of:
           0.12635154 = score(doc=4059,freq=1.0), product of:
             0.2556515 = queryWeight, product of:
               3.6944284 = boost
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.010938529 = queryNorm
             0.49423352 = fieldWeight in 4059, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.078125 = fieldNorm(doc=4059)
         0.4477735 = weight(abstract_txt:audio in 4059) [ClassicSimilarity], result of:
           0.4477735 = score(doc=4059,freq=3.0), product of:
             0.49652278 = queryWeight, product of:
               6.8110127 = boost
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.010938529 = queryNorm
             0.90181863 = fieldWeight in 4059, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.078125 = fieldNorm(doc=4059)
       0.2 = coord(5/25)
   

5. Downie, J.S.: ¬A sample of music information retrieval approaches (2004) 0.12

   0.12432835 = sum of:
     0.12432835 = product of:
       0.62164176 = sum of:
         0.0049507734 = weight(abstract_txt:that in 4056) [ClassicSimilarity], result of:
           0.0049507734 = score(doc=4056,freq=4.0), product of:
             0.026795618 = queryWeight, product of:
               1.0358231 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010938529 = queryNorm
             0.18476056 = fieldWeight in 4056, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0390625 = fieldNorm(doc=4056)
         0.0074284356 = weight(abstract_txt:systems in 4056) [ClassicSimilarity], result of:
           0.0074284356 = score(doc=4056,freq=1.0), product of:
             0.055748515 = queryWeight, product of:
               1.494068 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.010938529 = queryNorm
             0.13324903 = fieldWeight in 4056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0390625 = fieldNorm(doc=4056)
         0.25270307 = weight(abstract_txt:music in 4056) [ClassicSimilarity], result of:
           0.25270307 = score(doc=4056,freq=16.0), product of:
             0.2556515 = queryWeight, product of:
               3.6944284 = boost
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.010938529 = queryNorm
             0.98846704 = fieldWeight in 4056, product of:
               4.0 = tf(freq=16.0), with freq of:
                 16.0 = termFreq=16.0
               6.326189 = idf(docFreq=215, maxDocs=44421)
               0.0390625 = fieldNorm(doc=4056)
         0.17375667 = weight(abstract_txt:lyrics in 4056) [ClassicSimilarity], result of:
           0.17375667 = score(doc=4056,freq=1.0), product of:
             0.45596278 = queryWeight, product of:
               4.2728577 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.010938529 = queryNorm
             0.38107646 = fieldWeight in 4056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.0390625 = fieldNorm(doc=4056)
         0.18280277 = weight(abstract_txt:audio in 4056) [ClassicSimilarity], result of:
           0.18280277 = score(doc=4056,freq=2.0), product of:
             0.49652278 = queryWeight, product of:
               6.8110127 = boost
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.010938529 = queryNorm
             0.3681659 = fieldWeight in 4056, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.664515 = idf(docFreq=153, maxDocs=44421)
               0.0390625 = fieldNorm(doc=4056)
       0.2 = coord(5/25)