Document (#40355)

Author

Hu, X.
Choi, K.
Downie, J.S.

Title

¬A framework for evaluating multimodal music mood classification

Source

Journal of the Association for Information Science and Technology. 68(2017) no.2, S.273-285

Year

2017

Abstract

This research proposes a framework for music mood classification that uses multiple and complementary information sources, namely, music audio, lyric text, and social tags associated with music pieces. This article presents the framework and a thorough evaluation of each of its components. Experimental results on a large data set of 18 mood categories show that combining lyrics and audio significantly outperformed systems using audio-only features. Automatic feature selection techniques were further proved to have reduced feature space. In addition, the examination of learning curves shows that the hybrid systems using lyrics and audio needed fewer training samples and shorter audio clips to achieve the same or better classification accuracies than systems using lyrics or audio singularly. Last but not least, performance comparisons reveal the relative importance of audio and lyric features across mood categories.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23649/full.

Field

Musik

Similar documents (author)

1. Hu, X.; Lee, J.H.; Bainbridge, D.; Choi, K.; Organisciak, P.; Downie, J.S.: ¬The MIREX grand challenge : a framework of holistic user-experience evaluation in music information retrieval (2017) 3.19

   3.1850033 = sum of:
     3.1850033 = sum of:
       1.2048826 = weight(author_txt:choi in 3321) [ClassicSimilarity], result of:
         1.2048826 = score(doc=3321,freq=1.0), product of:
           0.5832735 = queryWeight, product of:
             8.2629 = idf(docFreq=30, maxDocs=44218)
             0.07058944 = queryNorm
           2.065725 = fieldWeight in 3321, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.2629 = idf(docFreq=30, maxDocs=44218)
             0.25 = fieldNorm(doc=3321)
       1.9801208 = weight(author_txt:downie in 3321) [ClassicSimilarity], result of:
         1.9801208 = score(doc=3321,freq=1.0), product of:
           0.8122758 = queryWeight, product of:
             1.1800914 = boost
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.07058944 = queryNorm
           2.4377444 = fieldWeight in 3321, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.25 = fieldNorm(doc=3321)
   

2. Downie, J.S.: ¬The MusiFind Music Information Retrieval project, phase III : evaluation of indexing options (1995) 2.48

   2.475151 = sum of:
     2.475151 = product of:
       4.950302 = sum of:
         4.950302 = weight(author_txt:downie in 2557) [ClassicSimilarity], result of:
           4.950302 = score(doc=2557,freq=1.0), product of:
             0.8122758 = queryWeight, product of:
               1.1800914 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.07058944 = queryNorm
             6.094361 = fieldWeight in 2557, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.625 = fieldNorm(doc=2557)
       0.5 = coord(1/2)
   

3. Downie, J.S.: ¬A sample of music information retrieval approaches (2004) 2.48

   2.475151 = sum of:
     2.475151 = product of:
       4.950302 = sum of:
         4.950302 = weight(author_txt:downie in 3056) [ClassicSimilarity], result of:
           4.950302 = score(doc=3056,freq=1.0), product of:
             0.8122758 = queryWeight, product of:
               1.1800914 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.07058944 = queryNorm
             6.094361 = fieldWeight in 3056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.625 = fieldNorm(doc=3056)
       0.5 = coord(1/2)
   

4. Downie, J.S.: Music information retrieval (2002) 2.48

   2.475151 = sum of:
     2.475151 = product of:
       4.950302 = sum of:
         4.950302 = weight(author_txt:downie in 4287) [ClassicSimilarity], result of:
           4.950302 = score(doc=4287,freq=1.0), product of:
             0.8122758 = queryWeight, product of:
               1.1800914 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.07058944 = queryNorm
             6.094361 = fieldWeight in 4287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.625 = fieldNorm(doc=4287)
       0.5 = coord(1/2)
   

5. Choi, Y.: Effects of contextual factors on image searching on the Web (2010) 1.51

   1.5061033 = sum of:
     1.5061033 = product of:
       3.0122066 = sum of:
         3.0122066 = weight(author_txt:choi in 3995) [ClassicSimilarity], result of:
           3.0122066 = score(doc=3995,freq=1.0), product of:
             0.5832735 = queryWeight, product of:
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.07058944 = queryNorm
             5.164313 = fieldWeight in 3995, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.625 = fieldNorm(doc=3995)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Hu, X.; Yang, Y.-H.: ¬The mood of Chinese Pop music : representation and recognition (2017) 0.32

   0.32134613 = sum of:
     0.32134613 = product of:
       1.1476648 = sum of:
         0.006896609 = weight(abstract_txt:that in 3755) [ClassicSimilarity], result of:
           0.006896609 = score(doc=3755,freq=3.0), product of:
             0.02688703 = queryWeight, product of:
               1.0373174 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.010939036 = queryNorm
             0.2565032 = fieldWeight in 3755, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
         0.018661708 = weight(abstract_txt:features in 3755) [ClassicSimilarity], result of:
           0.018661708 = score(doc=3755,freq=1.0), product of:
             0.06578042 = queryWeight, product of:
               1.3247775 = boost
               4.5391517 = idf(docFreq=1283, maxDocs=44218)
               0.010939036 = queryNorm
             0.28369698 = fieldWeight in 3755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5391517 = idf(docFreq=1283, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
         0.027697977 = weight(abstract_txt:categories in 3755) [ClassicSimilarity], result of:
           0.027697977 = score(doc=3755,freq=1.0), product of:
             0.085590936 = queryWeight, product of:
               1.5111532 = boost
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.010939036 = queryNorm
             0.32360876 = fieldWeight in 3755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
         0.019042095 = weight(abstract_txt:classification in 3755) [ClassicSimilarity], result of:
           0.019042095 = score(doc=3755,freq=1.0), product of:
             0.07631958 = queryWeight, product of:
               1.7476652 = boost
               3.9920752 = idf(docFreq=2218, maxDocs=44218)
               0.010939036 = queryNorm
             0.2495047 = fieldWeight in 3755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9920752 = idf(docFreq=2218, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
         0.225437 = weight(abstract_txt:music in 3755) [ClassicSimilarity], result of:
           0.225437 = score(doc=3755,freq=5.0), product of:
             0.25517172 = queryWeight, product of:
               3.689996 = boost
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.010939036 = queryNorm
             0.8834717 = fieldWeight in 3755, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
         0.64179295 = weight(abstract_txt:mood in 3755) [ClassicSimilarity], result of:
           0.64179295 = score(doc=3755,freq=4.0), product of:
             0.5521398 = queryWeight, product of:
               5.427929 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.010939036 = queryNorm
             1.162374 = fieldWeight in 3755, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
         0.20813645 = weight(abstract_txt:audio in 3755) [ClassicSimilarity], result of:
           0.20813645 = score(doc=3755,freq=1.0), product of:
             0.49855974 = queryWeight, product of:
               6.8231864 = boost
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.010939036 = queryNorm
             0.41747546 = fieldWeight in 3755, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.0625 = fieldNorm(doc=3755)
       0.28 = coord(7/25)
   

2. Nagarajan, K.S.: Documentation of compositions in carnatic music : need for and utility of a computerized database (2006) 0.13

   0.13387008 = sum of:
     0.13387008 = product of:
       0.6693504 = sum of:
         0.0049771992 = weight(abstract_txt:that in 1500) [ClassicSimilarity], result of:
           0.0049771992 = score(doc=1500,freq=1.0), product of:
             0.02688703 = queryWeight, product of:
               1.0373174 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.010939036 = queryNorm
             0.18511525 = fieldWeight in 1500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.078125 = fieldNorm(doc=1500)
         0.05591711 = weight(abstract_txt:pieces in 1500) [ClassicSimilarity], result of:
           0.05591711 = score(doc=1500,freq=1.0), product of:
             0.093513764 = queryWeight, product of:
               1.116908 = boost
               7.653836 = idf(docFreq=56, maxDocs=44218)
               0.010939036 = queryNorm
             0.59795594 = fieldWeight in 1500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.653836 = idf(docFreq=56, maxDocs=44218)
               0.078125 = fieldNorm(doc=1500)
         0.01485967 = weight(abstract_txt:systems in 1500) [ClassicSimilarity], result of:
           0.01485967 = score(doc=1500,freq=1.0), product of:
             0.055747528 = queryWeight, product of:
               1.4936645 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.010939036 = queryNorm
             0.26655298 = fieldWeight in 1500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=1500)
         0.3334258 = weight(abstract_txt:music in 1500) [ClassicSimilarity], result of:
           0.3334258 = score(doc=1500,freq=7.0), product of:
             0.25517172 = queryWeight, product of:
               3.689996 = boost
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.010939036 = queryNorm
             1.3066722 = fieldWeight in 1500, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.078125 = fieldNorm(doc=1500)
         0.26017058 = weight(abstract_txt:audio in 1500) [ClassicSimilarity], result of:
           0.26017058 = score(doc=1500,freq=1.0), product of:
             0.49855974 = queryWeight, product of:
               6.8231864 = boost
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.010939036 = queryNorm
             0.5218443 = fieldWeight in 1500, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.078125 = fieldNorm(doc=1500)
       0.2 = coord(5/25)
   

3. Dubnov, S.; McAdams, S.; Reynolds, R.: Structural and affective aspects of music from statistical audio signal analysis (2006) 0.13

   0.13148415 = sum of:
     0.13148415 = product of:
       0.65742075 = sum of:
         0.007038822 = weight(abstract_txt:that in 6011) [ClassicSimilarity], result of:
           0.007038822 = score(doc=6011,freq=2.0), product of:
             0.02688703 = queryWeight, product of:
               1.0373174 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.010939036 = queryNorm
             0.26179248 = fieldWeight in 6011, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.078125 = fieldNorm(doc=6011)
         0.01485967 = weight(abstract_txt:systems in 6011) [ClassicSimilarity], result of:
           0.01485967 = score(doc=6011,freq=1.0), product of:
             0.055747528 = queryWeight, product of:
               1.4936645 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.010939036 = queryNorm
             0.26655298 = fieldWeight in 6011, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=6011)
         0.015539271 = weight(abstract_txt:using in 6011) [ClassicSimilarity], result of:
           0.015539271 = score(doc=6011,freq=1.0), product of:
             0.05743455 = queryWeight, product of:
               1.5160966 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.010939036 = queryNorm
             0.27055615 = fieldWeight in 6011, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.078125 = fieldNorm(doc=6011)
         0.25204623 = weight(abstract_txt:music in 6011) [ClassicSimilarity], result of:
           0.25204623 = score(doc=6011,freq=4.0), product of:
             0.25517172 = queryWeight, product of:
               3.689996 = boost
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.010939036 = queryNorm
             0.9877514 = fieldWeight in 6011, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.078125 = fieldNorm(doc=6011)
         0.36793676 = weight(abstract_txt:audio in 6011) [ClassicSimilarity], result of:
           0.36793676 = score(doc=6011,freq=2.0), product of:
             0.49855974 = queryWeight, product of:
               6.8231864 = boost
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.010939036 = queryNorm
             0.7379993 = fieldWeight in 6011, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.078125 = fieldNorm(doc=6011)
       0.2 = coord(5/25)
   

4. Tzanetakis, G.; Cook, P.: Music analysis and retrieval systems for audio signals (2004) 0.13

   0.12758137 = sum of:
     0.12758137 = product of:
       0.63790685 = sum of:
         0.0049771992 = weight(abstract_txt:that in 3059) [ClassicSimilarity], result of:
           0.0049771992 = score(doc=3059,freq=1.0), product of:
             0.02688703 = queryWeight, product of:
               1.0373174 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.010939036 = queryNorm
             0.18511525 = fieldWeight in 3059, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.078125 = fieldNorm(doc=3059)
         0.021014748 = weight(abstract_txt:systems in 3059) [ClassicSimilarity], result of:
           0.021014748 = score(doc=3059,freq=2.0), product of:
             0.055747528 = queryWeight, product of:
               1.4936645 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.010939036 = queryNorm
             0.37696287 = fieldWeight in 3059, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=3059)
         0.035263162 = weight(abstract_txt:framework in 3059) [ClassicSimilarity], result of:
           0.035263162 = score(doc=3059,freq=1.0), product of:
             0.09918218 = queryWeight, product of:
               1.9923108 = boost
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.010939036 = queryNorm
             0.3555393 = fieldWeight in 3059, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.078125 = fieldNorm(doc=3059)
         0.12602311 = weight(abstract_txt:music in 3059) [ClassicSimilarity], result of:
           0.12602311 = score(doc=3059,freq=1.0), product of:
             0.25517172 = queryWeight, product of:
               3.689996 = boost
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.010939036 = queryNorm
             0.4938757 = fieldWeight in 3059, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.078125 = fieldNorm(doc=3059)
         0.45062864 = weight(abstract_txt:audio in 3059) [ClassicSimilarity], result of:
           0.45062864 = score(doc=3059,freq=3.0), product of:
             0.49855974 = queryWeight, product of:
               6.8231864 = boost
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.010939036 = queryNorm
             0.90386087 = fieldWeight in 3059, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.078125 = fieldNorm(doc=3059)
       0.2 = coord(5/25)
   

5. Downie, J.S.: ¬A sample of music information retrieval approaches (2004) 0.12

   0.12437183 = sum of:
     0.12437183 = product of:
       0.62185913 = sum of:
         0.0049771992 = weight(abstract_txt:that in 3056) [ClassicSimilarity], result of:
           0.0049771992 = score(doc=3056,freq=4.0), product of:
             0.02688703 = queryWeight, product of:
               1.0373174 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.010939036 = queryNorm
             0.18511525 = fieldWeight in 3056, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0390625 = fieldNorm(doc=3056)
         0.007429835 = weight(abstract_txt:systems in 3056) [ClassicSimilarity], result of:
           0.007429835 = score(doc=3056,freq=1.0), product of:
             0.055747528 = queryWeight, product of:
               1.4936645 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.010939036 = queryNorm
             0.13327649 = fieldWeight in 3056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0390625 = fieldNorm(doc=3056)
         0.25204623 = weight(abstract_txt:music in 3056) [ClassicSimilarity], result of:
           0.25204623 = score(doc=3056,freq=16.0), product of:
             0.25517172 = queryWeight, product of:
               3.689996 = boost
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.010939036 = queryNorm
             0.9877514 = fieldWeight in 3056, product of:
               4.0 = tf(freq=16.0), with freq of:
                 16.0 = termFreq=16.0
               6.321609 = idf(docFreq=215, maxDocs=44218)
               0.0390625 = fieldNorm(doc=3056)
         0.17343752 = weight(abstract_txt:lyrics in 3056) [ClassicSimilarity], result of:
           0.17343752 = score(doc=3056,freq=1.0), product of:
             0.455339 = queryWeight, product of:
               4.2688184 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.010939036 = queryNorm
             0.38089755 = fieldWeight in 3056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.0390625 = fieldNorm(doc=3056)
         0.18396838 = weight(abstract_txt:audio in 3056) [ClassicSimilarity], result of:
           0.18396838 = score(doc=3056,freq=2.0), product of:
             0.49855974 = queryWeight, product of:
               6.8231864 = boost
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.010939036 = queryNorm
             0.36899966 = fieldWeight in 3056, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6796074 = idf(docFreq=150, maxDocs=44218)
               0.0390625 = fieldNorm(doc=3056)
       0.2 = coord(5/25)