Document (#41686)

Author

Kim, H.H.
Kim, Y.H.

Title

Video summarization using event-related potential responses to shot boundaries in real-time video watching

Source

Journal of the Association for Information Science and Technology. 70(2019) no.2, S.164-175

Year

2019

Abstract

Our aim was to develop an event-related potential (ERP)-based method to construct a video skim consisting of key shots to bridge the semantic gap between the topic inferred from a whole video and that from its summary. Mayer's cognitive model was examined, wherein the topic integration process of a user evoked by a visual stimulus can be associated with long-latency ERP components. We determined that long-latency ERP components are suitable for measuring a user's neuronal response through a literature review. We hypothesized that N300 is specific to the categorization of all shots regardless of topic relevance, N400 is specific for the semantic mismatching process for topic-irrelevant shots, and P600 is specific for the context updating process for topic-relevant shots. In our experiment, the N400 component led to more negative ERP signals in response to topic-irrelevant shots than to topic-relevant shots and showed a fronto-central scalp pattern. P600 elicited more positive ERP signals for topic-relevant shots than for topic-irrelevant shots and showed a fronto-central scalp pattern. We used discriminant and artificial neural network (ANN) analyses to decode video shot relevance and observed that the ANN produced particularly high success rates: 91.3% from the training set and 100% from the test set.

Content

Vgl.: https://onlinelibrary.wiley.com/doi/10.1002/asi.24103.

Theme

Automatisches Abstracting

Form

Videos

Similar documents (content)

1. Kim, H.H.; Kim, Y.H.: ERP/MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos (2019) 0.84

   0.84249985 = sum of:
     0.84249985 = product of:
       1.504464 = sum of:
         0.05778276 = weight(abstract_txt:discriminant in 5358) [ClassicSimilarity], result of:
           0.05778276 = score(doc=5358,freq=1.0), product of:
             0.10285248 = queryWeight, product of:
               1.0044917 = boost
               8.988837 = idf(docFreq=14, maxDocs=44218)
               0.01139108 = queryNorm
             0.5618023 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.988837 = idf(docFreq=14, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.015823405 = weight(abstract_txt:potential in 5358) [ClassicSimilarity], result of:
           0.015823405 = score(doc=5358,freq=1.0), product of:
             0.054646105 = queryWeight, product of:
               1.0354606 = boost
               4.632983 = idf(docFreq=1168, maxDocs=44218)
               0.01139108 = queryNorm
             0.28956145 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.632983 = idf(docFreq=1168, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.007332715 = weight(abstract_txt:that in 5358) [ClassicSimilarity], result of:
           0.007332715 = score(doc=5358,freq=3.0), product of:
             0.028587228 = queryWeight, product of:
               1.0591439 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.01139108 = queryNorm
             0.2565032 = fieldWeight in 5358, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.042680725 = weight(abstract_txt:relevance in 5358) [ClassicSimilarity], result of:
           0.042680725 = score(doc=5358,freq=5.0), product of:
             0.061923727 = queryWeight, product of:
               1.1022561 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.01139108 = queryNorm
             0.6892467 = fieldWeight in 5358, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.027758945 = weight(abstract_txt:components in 5358) [ClassicSimilarity], result of:
           0.027758945 = score(doc=5358,freq=1.0), product of:
             0.07948671 = queryWeight, product of:
               1.248824 = boost
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.01139108 = queryNorm
             0.3492275 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.03090983 = weight(abstract_txt:showed in 5358) [ClassicSimilarity], result of:
           0.03090983 = score(doc=5358,freq=1.0), product of:
             0.08539327 = queryWeight, product of:
               1.2943919 = boost
               5.7915254 = idf(docFreq=366, maxDocs=44218)
               0.01139108 = queryNorm
             0.36197034 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7915254 = idf(docFreq=366, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.035480067 = weight(abstract_txt:process in 5358) [ClassicSimilarity], result of:
           0.035480067 = score(doc=5358,freq=5.0), product of:
             0.062669456 = queryWeight, product of:
               1.3580868 = boost
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.01139108 = queryNorm
             0.5661461 = fieldWeight in 5358, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.02377462 = weight(abstract_txt:relevant in 5358) [ClassicSimilarity], result of:
           0.02377462 = score(doc=5358,freq=1.0), product of:
             0.0820601 = queryWeight, product of:
               1.5540522 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.01139108 = queryNorm
             0.28972206 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.05470695 = weight(abstract_txt:event in 5358) [ClassicSimilarity], result of:
           0.05470695 = score(doc=5358,freq=1.0), product of:
             0.12494558 = queryWeight, product of:
               1.5657212 = boost
               7.0055394 = idf(docFreq=108, maxDocs=44218)
               0.01139108 = queryNorm
             0.4378462 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0055394 = idf(docFreq=108, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.17851833 = weight(abstract_txt:shot in 5358) [ClassicSimilarity], result of:
           0.17851833 = score(doc=5358,freq=3.0), product of:
             0.19059318 = queryWeight, product of:
               1.9337826 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.01139108 = queryNorm
             0.9366459 = fieldWeight in 5358, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.09751234 = weight(abstract_txt:irrelevant in 5358) [ClassicSimilarity], result of:
           0.09751234 = score(doc=5358,freq=1.0), product of:
             0.21026292 = queryWeight, product of:
               2.4876022 = boost
               7.4202213 = idf(docFreq=71, maxDocs=44218)
               0.01139108 = queryNorm
             0.46376383 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4202213 = idf(docFreq=71, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.24271192 = weight(abstract_txt:video in 5358) [ClassicSimilarity], result of:
           0.24271192 = score(doc=5358,freq=7.0), product of:
             0.2393503 = queryWeight, product of:
               3.4264228 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.01139108 = queryNorm
             1.0140448 = fieldWeight in 5358, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.26272842 = weight(abstract_txt:topic in 5358) [ClassicSimilarity], result of:
           0.26272842 = score(doc=5358,freq=8.0), product of:
             0.29358786 = queryWeight, product of:
               5.091307 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.01139108 = queryNorm
             0.8948885 = fieldWeight in 5358, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
         0.426743 = weight(abstract_txt:shots in 5358) [ClassicSimilarity], result of:
           0.426743 = score(doc=5358,freq=1.0), product of:
             0.7801118 = queryWeight, product of:
               7.824604 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.01139108 = queryNorm
             0.547028 = fieldWeight in 5358, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=5358)
       0.56 = coord(14/25)
   

2. Chua, T.-S.; Ruan, L.-Q.: ¬A video retrieval and sequencing system (1995) 0.35

   0.34814525 = sum of:
     0.34814525 = product of:
       1.4506053 = sum of:
         0.019833962 = weight(abstract_txt:process in 3838) [ClassicSimilarity], result of:
           0.019833962 = score(doc=3838,freq=1.0), product of:
             0.062669456 = queryWeight, product of:
               1.3580868 = boost
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.01139108 = queryNorm
             0.3164853 = fieldWeight in 3838, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.078125 = fieldNorm(doc=3838)
         0.023955321 = weight(abstract_txt:specific in 3838) [ClassicSimilarity], result of:
           0.023955321 = score(doc=3838,freq=1.0), product of:
             0.07107513 = queryWeight, product of:
               1.4463001 = boost
               4.314141 = idf(docFreq=1607, maxDocs=44218)
               0.01139108 = queryNorm
             0.33704224 = fieldWeight in 3838, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.314141 = idf(docFreq=1607, maxDocs=44218)
               0.078125 = fieldNorm(doc=3838)
         0.029718276 = weight(abstract_txt:relevant in 3838) [ClassicSimilarity], result of:
           0.029718276 = score(doc=3838,freq=1.0), product of:
             0.0820601 = queryWeight, product of:
               1.5540522 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.01139108 = queryNorm
             0.36215258 = fieldWeight in 3838, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.078125 = fieldNorm(doc=3838)
         0.12883452 = weight(abstract_txt:shot in 3838) [ClassicSimilarity], result of:
           0.12883452 = score(doc=3838,freq=1.0), product of:
             0.19059318 = queryWeight, product of:
               1.9337826 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.01139108 = queryNorm
             0.675966 = fieldWeight in 3838, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.078125 = fieldNorm(doc=3838)
         0.32433748 = weight(abstract_txt:video in 3838) [ClassicSimilarity], result of:
           0.32433748 = score(doc=3838,freq=8.0), product of:
             0.2393503 = queryWeight, product of:
               3.4264228 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.01139108 = queryNorm
             1.3550744 = fieldWeight in 3838, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.078125 = fieldNorm(doc=3838)
         0.9239257 = weight(abstract_txt:shots in 3838) [ClassicSimilarity], result of:
           0.9239257 = score(doc=3838,freq=3.0), product of:
             0.7801118 = queryWeight, product of:
               7.824604 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.01139108 = queryNorm
             1.1843504 = fieldWeight in 3838, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.078125 = fieldNorm(doc=3838)
       0.24 = coord(6/25)
   

3. Kannan, R.; Ghinea, G.; Swaminathan, S.: What do you wish to see? : A summarization system for movies based on user preferences (2015) 0.27

   0.27263087 = sum of:
     0.27263087 = product of:
       0.9736817 = sum of:
         0.013845479 = weight(abstract_txt:potential in 2683) [ClassicSimilarity], result of:
           0.013845479 = score(doc=2683,freq=1.0), product of:
             0.054646105 = queryWeight, product of:
               1.0354606 = boost
               4.632983 = idf(docFreq=1168, maxDocs=44218)
               0.01139108 = queryNorm
             0.25336626 = fieldWeight in 2683, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.632983 = idf(docFreq=1168, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
         0.0037043523 = weight(abstract_txt:that in 2683) [ClassicSimilarity], result of:
           0.0037043523 = score(doc=2683,freq=1.0), product of:
             0.028587228 = queryWeight, product of:
               1.0591439 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.01139108 = queryNorm
             0.12958068 = fieldWeight in 2683, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
         0.016701477 = weight(abstract_txt:relevance in 2683) [ClassicSimilarity], result of:
           0.016701477 = score(doc=2683,freq=1.0), product of:
             0.061923727 = queryWeight, product of:
               1.1022561 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.01139108 = queryNorm
             0.26971045 = fieldWeight in 2683, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
         0.0058791507 = weight(abstract_txt:from in 2683) [ClassicSimilarity], result of:
           0.0058791507 = score(doc=2683,freq=1.0), product of:
             0.038896177 = queryWeight, product of:
               1.2354422 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01139108 = queryNorm
             0.15114984 = fieldWeight in 2683, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
         0.09018415 = weight(abstract_txt:shot in 2683) [ClassicSimilarity], result of:
           0.09018415 = score(doc=2683,freq=1.0), product of:
             0.19059318 = queryWeight, product of:
               1.9337826 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.01139108 = queryNorm
             0.47317618 = fieldWeight in 2683, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
         0.19661914 = weight(abstract_txt:video in 2683) [ClassicSimilarity], result of:
           0.19661914 = score(doc=2683,freq=6.0), product of:
             0.2393503 = queryWeight, product of:
               3.4264228 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.01139108 = queryNorm
             0.8214702 = fieldWeight in 2683, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
         0.64674795 = weight(abstract_txt:shots in 2683) [ClassicSimilarity], result of:
           0.64674795 = score(doc=2683,freq=3.0), product of:
             0.7801118 = queryWeight, product of:
               7.824604 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.01139108 = queryNorm
             0.82904524 = fieldWeight in 2683, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2683)
       0.28 = coord(7/25)
   

4. Turner, J.M.: Comparing user-assigned terms with indexer-assigned terms for storage and retrieval of moving images : research results (1995) 0.25

   0.25438267 = sum of:
     0.25438267 = product of:
       1.5898917 = sum of:
         0.01745655 = weight(abstract_txt:from in 3831) [ClassicSimilarity], result of:
           0.01745655 = score(doc=3831,freq=3.0), product of:
             0.038896177 = queryWeight, product of:
               1.2354422 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01139108 = queryNorm
             0.4487986 = fieldWeight in 3831, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.09375 = fieldNorm(doc=3831)
         0.15460141 = weight(abstract_txt:shot in 3831) [ClassicSimilarity], result of:
           0.15460141 = score(doc=3831,freq=1.0), product of:
             0.19059318 = queryWeight, product of:
               1.9337826 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.01139108 = queryNorm
             0.8111592 = fieldWeight in 3831, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.09375 = fieldNorm(doc=3831)
         0.13760474 = weight(abstract_txt:video in 3831) [ClassicSimilarity], result of:
           0.13760474 = score(doc=3831,freq=1.0), product of:
             0.2393503 = queryWeight, product of:
               3.4264228 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.01139108 = queryNorm
             0.5749094 = fieldWeight in 3831, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.09375 = fieldNorm(doc=3831)
         1.280229 = weight(abstract_txt:shots in 3831) [ClassicSimilarity], result of:
           1.280229 = score(doc=3831,freq=4.0), product of:
             0.7801118 = queryWeight, product of:
               7.824604 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.01139108 = queryNorm
             1.641084 = fieldWeight in 3831, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.09375 = fieldNorm(doc=3831)
       0.16 = coord(4/25)
   

5. Turner, J.M.: Cross-language transfer of indexing concepts for storage and retrieval of moving images : preliminary results (1996) 0.17

   0.16894226 = sum of:
     0.16894226 = product of:
       0.8447113 = sum of:
         0.03356651 = weight(abstract_txt:potential in 7400) [ClassicSimilarity], result of:
           0.03356651 = score(doc=7400,freq=2.0), product of:
             0.054646105 = queryWeight, product of:
               1.0354606 = boost
               4.632983 = idf(docFreq=1168, maxDocs=44218)
               0.01139108 = queryNorm
             0.61425257 = fieldWeight in 7400, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.632983 = idf(docFreq=1168, maxDocs=44218)
               0.09375 = fieldNorm(doc=7400)
         0.006350318 = weight(abstract_txt:that in 7400) [ClassicSimilarity], result of:
           0.006350318 = score(doc=7400,freq=1.0), product of:
             0.028587228 = queryWeight, product of:
               1.0591439 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.01139108 = queryNorm
             0.22213829 = fieldWeight in 7400, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.09375 = fieldNorm(doc=7400)
         0.010078544 = weight(abstract_txt:from in 7400) [ClassicSimilarity], result of:
           0.010078544 = score(doc=7400,freq=1.0), product of:
             0.038896177 = queryWeight, product of:
               1.2354422 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01139108 = queryNorm
             0.259114 = fieldWeight in 7400, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.09375 = fieldNorm(doc=7400)
         0.15460141 = weight(abstract_txt:shot in 7400) [ClassicSimilarity], result of:
           0.15460141 = score(doc=7400,freq=1.0), product of:
             0.19059318 = queryWeight, product of:
               1.9337826 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.01139108 = queryNorm
             0.8111592 = fieldWeight in 7400, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.09375 = fieldNorm(doc=7400)
         0.6401145 = weight(abstract_txt:shots in 7400) [ClassicSimilarity], result of:
           0.6401145 = score(doc=7400,freq=1.0), product of:
             0.7801118 = queryWeight, product of:
               7.824604 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.01139108 = queryNorm
             0.820542 = fieldWeight in 7400, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.09375 = fieldNorm(doc=7400)
       0.2 = coord(5/25)