Document (#35449)

Author

Kim, H.H.
Kim, Y.H.

Title

Toward a conceptual framework of key-frame extraction and storyboard display for video summarization

Source

Journal of the American Society for Information Science and Technology. 61(2010) no.5, S.927-939

Year

2010

Abstract

Two key problems in developing a storyboard are (a) the extraction of video key frames and (b) the display of the storyboard. On the basis of our findings from a preliminary study as well as the results of previous studies on the computerized extraction of key frames and human recognition of images and videos, we propose an algorithm for the extraction of key frames and the structural display of a storyboard. In order to evaluate the proposed algorithm, we conducted an experiment, the results of which suggest that participants produce better summaries of the given videos when they view storyboards that are composed of key frames extracted using the proposed algorithmic method. This finding held, regardless of whether the display pattern used was sequential or structural. In contrast, the experimental results suggest that in the case of employing a mechanical method, the use of a structural display pattern yields greater performance in terms of participants' ability to summarize the given videos. Elaborating on our results, we discuss the practical implications of our findings for video summarization and retrieval.

Form

Videos

Similar documents (content)

1. Tse, T.; Vegh, S.; Marchionini, G.; Shneiderman, B.: ¬An exploratory study of video browsing user interface designs and research methodologies : effectiveness in information seeking tasks (1999) 0.24

   0.23627417 = sum of:
     0.23627417 = product of:
       0.98447573 = sum of:
         0.031941928 = weight(abstract_txt:participants in 677) [ClassicSimilarity], result of:
           0.031941928 = score(doc=677,freq=1.0), product of:
             0.0954545 = queryWeight, product of:
               1.5132006 = boost
               5.3540783 = idf(docFreq=570, maxDocs=44421)
               0.011781896 = queryNorm
             0.3346299 = fieldWeight in 677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3540783 = idf(docFreq=570, maxDocs=44421)
               0.0625 = fieldNorm(doc=677)
         0.01752135 = weight(abstract_txt:results in 677) [ClassicSimilarity], result of:
           0.01752135 = score(doc=677,freq=1.0), product of:
             0.080589294 = queryWeight, product of:
               1.96631 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.011781896 = queryNorm
             0.21741535 = fieldWeight in 677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=677)
         0.14350162 = weight(abstract_txt:video in 677) [ClassicSimilarity], result of:
           0.14350162 = score(doc=677,freq=4.0), product of:
             0.18741482 = queryWeight, product of:
               2.5968442 = boost
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.011781896 = queryNorm
             0.7656898 = fieldWeight in 677, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.0625 = fieldNorm(doc=677)
         0.22302191 = weight(abstract_txt:display in 677) [ClassicSimilarity], result of:
           0.22302191 = score(doc=677,freq=4.0), product of:
             0.29813573 = queryWeight, product of:
               4.2283926 = boost
               5.98444 = idf(docFreq=303, maxDocs=44421)
               0.011781896 = queryNorm
             0.748055 = fieldWeight in 677, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.98444 = idf(docFreq=303, maxDocs=44421)
               0.0625 = fieldNorm(doc=677)
         0.18203895 = weight(abstract_txt:frames in 677) [ClassicSimilarity], result of:
           0.18203895 = score(doc=677,freq=1.0), product of:
             0.3837148 = queryWeight, product of:
               4.2905955 = boost
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.011781896 = queryNorm
             0.4744121 = fieldWeight in 677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.0625 = fieldNorm(doc=677)
         0.3864499 = weight(abstract_txt:storyboard in 677) [ClassicSimilarity], result of:
           0.3864499 = score(doc=677,freq=1.0), product of:
             0.63381296 = queryWeight, product of:
               5.514344 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.011781896 = queryNorm
             0.6097223 = fieldWeight in 677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.0625 = fieldNorm(doc=677)
       0.24 = coord(6/25)
   

2. Kim, H.H.: Toward video semantic search based on a structured folksonomy (2011) 0.19

   0.19117777 = sum of:
     0.19117777 = product of:
       0.68277776 = sum of:
         0.008258211 = weight(abstract_txt:that in 350) [ClassicSimilarity], result of:
           0.008258211 = score(doc=350,freq=4.0), product of:
             0.02793552 = queryWeight, product of:
               1.0025876 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.011781896 = queryNorm
             0.2956169 = fieldWeight in 350, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
         0.028798996 = weight(abstract_txt:proposed in 350) [ClassicSimilarity], result of:
           0.028798996 = score(doc=350,freq=2.0), product of:
             0.07070724 = queryWeight, product of:
               1.3023576 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.011781896 = queryNorm
             0.4072991 = fieldWeight in 350, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
         0.029089704 = weight(abstract_txt:suggest in 350) [ClassicSimilarity], result of:
           0.029089704 = score(doc=350,freq=1.0), product of:
             0.08968405 = queryWeight, product of:
               1.4667493 = boost
               5.189722 = idf(docFreq=672, maxDocs=44421)
               0.011781896 = queryNorm
             0.32435763 = fieldWeight in 350, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.189722 = idf(docFreq=672, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
         0.024778929 = weight(abstract_txt:results in 350) [ClassicSimilarity], result of:
           0.024778929 = score(doc=350,freq=2.0), product of:
             0.080589294 = queryWeight, product of:
               1.96631 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.011781896 = queryNorm
             0.30747172 = fieldWeight in 350, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
         0.10147097 = weight(abstract_txt:video in 350) [ClassicSimilarity], result of:
           0.10147097 = score(doc=350,freq=2.0), product of:
             0.18741482 = queryWeight, product of:
               2.5968442 = boost
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.011781896 = queryNorm
             0.54142445 = fieldWeight in 350, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
         0.10393102 = weight(abstract_txt:videos in 350) [ClassicSimilarity], result of:
           0.10393102 = score(doc=350,freq=1.0), product of:
             0.23992907 = queryWeight, product of:
               2.9382272 = boost
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.011781896 = queryNorm
             0.43317392 = fieldWeight in 350, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
         0.3864499 = weight(abstract_txt:storyboard in 350) [ClassicSimilarity], result of:
           0.3864499 = score(doc=350,freq=1.0), product of:
             0.63381296 = queryWeight, product of:
               5.514344 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.011781896 = queryNorm
             0.6097223 = fieldWeight in 350, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.0625 = fieldNorm(doc=350)
       0.28 = coord(7/25)
   

3. Kim, H.H.; Kim, Y.H.: Generic speech summarization of transcribed lecture videos : using tags and their semantic relations (2016) 0.18

   0.17606638 = sum of:
     0.17606638 = product of:
       0.48907328 = sum of:
         0.009232962 = weight(abstract_txt:that in 3640) [ClassicSimilarity], result of:
           0.009232962 = score(doc=3640,freq=5.0), product of:
             0.02793552 = queryWeight, product of:
               1.0025876 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.011781896 = queryNorm
             0.33050975 = fieldWeight in 3640, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.014754552 = weight(abstract_txt:findings in 3640) [ClassicSimilarity], result of:
           0.014754552 = score(doc=3640,freq=1.0), product of:
             0.057039145 = queryWeight, product of:
               1.1697278 = boost
               4.1387863 = idf(docFreq=1924, maxDocs=44421)
               0.011781896 = queryNorm
             0.25867414 = fieldWeight in 3640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1387863 = idf(docFreq=1924, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.07569224 = weight(abstract_txt:elaborating in 3640) [ClassicSimilarity], result of:
           0.07569224 = score(doc=3640,freq=1.0), product of:
             0.13466246 = queryWeight, product of:
               1.270886 = boost
               8.993418 = idf(docFreq=14, maxDocs=44421)
               0.011781896 = queryNorm
             0.5620886 = fieldWeight in 3640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.993418 = idf(docFreq=14, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.05359743 = weight(abstract_txt:method in 3640) [ClassicSimilarity], result of:
           0.05359743 = score(doc=3640,freq=8.0), product of:
             0.06739406 = queryWeight, product of:
               1.2714788 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.011781896 = queryNorm
             0.79528415 = fieldWeight in 3640, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.028798996 = weight(abstract_txt:proposed in 3640) [ClassicSimilarity], result of:
           0.028798996 = score(doc=3640,freq=2.0), product of:
             0.07070724 = queryWeight, product of:
               1.3023576 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.011781896 = queryNorm
             0.4072991 = fieldWeight in 3640, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.024778929 = weight(abstract_txt:results in 3640) [ClassicSimilarity], result of:
           0.024778929 = score(doc=3640,freq=2.0), product of:
             0.080589294 = queryWeight, product of:
               1.96631 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.011781896 = queryNorm
             0.30747172 = fieldWeight in 3640, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.106536336 = weight(abstract_txt:summarization in 3640) [ClassicSimilarity], result of:
           0.106536336 = score(doc=3640,freq=2.0), product of:
             0.16912618 = queryWeight, product of:
               2.0142052 = boost
               7.1267567 = idf(docFreq=96, maxDocs=44421)
               0.011781896 = queryNorm
             0.6299222 = fieldWeight in 3640, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.1267567 = idf(docFreq=96, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.07175081 = weight(abstract_txt:video in 3640) [ClassicSimilarity], result of:
           0.07175081 = score(doc=3640,freq=1.0), product of:
             0.18741482 = queryWeight, product of:
               2.5968442 = boost
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.011781896 = queryNorm
             0.3828449 = fieldWeight in 3640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
         0.10393102 = weight(abstract_txt:videos in 3640) [ClassicSimilarity], result of:
           0.10393102 = score(doc=3640,freq=1.0), product of:
             0.23992907 = queryWeight, product of:
               2.9382272 = boost
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.011781896 = queryNorm
             0.43317392 = fieldWeight in 3640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.0625 = fieldNorm(doc=3640)
       0.36 = coord(9/25)
   

4. lyer, H.; Lewis, C.D.: Prioritization strategies for video storyboard keyframes (2007) 0.16

   0.1627698 = sum of:
     0.1627698 = product of:
       0.813849 = sum of:
         0.050362237 = weight(abstract_txt:summarize in 1206) [ClassicSimilarity], result of:
           0.050362237 = score(doc=1206,freq=1.0), product of:
             0.10263189 = queryWeight, product of:
               1.1094928 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.011781896 = queryNorm
             0.4907075 = fieldWeight in 1206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.0625 = fieldNorm(doc=1206)
         0.020363964 = weight(abstract_txt:proposed in 1206) [ClassicSimilarity], result of:
           0.020363964 = score(doc=1206,freq=1.0), product of:
             0.07070724 = queryWeight, product of:
               1.3023576 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.011781896 = queryNorm
             0.28800395 = fieldWeight in 1206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=1206)
         0.12427604 = weight(abstract_txt:video in 1206) [ClassicSimilarity], result of:
           0.12427604 = score(doc=1206,freq=3.0), product of:
             0.18741482 = queryWeight, product of:
               2.5968442 = boost
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.011781896 = queryNorm
             0.6631068 = fieldWeight in 1206, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.0625 = fieldNorm(doc=1206)
         0.23239681 = weight(abstract_txt:videos in 1206) [ClassicSimilarity], result of:
           0.23239681 = score(doc=1206,freq=5.0), product of:
             0.23992907 = queryWeight, product of:
               2.9382272 = boost
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.011781896 = queryNorm
             0.96860635 = fieldWeight in 1206, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.0625 = fieldNorm(doc=1206)
         0.3864499 = weight(abstract_txt:storyboard in 1206) [ClassicSimilarity], result of:
           0.3864499 = score(doc=1206,freq=1.0), product of:
             0.63381296 = queryWeight, product of:
               5.514344 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.011781896 = queryNorm
             0.6097223 = fieldWeight in 1206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.0625 = fieldNorm(doc=1206)
       0.2 = coord(5/25)
   

5. Lee, Y.-S.; Wu, Y.-C.; Yang, J.-C.: BVideoQA : Online English/Chinese bilingual video question answering (2009) 0.13

   0.12550254 = sum of:
     0.12550254 = product of:
       0.39219546 = sum of:
         0.0041291057 = weight(abstract_txt:that in 3739) [ClassicSimilarity], result of:
           0.0041291057 = score(doc=3739,freq=1.0), product of:
             0.02793552 = queryWeight, product of:
               1.0025876 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.011781896 = queryNorm
             0.14780845 = fieldWeight in 3739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.03282159 = weight(abstract_txt:method in 3739) [ClassicSimilarity], result of:
           0.03282159 = score(doc=3739,freq=3.0), product of:
             0.06739406 = queryWeight, product of:
               1.2714788 = boost
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.011781896 = queryNorm
             0.4870101 = fieldWeight in 3739, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4988065 = idf(docFreq=1342, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.020363964 = weight(abstract_txt:proposed in 3739) [ClassicSimilarity], result of:
           0.020363964 = score(doc=3739,freq=1.0), product of:
             0.07070724 = queryWeight, product of:
               1.3023576 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.011781896 = queryNorm
             0.28800395 = fieldWeight in 3739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.0386436 = weight(abstract_txt:algorithm in 3739) [ClassicSimilarity], result of:
           0.0386436 = score(doc=3739,freq=1.0), product of:
             0.108377896 = queryWeight, product of:
               1.6123847 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.011781896 = queryNorm
             0.35656348 = fieldWeight in 3739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.050508797 = weight(abstract_txt:pattern in 3739) [ClassicSimilarity], result of:
           0.050508797 = score(doc=3739,freq=1.0), product of:
             0.12955883 = queryWeight, product of:
               1.7629168 = boost
               6.2376356 = idf(docFreq=235, maxDocs=44421)
               0.011781896 = queryNorm
             0.38985223 = fieldWeight in 3739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2376356 = idf(docFreq=235, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.01752135 = weight(abstract_txt:results in 3739) [ClassicSimilarity], result of:
           0.01752135 = score(doc=3739,freq=1.0), product of:
             0.080589294 = queryWeight, product of:
               1.96631 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.011781896 = queryNorm
             0.21741535 = fieldWeight in 3739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.12427604 = weight(abstract_txt:video in 3739) [ClassicSimilarity], result of:
           0.12427604 = score(doc=3739,freq=3.0), product of:
             0.18741482 = queryWeight, product of:
               2.5968442 = boost
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.011781896 = queryNorm
             0.6631068 = fieldWeight in 3739, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1255183 = idf(docFreq=263, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
         0.10393102 = weight(abstract_txt:videos in 3739) [ClassicSimilarity], result of:
           0.10393102 = score(doc=3739,freq=1.0), product of:
             0.23992907 = queryWeight, product of:
               2.9382272 = boost
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.011781896 = queryNorm
             0.43317392 = fieldWeight in 3739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.0625 = fieldNorm(doc=3739)
       0.32 = coord(8/25)