Document (#37613)

Author

Wilson, M.J.
Wilson, M.L.

Title

¬A comparison of techniques for measuring sensemaking and learning within participant-generated summaries

Source

Journal of the American Society for Information Science and Technology. 64(2013) no.2, S.291-306

Year

2013

Abstract

While it is easy to identify whether someone has found a piece of information during a search task, it is much harder to measure how much someone has learned during the search process. Searchers who are learning often exhibit exploratory behaviors, and so current research is often focused on improving support for exploratory search. Consequently, we need effective measures of learning to demonstrate better support for exploratory search. Some approaches, such as quizzes, measure recall when learning from a fixed source of information. This research, however, focuses on techniques for measuring open-ended learning, which often involve analyzing handwritten summaries produced by participants after a task. There are two common techniques for analyzing such summaries: (a) counting facts and statements and (b) judging topic coverage. Both of these techniques, however, can be easily confounded by simple variables such as summary length. This article presents a new technique that measures depth of learning within written summaries based on Bloom's taxonomy (B.S. Bloom & M.D. Engelhart, 1956). This technique was generated using grounded theory and is designed to be less susceptible to such confounding variables. Together, these three categories of measure were compared by applying them to a large collection of written summaries produced in a task-based study, and our results provide insights into each of their strengths and weaknesses. Both fact-to-statement ratio and our own measure of depth of learning were effective while being less affected by confounding variables. Recommendations and clear areas of future work are provided to help continued research into supporting sensemaking and learning.

Theme

Referieren
Inhaltsanalyse

Similar documents (author)

1. Wilson, P.: Some fundamental concepts in information retrieval (1978) 4.39

   4.3928995 = sum of:
     4.3928995 = weight(author_txt:wilson in 839) [ClassicSimilarity], result of:
       4.3928995 = fieldWeight in 839, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.028639 = idf(docFreq=106, maxDocs=44421)
         0.625 = fieldNorm(doc=839)
   

2. Wilson, P.: Situational relevance (1971) 4.39

   4.3928995 = sum of:
     4.3928995 = weight(author_txt:wilson in 840) [ClassicSimilarity], result of:
       4.3928995 = fieldWeight in 840, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.028639 = idf(docFreq=106, maxDocs=44421)
         0.625 = fieldNorm(doc=840)
   

3. Wilson, T.D.: ¬An introduction to chain indexing (1971) 4.39

   4.3928995 = sum of:
     4.3928995 = weight(author_txt:wilson in 1335) [ClassicSimilarity], result of:
       4.3928995 = fieldWeight in 1335, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.028639 = idf(docFreq=106, maxDocs=44421)
         0.625 = fieldNorm(doc=1335)
   

4. Wilson, A.: ¬The hierarchy of belief : ideological tendentiousness in universal classification (1992) 4.39

   4.3928995 = sum of:
     4.3928995 = weight(author_txt:wilson in 2117) [ClassicSimilarity], result of:
       4.3928995 = fieldWeight in 2117, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.028639 = idf(docFreq=106, maxDocs=44421)
         0.625 = fieldNorm(doc=2117)
   

5. Wilson, F.: Article-level access in the online catalog at Vanderbilt University (1989) 4.39

   4.3928995 = sum of:
     4.3928995 = weight(author_txt:wilson in 2155) [ClassicSimilarity], result of:
       4.3928995 = fieldWeight in 2155, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.028639 = idf(docFreq=106, maxDocs=44421)
         0.625 = fieldNorm(doc=2155)
   

Similar documents (content)

1. Tenopir, C.; Wang, P.; Zhang, Y.; Simmons, B.; Pollard, R.: Academic users' interactions with ScienceDirect in search tasks : affective and cognitive behaviors (2008) 0.13

   0.13369071 = sum of:
     0.13369071 = product of:
       0.47746685 = sum of:
         0.033750016 = weight(abstract_txt:less in 3027) [ClassicSimilarity], result of:
           0.033750016 = score(doc=3027,freq=1.0), product of:
             0.10243287 = queryWeight, product of:
               1.0807682 = boost
               5.271748 = idf(docFreq=619, maxDocs=44421)
               0.017978445 = queryNorm
             0.32948425 = fieldWeight in 3027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.271748 = idf(docFreq=619, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
         0.040334903 = weight(abstract_txt:technique in 3027) [ClassicSimilarity], result of:
           0.040334903 = score(doc=3027,freq=1.0), product of:
             0.11535702 = queryWeight, product of:
               1.1469247 = boost
               5.5944448 = idf(docFreq=448, maxDocs=44421)
               0.017978445 = queryNorm
             0.3496528 = fieldWeight in 3027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5944448 = idf(docFreq=448, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
         0.053101175 = weight(abstract_txt:often in 3027) [ClassicSimilarity], result of:
           0.053101175 = score(doc=3027,freq=2.0), product of:
             0.12589636 = queryWeight, product of:
               1.4674561 = boost
               4.7719507 = idf(docFreq=1021, maxDocs=44421)
               0.017978445 = queryNorm
             0.42178482 = fieldWeight in 3027, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7719507 = idf(docFreq=1021, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
         0.03895108 = weight(abstract_txt:search in 3027) [ClassicSimilarity], result of:
           0.03895108 = score(doc=3027,freq=3.0), product of:
             0.09845533 = queryWeight, product of:
               1.4984682 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017978445 = queryNorm
             0.39562184 = fieldWeight in 3027, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
         0.04079629 = weight(abstract_txt:task in 3027) [ClassicSimilarity], result of:
           0.04079629 = score(doc=3027,freq=1.0), product of:
             0.13305593 = queryWeight, product of:
               1.5086054 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017978445 = queryNorm
             0.3066101 = fieldWeight in 3027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
         0.074014746 = weight(abstract_txt:measure in 3027) [ClassicSimilarity], result of:
           0.074014746 = score(doc=3027,freq=1.0), product of:
             0.21784395 = queryWeight, product of:
               2.2289517 = boost
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.017978445 = queryNorm
             0.3397604 = fieldWeight in 3027, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
         0.19651863 = weight(abstract_txt:learning in 3027) [ClassicSimilarity], result of:
           0.19651863 = score(doc=3027,freq=4.0), product of:
             0.33153227 = queryWeight, product of:
               3.8887122 = boost
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.017978445 = queryNorm
             0.59275866 = fieldWeight in 3027, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.0625 = fieldNorm(doc=3027)
       0.28 = coord(7/25)
   

2. Abdi, A.; Idris, N.; Alguliev, R.M.; Aliguliyev, R.M.: Automatic summarization assessment through a combination of semantic and syntactic information for intelligent educational systems (2015) 0.12

   0.12411573 = sum of:
     0.12411573 = product of:
       0.62057865 = sum of:
         0.018446038 = weight(abstract_txt:such in 3681) [ClassicSimilarity], result of:
           0.018446038 = score(doc=3681,freq=1.0), product of:
             0.08627177 = queryWeight, product of:
               1.4026918 = boost
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.017978445 = queryNorm
             0.21381313 = fieldWeight in 3681, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.0625 = fieldNorm(doc=3681)
         0.04079629 = weight(abstract_txt:task in 3681) [ClassicSimilarity], result of:
           0.04079629 = score(doc=3681,freq=1.0), product of:
             0.13305593 = queryWeight, product of:
               1.5086054 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017978445 = queryNorm
             0.3066101 = fieldWeight in 3681, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=3681)
         0.08577752 = weight(abstract_txt:techniques in 3681) [ClassicSimilarity], result of:
           0.08577752 = score(doc=3681,freq=4.0), product of:
             0.15141271 = queryWeight, product of:
               1.8582711 = boost
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.017978445 = queryNorm
             0.5665147 = fieldWeight in 3681, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.0625 = fieldNorm(doc=3681)
         0.074014746 = weight(abstract_txt:measure in 3681) [ClassicSimilarity], result of:
           0.074014746 = score(doc=3681,freq=1.0), product of:
             0.21784395 = queryWeight, product of:
               2.2289517 = boost
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.017978445 = queryNorm
             0.3397604 = fieldWeight in 3681, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4361663 = idf(docFreq=525, maxDocs=44421)
               0.0625 = fieldNorm(doc=3681)
         0.40154403 = weight(abstract_txt:summaries in 3681) [ClassicSimilarity], result of:
           0.40154403 = score(doc=3681,freq=4.0), product of:
             0.45642784 = queryWeight, product of:
               3.6071897 = boost
               7.0380287 = idf(docFreq=105, maxDocs=44421)
               0.017978445 = queryNorm
             0.8797536 = fieldWeight in 3681, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.0380287 = idf(docFreq=105, maxDocs=44421)
               0.0625 = fieldNorm(doc=3681)
       0.2 = coord(5/25)
   

3. Dumais, S.T.: Latent semantic analysis (2003) 0.12

   0.121703364 = sum of:
     0.121703364 = product of:
       0.3042584 = sum of:
         0.019988002 = weight(abstract_txt:much in 3462) [ClassicSimilarity], result of:
           0.019988002 = score(doc=3462,freq=2.0), product of:
             0.091014996 = queryWeight, product of:
               1.0187541 = boost
               4.969257 = idf(docFreq=838, maxDocs=44421)
               0.017978445 = queryNorm
             0.2196122 = fieldWeight in 3462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.969257 = idf(docFreq=838, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.019258535 = weight(abstract_txt:generated in 3462) [ClassicSimilarity], result of:
           0.019258535 = score(doc=3462,freq=1.0), product of:
             0.11186446 = queryWeight, product of:
               1.1294291 = boost
               5.509105 = idf(docFreq=488, maxDocs=44421)
               0.017978445 = queryNorm
             0.17215954 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.509105 = idf(docFreq=488, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.034931052 = weight(abstract_txt:technique in 3462) [ClassicSimilarity], result of:
           0.034931052 = score(doc=3462,freq=3.0), product of:
             0.11535702 = queryWeight, product of:
               1.1469247 = boost
               5.5944448 = idf(docFreq=448, maxDocs=44421)
               0.017978445 = queryNorm
             0.3028082 = fieldWeight in 3462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5944448 = idf(docFreq=448, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.035560247 = weight(abstract_txt:analyzing in 3462) [ClassicSimilarity], result of:
           0.035560247 = score(doc=3462,freq=2.0), product of:
             0.13363181 = queryWeight, product of:
               1.2344339 = boost
               6.021295 = idf(docFreq=292, maxDocs=44421)
               0.017978445 = queryNorm
             0.26610616 = fieldWeight in 3462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.021295 = idf(docFreq=292, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.009223019 = weight(abstract_txt:such in 3462) [ClassicSimilarity], result of:
           0.009223019 = score(doc=3462,freq=1.0), product of:
             0.08627177 = queryWeight, product of:
               1.4026918 = boost
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.017978445 = queryNorm
             0.10690656 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.026550587 = weight(abstract_txt:often in 3462) [ClassicSimilarity], result of:
           0.026550587 = score(doc=3462,freq=2.0), product of:
             0.12589636 = queryWeight, product of:
               1.4674561 = boost
               4.7719507 = idf(docFreq=1021, maxDocs=44421)
               0.017978445 = queryNorm
             0.21089241 = fieldWeight in 3462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7719507 = idf(docFreq=1021, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.01947554 = weight(abstract_txt:search in 3462) [ClassicSimilarity], result of:
           0.01947554 = score(doc=3462,freq=3.0), product of:
             0.09845533 = queryWeight, product of:
               1.4984682 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017978445 = queryNorm
             0.19781092 = fieldWeight in 3462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.05981482 = weight(abstract_txt:someone in 3462) [ClassicSimilarity], result of:
           0.05981482 = score(doc=3462,freq=1.0), product of:
             0.23813096 = queryWeight, product of:
               1.647862 = boost
               8.037906 = idf(docFreq=38, maxDocs=44421)
               0.017978445 = queryNorm
             0.25118455 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.037906 = idf(docFreq=38, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.030326935 = weight(abstract_txt:techniques in 3462) [ClassicSimilarity], result of:
           0.030326935 = score(doc=3462,freq=2.0), product of:
             0.15141271 = queryWeight, product of:
               1.8582711 = boost
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.017978445 = queryNorm
             0.20029318 = fieldWeight in 3462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
         0.049129657 = weight(abstract_txt:learning in 3462) [ClassicSimilarity], result of:
           0.049129657 = score(doc=3462,freq=1.0), product of:
             0.33153227 = queryWeight, product of:
               3.8887122 = boost
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.017978445 = queryNorm
             0.14818966 = fieldWeight in 3462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.03125 = fieldNorm(doc=3462)
       0.4 = coord(10/25)
   

4. Spina, D.; Trippas, J.R.; Cavedon, L.; Sanderson, M.: Extracting audio summaries to support effective spoken document search (2017) 0.12

   0.117438965 = sum of:
     0.117438965 = product of:
       0.73399353 = sum of:
         0.0680892 = weight(abstract_txt:generated in 4788) [ClassicSimilarity], result of:
           0.0680892 = score(doc=4788,freq=2.0), product of:
             0.11186446 = queryWeight, product of:
               1.1294291 = boost
               5.509105 = idf(docFreq=488, maxDocs=44421)
               0.017978445 = queryNorm
             0.6086759 = fieldWeight in 4788, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.509105 = idf(docFreq=488, maxDocs=44421)
               0.078125 = fieldNorm(doc=4788)
         0.023057545 = weight(abstract_txt:such in 4788) [ClassicSimilarity], result of:
           0.023057545 = score(doc=4788,freq=1.0), product of:
             0.08627177 = queryWeight, product of:
               1.4026918 = boost
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.017978445 = queryNorm
             0.2672664 = fieldWeight in 4788, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.078125 = fieldNorm(doc=4788)
         0.028110519 = weight(abstract_txt:search in 4788) [ClassicSimilarity], result of:
           0.028110519 = score(doc=4788,freq=1.0), product of:
             0.09845533 = queryWeight, product of:
               1.4984682 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.017978445 = queryNorm
             0.28551546 = fieldWeight in 4788, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.078125 = fieldNorm(doc=4788)
         0.61473626 = weight(abstract_txt:summaries in 4788) [ClassicSimilarity], result of:
           0.61473626 = score(doc=4788,freq=6.0), product of:
             0.45642784 = queryWeight, product of:
               3.6071897 = boost
               7.0380287 = idf(docFreq=105, maxDocs=44421)
               0.017978445 = queryNorm
             1.3468422 = fieldWeight in 4788, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.0380287 = idf(docFreq=105, maxDocs=44421)
               0.078125 = fieldNorm(doc=4788)
       0.16 = coord(4/25)
   

5. Zhang, P.; Soergel, D.: Towards a comprehensive model of the cognitive process and mechanisms of individual sensemaking (2014) 0.10

   0.10497118 = sum of:
     0.10497118 = product of:
       0.6560699 = sum of:
         0.018446038 = weight(abstract_txt:such in 2344) [ClassicSimilarity], result of:
           0.018446038 = score(doc=2344,freq=1.0), product of:
             0.08627177 = queryWeight, product of:
               1.4026918 = boost
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.017978445 = queryNorm
             0.21381313 = fieldWeight in 2344, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.42101 = idf(docFreq=3945, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.04079629 = weight(abstract_txt:task in 2344) [ClassicSimilarity], result of:
           0.04079629 = score(doc=2344,freq=1.0), product of:
             0.13305593 = queryWeight, product of:
               1.5086054 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.017978445 = queryNorm
             0.3066101 = fieldWeight in 2344, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.45786792 = weight(abstract_txt:sensemaking in 2344) [ClassicSimilarity], result of:
           0.45786792 = score(doc=2344,freq=10.0), product of:
             0.27044782 = queryWeight, product of:
               1.7561219 = boost
               8.565973 = idf(docFreq=22, maxDocs=44421)
               0.017978445 = queryNorm
             1.6929991 = fieldWeight in 2344, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               8.565973 = idf(docFreq=22, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
         0.13895965 = weight(abstract_txt:learning in 2344) [ClassicSimilarity], result of:
           0.13895965 = score(doc=2344,freq=2.0), product of:
             0.33153227 = queryWeight, product of:
               3.8887122 = boost
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.017978445 = queryNorm
             0.41914365 = fieldWeight in 2344, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7420692 = idf(docFreq=1052, maxDocs=44421)
               0.0625 = fieldNorm(doc=2344)
       0.16 = coord(4/25)