Document (#43882)

Author

Sluis, F. van der
Broek, E.L. van den

Title

Feedback beyond accuracy : using eye-tracking to detect comprehensibility and interest during reading

Source

Journal of the Association for Information Science and Technology. 74(2023) no.1, S.3-16

Year

2023

Abstract

Knowing what information a user wants is a paramount challenge to information science and technology. Implicit feedback is key to solving this challenge, as it allows information systems to learn about a user's needs and preferences. The available feedback, however, tends to be limited and its interpretation shows to be difficult. To tackle this challenge, we present a user study that explores whether tracking the eyes can unpack part of the complexity inherent to relevance and relevance decisions. The eye behavior of 30 participants reading 18 news articles was compared with their subjectively appraised comprehensibility and interest at a discourse level. Using linear regression models, the eye-tracking signal explained 49.93% (comprehensibility) and 30.41% (interest) of variance (p < .001). We conclude that eye behavior provides implicit feedback beyond accuracy that enables new forms of adaptation and interaction support for personalized information systems.

Content

Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24657. https://doi.org/10.1002/asi.24657.

Theme

Benutzerstudien

Similar documents (content)

1. Lorigo, L.; Pan, B.; Hembrooke, H.; Joachims, T.; Granka, L.; Gay, G.: ¬The influence of task and gender on search and evaluation behavior using Google (2006) 0.21

   0.20715526 = sum of:
     0.20715526 = product of:
       0.5754312 = sum of:
         0.009244103 = weight(abstract_txt:that in 978) [ClassicSimilarity], result of:
           0.009244103 = score(doc=978,freq=3.0), product of:
             0.03603894 = queryWeight, product of:
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015209672 = queryNorm
             0.2565032 = fieldWeight in 978, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.013367621 = weight(abstract_txt:user in 978) [ClassicSimilarity], result of:
           0.013367621 = score(doc=978,freq=1.0), product of:
             0.05806417 = queryWeight, product of:
               1.0363883 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.015209672 = queryNorm
             0.23022151 = fieldWeight in 978, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.007590004 = weight(abstract_txt:information in 978) [ClassicSimilarity], result of:
           0.007590004 = score(doc=978,freq=1.0), product of:
             0.050162226 = queryWeight, product of:
               1.3622969 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.015209672 = queryNorm
             0.15130915 = fieldWeight in 978, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.053466942 = weight(abstract_txt:behavior in 978) [ClassicSimilarity], result of:
           0.053466942 = score(doc=978,freq=2.0), product of:
             0.116123214 = queryWeight, product of:
               1.465642 = boost
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.015209672 = queryNorm
             0.46043286 = fieldWeight in 978, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.050954897 = weight(abstract_txt:beyond in 978) [ClassicSimilarity], result of:
           0.050954897 = score(doc=978,freq=1.0), product of:
             0.14168681 = queryWeight, product of:
               1.6189487 = boost
               5.754088 = idf(docFreq=380, maxDocs=44218)
               0.015209672 = queryNorm
             0.3596305 = fieldWeight in 978, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.754088 = idf(docFreq=380, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.080920935 = weight(abstract_txt:implicit in 978) [ClassicSimilarity], result of:
           0.080920935 = score(doc=978,freq=1.0), product of:
             0.19286174 = queryWeight, product of:
               1.8888236 = boost
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.015209672 = queryNorm
             0.41958004 = fieldWeight in 978, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.05346756 = weight(abstract_txt:interest in 978) [ClassicSimilarity], result of:
           0.05346756 = score(doc=978,freq=1.0), product of:
             0.16747995 = queryWeight, product of:
               2.1557353 = boost
               5.1079607 = idf(docFreq=726, maxDocs=44218)
               0.015209672 = queryNorm
             0.31924754 = fieldWeight in 978, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1079607 = idf(docFreq=726, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.14752539 = weight(abstract_txt:tracking in 978) [ClassicSimilarity], result of:
           0.14752539 = score(doc=978,freq=1.0), product of:
             0.32946837 = queryWeight, product of:
               3.0235758 = boost
               7.1642876 = idf(docFreq=92, maxDocs=44218)
               0.015209672 = queryNorm
             0.44776797 = fieldWeight in 978, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1642876 = idf(docFreq=92, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
         0.15889373 = weight(abstract_txt:feedback in 978) [ClassicSimilarity], result of:
           0.15889373 = score(doc=978,freq=2.0), product of:
             0.30241963 = queryWeight, product of:
               3.3449404 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.015209672 = queryNorm
             0.52540815 = fieldWeight in 978, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.0625 = fieldNorm(doc=978)
       0.36 = coord(9/25)
   

2. Stock, W.G.: On relevance distributions (2006) 0.19

   0.18598072 = sum of:
     0.18598072 = product of:
       0.6642169 = sum of:
         0.0093399 = weight(abstract_txt:that in 5116) [ClassicSimilarity], result of:
           0.0093399 = score(doc=5116,freq=1.0), product of:
             0.03603894 = queryWeight, product of:
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015209672 = queryNorm
             0.25916135 = fieldWeight in 5116, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
         0.023393337 = weight(abstract_txt:user in 5116) [ClassicSimilarity], result of:
           0.023393337 = score(doc=5116,freq=1.0), product of:
             0.05806417 = queryWeight, product of:
               1.0363883 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.015209672 = queryNorm
             0.40288764 = fieldWeight in 5116, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
         0.013282507 = weight(abstract_txt:information in 5116) [ClassicSimilarity], result of:
           0.013282507 = score(doc=5116,freq=1.0), product of:
             0.050162226 = queryWeight, product of:
               1.3622969 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.015209672 = queryNorm
             0.264791 = fieldWeight in 5116, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
         0.097248785 = weight(abstract_txt:relevance in 5116) [ClassicSimilarity], result of:
           0.097248785 = score(doc=5116,freq=3.0), product of:
             0.10408684 = queryWeight, product of:
               1.3876064 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015209672 = queryNorm
             0.93430436 = fieldWeight in 5116, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
         0.06616197 = weight(abstract_txt:behavior in 5116) [ClassicSimilarity], result of:
           0.06616197 = score(doc=5116,freq=1.0), product of:
             0.116123214 = queryWeight, product of:
               1.465642 = boost
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.015209672 = queryNorm
             0.5697566 = fieldWeight in 5116, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
         0.2581694 = weight(abstract_txt:tracking in 5116) [ClassicSimilarity], result of:
           0.2581694 = score(doc=5116,freq=1.0), product of:
             0.32946837 = queryWeight, product of:
               3.0235758 = boost
               7.1642876 = idf(docFreq=92, maxDocs=44218)
               0.015209672 = queryNorm
             0.78359395 = fieldWeight in 5116, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1642876 = idf(docFreq=92, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
         0.19662097 = weight(abstract_txt:feedback in 5116) [ClassicSimilarity], result of:
           0.19662097 = score(doc=5116,freq=1.0), product of:
             0.30241963 = queryWeight, product of:
               3.3449404 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.015209672 = queryNorm
             0.6501594 = fieldWeight in 5116, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.109375 = fieldNorm(doc=5116)
       0.28 = coord(7/25)
   

3. Jung, S.; Herlocker, J.L.; Webster, J.: Click data as implicit relevance feedback in web search (2007) 0.14

   0.13665698 = sum of:
     0.13665698 = product of:
       0.48806065 = sum of:
         0.012355525 = weight(abstract_txt:that in 912) [ClassicSimilarity], result of:
           0.012355525 = score(doc=912,freq=7.0), product of:
             0.03603894 = queryWeight, product of:
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015209672 = queryNorm
             0.3428382 = fieldWeight in 912, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
         0.016541587 = weight(abstract_txt:user in 912) [ClassicSimilarity], result of:
           0.016541587 = score(doc=912,freq=2.0), product of:
             0.05806417 = queryWeight, product of:
               1.0363883 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.015209672 = queryNorm
             0.28488457 = fieldWeight in 912, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
         0.009392151 = weight(abstract_txt:information in 912) [ClassicSimilarity], result of:
           0.009392151 = score(doc=912,freq=2.0), product of:
             0.050162226 = queryWeight, product of:
               1.3622969 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.015209672 = queryNorm
             0.18723552 = fieldWeight in 912, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
         0.07427498 = weight(abstract_txt:relevance in 912) [ClassicSimilarity], result of:
           0.07427498 = score(doc=912,freq=7.0), product of:
             0.10408684 = queryWeight, product of:
               1.3876064 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015209672 = queryNorm
             0.7135867 = fieldWeight in 912, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
         0.044585533 = weight(abstract_txt:beyond in 912) [ClassicSimilarity], result of:
           0.044585533 = score(doc=912,freq=1.0), product of:
             0.14168681 = queryWeight, product of:
               1.6189487 = boost
               5.754088 = idf(docFreq=380, maxDocs=44218)
               0.015209672 = queryNorm
             0.31467667 = fieldWeight in 912, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.754088 = idf(docFreq=380, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
         0.07080582 = weight(abstract_txt:implicit in 912) [ClassicSimilarity], result of:
           0.07080582 = score(doc=912,freq=1.0), product of:
             0.19286174 = queryWeight, product of:
               1.8888236 = boost
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.015209672 = queryNorm
             0.36713254 = fieldWeight in 912, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
         0.26010507 = weight(abstract_txt:feedback in 912) [ClassicSimilarity], result of:
           0.26010507 = score(doc=912,freq=7.0), product of:
             0.30241963 = queryWeight, product of:
               3.3449404 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.015209672 = queryNorm
             0.86008 = fieldWeight in 912, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.0546875 = fieldNorm(doc=912)
       0.28 = coord(7/25)
   

4. Zhang, Y.: Complex adaptive filtering user profile using graphical models (2008) 0.14

   0.13558312 = sum of:
     0.13558312 = product of:
       0.48422542 = sum of:
         0.011555129 = weight(abstract_txt:that in 2445) [ClassicSimilarity], result of:
           0.011555129 = score(doc=2445,freq=3.0), product of:
             0.03603894 = queryWeight, product of:
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015209672 = queryNorm
             0.320629 = fieldWeight in 2445, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
         0.05012858 = weight(abstract_txt:user in 2445) [ClassicSimilarity], result of:
           0.05012858 = score(doc=2445,freq=9.0), product of:
             0.05806417 = queryWeight, product of:
               1.0363883 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.015209672 = queryNorm
             0.86333066 = fieldWeight in 2445, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
         0.018975008 = weight(abstract_txt:information in 2445) [ClassicSimilarity], result of:
           0.018975008 = score(doc=2445,freq=4.0), product of:
             0.050162226 = queryWeight, product of:
               1.3622969 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.015209672 = queryNorm
             0.37827286 = fieldWeight in 2445, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
         0.040104724 = weight(abstract_txt:relevance in 2445) [ClassicSimilarity], result of:
           0.040104724 = score(doc=2445,freq=1.0), product of:
             0.10408684 = queryWeight, product of:
               1.3876064 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015209672 = queryNorm
             0.38530064 = fieldWeight in 2445, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
         0.06369362 = weight(abstract_txt:beyond in 2445) [ClassicSimilarity], result of:
           0.06369362 = score(doc=2445,freq=1.0), product of:
             0.14168681 = queryWeight, product of:
               1.6189487 = boost
               5.754088 = idf(docFreq=380, maxDocs=44218)
               0.015209672 = queryNorm
             0.4495381 = fieldWeight in 2445, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.754088 = idf(docFreq=380, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
         0.10115117 = weight(abstract_txt:implicit in 2445) [ClassicSimilarity], result of:
           0.10115117 = score(doc=2445,freq=1.0), product of:
             0.19286174 = queryWeight, product of:
               1.8888236 = boost
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.015209672 = queryNorm
             0.52447504 = fieldWeight in 2445, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
         0.19861719 = weight(abstract_txt:feedback in 2445) [ClassicSimilarity], result of:
           0.19861719 = score(doc=2445,freq=2.0), product of:
             0.30241963 = queryWeight, product of:
               3.3449404 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.015209672 = queryNorm
             0.6567602 = fieldWeight in 2445, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.078125 = fieldNorm(doc=2445)
       0.28 = coord(7/25)
   

5. Sela, M.; Lavie, T.; Inbar, O.; Oppenheim, I.; Meyer, J.: Personalizing news content : an experimental study (2015) 0.13

   0.1339521 = sum of:
     0.1339521 = product of:
       0.47840035 = sum of:
         0.0075477785 = weight(abstract_txt:that in 1604) [ClassicSimilarity], result of:
           0.0075477785 = score(doc=1604,freq=2.0), product of:
             0.03603894 = queryWeight, product of:
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015209672 = queryNorm
             0.20943399 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
         0.122685105 = weight(abstract_txt:personalized in 1604) [ClassicSimilarity], result of:
           0.122685105 = score(doc=1604,freq=6.0), product of:
             0.11117478 = queryWeight, product of:
               1.0140433 = boost
               7.208251 = idf(docFreq=88, maxDocs=44218)
               0.015209672 = queryNorm
             1.1035336 = fieldWeight in 1604, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.208251 = idf(docFreq=88, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
         0.026735242 = weight(abstract_txt:user in 1604) [ClassicSimilarity], result of:
           0.026735242 = score(doc=1604,freq=4.0), product of:
             0.05806417 = queryWeight, product of:
               1.0363883 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.015209672 = queryNorm
             0.46044302 = fieldWeight in 1604, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
         0.007590004 = weight(abstract_txt:information in 1604) [ClassicSimilarity], result of:
           0.007590004 = score(doc=1604,freq=1.0), product of:
             0.050162226 = queryWeight, product of:
               1.3622969 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.015209672 = queryNorm
             0.15130915 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
         0.08169216 = weight(abstract_txt:reading in 1604) [ClassicSimilarity], result of:
           0.08169216 = score(doc=1604,freq=2.0), product of:
             0.15404552 = queryWeight, product of:
               1.6880796 = boost
               5.9997935 = idf(docFreq=297, maxDocs=44218)
               0.015209672 = queryNorm
             0.5303118 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9997935 = idf(docFreq=297, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
         0.080920935 = weight(abstract_txt:implicit in 1604) [ClassicSimilarity], result of:
           0.080920935 = score(doc=1604,freq=1.0), product of:
             0.19286174 = queryWeight, product of:
               1.8888236 = boost
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.015209672 = queryNorm
             0.41958004 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7132807 = idf(docFreq=145, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
         0.1512291 = weight(abstract_txt:interest in 1604) [ClassicSimilarity], result of:
           0.1512291 = score(doc=1604,freq=8.0), product of:
             0.16747995 = queryWeight, product of:
               2.1557353 = boost
               5.1079607 = idf(docFreq=726, maxDocs=44218)
               0.015209672 = queryNorm
             0.9029684 = fieldWeight in 1604, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.1079607 = idf(docFreq=726, maxDocs=44218)
               0.0625 = fieldNorm(doc=1604)
       0.28 = coord(7/25)