Document (#43560)

Author

Hasanain, M.
Elsayed, T.

Title

Studying effectiveness of Web search for fact checking

Source

Journal of the Association for Information Science and Technology. 73(2022) no.5, S.738-751

Year

2022

Abstract

Web search is commonly used by fact checking systems as a source of evidence for claim verification. In this work, we demonstrate that the task of retrieving pages useful for fact checking, called evidential pages, is indeed different from the task of retrieving topically relevant pages that are typically optimized by search engines; thus, it should be handled differently. We conduct a comprehensive study on the performance of retrieving evidential pages over a test collection we developed for the task of re-ranking Web pages by usefulness for fact-checking. Results show that pages (retrieved by a commercial search engine) that are topically relevant to a claim are not always useful for verifying it, and that the engine's performance in retrieving evidential pages is weakly correlated with retrieval of topically relevant pages. Additionally, we identify types of evidence in evidential pages and some linguistic cues that can help predict page usefulness. Moreover, preliminary experiments show that a retrieval model leveraging those cues has a higher performance compared to the search engine. Finally, we show that existing systems have a long way to go to support effective fact checking. To that end, our work provides insights to guide design of better future systems for the task.

Content

Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24577. https://doi.org/10.1002/asi.24577.

Theme

Internet

Field

Kommunikationswissenschaften

Similar documents (content)

1. Liu, Y.; Zhang, M.; Cen, R.; Ru, L.; Ma, S.: Data cleansing for Web information retrieval using query independent features (2007) 0.17

   0.1691991 = sum of:
     0.1691991 = product of:
       0.6042825 = sum of:
         0.02757715 = weight(abstract_txt:useful in 1607) [ClassicSimilarity], result of:
           0.02757715 = score(doc=1607,freq=2.0), product of:
             0.06450783 = queryWeight, product of:
               1.2217886 = boost
               4.83662 = idf(docFreq=957, maxDocs=44421)
               0.010916273 = queryNorm
             0.42750084 = fieldWeight in 1607, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.83662 = idf(docFreq=957, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
         0.040844694 = weight(abstract_txt:engine in 1607) [ClassicSimilarity], result of:
           0.040844694 = score(doc=1607,freq=2.0), product of:
             0.08381786 = queryWeight, product of:
               1.3927019 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.010916273 = queryNorm
             0.48730296 = fieldWeight in 1607, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
         0.025489293 = weight(abstract_txt:performance in 1607) [ClassicSimilarity], result of:
           0.025489293 = score(doc=1607,freq=1.0), product of:
             0.08827921 = queryWeight, product of:
               1.7505105 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.010916273 = queryNorm
             0.28873494 = fieldWeight in 1607, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
         0.04206272 = weight(abstract_txt:search in 1607) [ClassicSimilarity], result of:
           0.04206272 = score(doc=1607,freq=4.0), product of:
             0.09207627 = queryWeight, product of:
               2.307989 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010916273 = queryNorm
             0.45682475 = fieldWeight in 1607, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
         0.040696576 = weight(abstract_txt:task in 1607) [ClassicSimilarity], result of:
           0.040696576 = score(doc=1607,freq=1.0), product of:
             0.13273071 = queryWeight, product of:
               2.4785094 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.010916273 = queryNorm
             0.3066101 = fieldWeight in 1607, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
         0.017768031 = weight(abstract_txt:that in 1607) [ClassicSimilarity], result of:
           0.017768031 = score(doc=1607,freq=3.0), product of:
             0.069403194 = queryWeight, product of:
               2.6883502 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010916273 = queryNorm
             0.25601172 = fieldWeight in 1607, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
         0.40984407 = weight(abstract_txt:pages in 1607) [ClassicSimilarity], result of:
           0.40984407 = score(doc=1607,freq=9.0), product of:
             0.38993475 = queryWeight, product of:
               6.3722396 = boost
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.010916273 = queryNorm
             1.051058 = fieldWeight in 1607, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.0625 = fieldNorm(doc=1607)
       0.28 = coord(7/25)
   

2. Choi, B.; Peng, X.: Dynamic and hierarchical classification of Web pages (2004) 0.17

   0.16533956 = sum of:
     0.16533956 = product of:
       0.59049845 = sum of:
         0.022216849 = weight(abstract_txt:systems in 3555) [ClassicSimilarity], result of:
           0.022216849 = score(doc=3555,freq=3.0), product of:
             0.04813132 = queryWeight, product of:
               1.292556 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.010916273 = queryNorm
             0.46158817 = fieldWeight in 3555, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
         0.027546242 = weight(abstract_txt:show in 3555) [ClassicSimilarity], result of:
           0.027546242 = score(doc=3555,freq=1.0), product of:
             0.080116406 = queryWeight, product of:
               1.6676166 = boost
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.010916273 = queryNorm
             0.34382772 = fieldWeight in 3555, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
         0.032071777 = weight(abstract_txt:relevant in 3555) [ClassicSimilarity], result of:
           0.032071777 = score(doc=3555,freq=1.0), product of:
             0.088666975 = queryWeight, product of:
               1.7543508 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.010916273 = queryNorm
             0.3617105 = fieldWeight in 3555, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
         0.037178546 = weight(abstract_txt:search in 3555) [ClassicSimilarity], result of:
           0.037178546 = score(doc=3555,freq=2.0), product of:
             0.09207627 = queryWeight, product of:
               2.307989 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010916273 = queryNorm
             0.40377986 = fieldWeight in 3555, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
         0.018134424 = weight(abstract_txt:that in 3555) [ClassicSimilarity], result of:
           0.018134424 = score(doc=3555,freq=2.0), product of:
             0.069403194 = queryWeight, product of:
               2.6883502 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010916273 = queryNorm
             0.2612909 = fieldWeight in 3555, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
         0.111813836 = weight(abstract_txt:retrieving in 3555) [ClassicSimilarity], result of:
           0.111813836 = score(doc=3555,freq=1.0), product of:
             0.22438294 = queryWeight, product of:
               3.2225509 = boost
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.010916273 = queryNorm
             0.498317 = fieldWeight in 3555, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.3784575 = idf(docFreq=204, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
         0.34153676 = weight(abstract_txt:pages in 3555) [ClassicSimilarity], result of:
           0.34153676 = score(doc=3555,freq=4.0), product of:
             0.38993475 = queryWeight, product of:
               6.3722396 = boost
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.010916273 = queryNorm
             0.8758818 = fieldWeight in 3555, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.078125 = fieldNorm(doc=3555)
       0.28 = coord(7/25)
   

3. Juneström, A.: Discourses of fact-checking in Swedish news media (2022) 0.16

   0.15522924 = sum of:
     0.15522924 = product of:
       0.9701828 = sum of:
         0.02565742 = weight(abstract_txt:relevant in 1687) [ClassicSimilarity], result of:
           0.02565742 = score(doc=1687,freq=1.0), product of:
             0.088666975 = queryWeight, product of:
               1.7543508 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.010916273 = queryNorm
             0.2893684 = fieldWeight in 1687, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0625 = fieldNorm(doc=1687)
         0.020516757 = weight(abstract_txt:that in 1687) [ClassicSimilarity], result of:
           0.020516757 = score(doc=1687,freq=4.0), product of:
             0.069403194 = queryWeight, product of:
               2.6883502 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010916273 = queryNorm
             0.2956169 = fieldWeight in 1687, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=1687)
         0.2645667 = weight(abstract_txt:fact in 1687) [ClassicSimilarity], result of:
           0.2645667 = score(doc=1687,freq=10.0), product of:
             0.23116723 = queryWeight, product of:
               3.6569834 = boost
               5.790671 = idf(docFreq=368, maxDocs=44421)
               0.010916273 = queryNorm
             1.1444818 = fieldWeight in 1687, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               5.790671 = idf(docFreq=368, maxDocs=44421)
               0.0625 = fieldNorm(doc=1687)
         0.6594419 = weight(abstract_txt:checking in 1687) [ClassicSimilarity], result of:
           0.6594419 = score(doc=1687,freq=10.0), product of:
             0.42496565 = queryWeight, product of:
               4.9583454 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.010916273 = queryNorm
             1.5517534 = fieldWeight in 1687, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.0625 = fieldNorm(doc=1687)
       0.16 = coord(4/25)
   

4. Choi, Y.: Effects of contextual factors on image searching on the Web (2010) 0.14

   0.14415377 = sum of:
     0.14415377 = product of:
       0.6006407 = sum of:
         0.031861614 = weight(abstract_txt:performance in 982) [ClassicSimilarity], result of:
           0.031861614 = score(doc=982,freq=1.0), product of:
             0.08827921 = queryWeight, product of:
               1.7505105 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.010916273 = queryNorm
             0.36091867 = fieldWeight in 982, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.078125 = fieldNorm(doc=982)
         0.052578397 = weight(abstract_txt:search in 982) [ClassicSimilarity], result of:
           0.052578397 = score(doc=982,freq=4.0), product of:
             0.09207627 = queryWeight, product of:
               2.307989 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010916273 = queryNorm
             0.5710309 = fieldWeight in 982, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.078125 = fieldNorm(doc=982)
         0.07194206 = weight(abstract_txt:task in 982) [ClassicSimilarity], result of:
           0.07194206 = score(doc=982,freq=2.0), product of:
             0.13273071 = queryWeight, product of:
               2.4785094 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.010916273 = queryNorm
             0.5420152 = fieldWeight in 982, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.078125 = fieldNorm(doc=982)
         0.012822973 = weight(abstract_txt:that in 982) [ClassicSimilarity], result of:
           0.012822973 = score(doc=982,freq=1.0), product of:
             0.069403194 = queryWeight, product of:
               2.6883502 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010916273 = queryNorm
             0.18476056 = fieldWeight in 982, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=982)
         0.2606673 = weight(abstract_txt:checking in 982) [ClassicSimilarity], result of:
           0.2606673 = score(doc=982,freq=1.0), product of:
             0.42496565 = queryWeight, product of:
               4.9583454 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.010916273 = queryNorm
             0.61338437 = fieldWeight in 982, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.078125 = fieldNorm(doc=982)
         0.17076838 = weight(abstract_txt:pages in 982) [ClassicSimilarity], result of:
           0.17076838 = score(doc=982,freq=1.0), product of:
             0.38993475 = queryWeight, product of:
               6.3722396 = boost
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.010916273 = queryNorm
             0.4379409 = fieldWeight in 982, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.078125 = fieldNorm(doc=982)
       0.24 = coord(6/25)
   

5. Vaughan, L.: New measurements for search engine evaluation proposed and tested (2004) 0.14

   0.14246033 = sum of:
     0.14246033 = product of:
       0.5087869 = sum of:
         0.021767976 = weight(abstract_txt:systems in 3535) [ClassicSimilarity], result of:
           0.021767976 = score(doc=3535,freq=2.0), product of:
             0.04813132 = queryWeight, product of:
               1.292556 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.010916273 = queryNorm
             0.4522622 = fieldWeight in 3535, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.08664468 = weight(abstract_txt:engine in 3535) [ClassicSimilarity], result of:
           0.08664468 = score(doc=3535,freq=4.0), product of:
             0.08381786 = queryWeight, product of:
               1.3927019 = boost
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.010916273 = queryNorm
             1.0337257 = fieldWeight in 3535, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5132036 = idf(docFreq=486, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.033055488 = weight(abstract_txt:show in 3535) [ClassicSimilarity], result of:
           0.033055488 = score(doc=3535,freq=1.0), product of:
             0.080116406 = queryWeight, product of:
               1.6676166 = boost
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.010916273 = queryNorm
             0.41259325 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.07646788 = weight(abstract_txt:performance in 3535) [ClassicSimilarity], result of:
           0.07646788 = score(doc=3535,freq=4.0), product of:
             0.08827921 = queryWeight, product of:
               1.7505105 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.010916273 = queryNorm
             0.86620486 = fieldWeight in 3535, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.07054133 = weight(abstract_txt:search in 3535) [ClassicSimilarity], result of:
           0.07054133 = score(doc=3535,freq=5.0), product of:
             0.09207627 = queryWeight, product of:
               2.307989 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010916273 = queryNorm
             0.7661184 = fieldWeight in 3535, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.015387568 = weight(abstract_txt:that in 3535) [ClassicSimilarity], result of:
           0.015387568 = score(doc=3535,freq=1.0), product of:
             0.069403194 = queryWeight, product of:
               2.6883502 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.010916273 = queryNorm
             0.22171268 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
         0.20492204 = weight(abstract_txt:pages in 3535) [ClassicSimilarity], result of:
           0.20492204 = score(doc=3535,freq=1.0), product of:
             0.38993475 = queryWeight, product of:
               6.3722396 = boost
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.010916273 = queryNorm
             0.525529 = fieldWeight in 3535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6056433 = idf(docFreq=443, maxDocs=44421)
               0.09375 = fieldNorm(doc=3535)
       0.28 = coord(7/25)