Document (#42055)

Author
Rodrigo, A.
Peñas, A.
Miyao, Y.
Kando, N.
Title
Do systems pass university entrance exams?
Source
Information processing and management. 54(2018) no.4, S.564-575
Year
2018
Abstract
Reading Comprehension tests are commonly used to assess the degree to which people comprehend what they read. This is why we work with the hypothesis that it is reasonable to use these tests to assess the degree to which a machine "comprehends" what it is reading. In this work, we evaluate Question Answering systems using Reading Comprehension tests from exams to enter University. This article analyses the datasets generated, the kind of inferences required, the methodology followed in three evaluation campaigns, the approaches presented by participants and current results. Besides, we study the evolution of systems and the main lessons learned in this evaluation process. We also show how current technologies are unable to pass university-entrance exams. This is because these tests require a deep understanding of texts, as well as detecting the similar meaning of phrases with different words. Future directions focused on these ideas seem more promising than including a massive amount of data for training systems, what has allowed systems to obtain outstanding results in Reading Comprehension tests with more straightforward questions. We think this study helps to increase the knowledge about how to develop better Question Answering systems.
Content
Vgl.: https://doi.org/10.1016/j.ipm.2018.03.002.
Theme
Information
Field
Informatik

Similar documents (author)

  1. Kando, N.: Information concepts reexamined (1994) 6.10 
    6.0972233 = sum of:
  6.0972233 = weight(author_txt:kando in 2194) [ClassicSimilarity], result of:
    6.0972233 = fieldWeight in 2194, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.755557 = idf(docFreq=6, maxDocs=44421)
      0.625 = fieldNorm(doc=2194)
  2. Hu, X.; Kando, N.: Task complexity and difficulty in music information retrieval (2017) 4.88 
    4.8777785 = sum of:
  4.8777785 = weight(author_txt:kando in 4690) [ClassicSimilarity], result of:
    4.8777785 = fieldWeight in 4690, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.755557 = idf(docFreq=6, maxDocs=44421)
      0.5 = fieldNorm(doc=4690)
  3. Fujii, A.; Iwayama, M.; Kando, N.: Introduction to the special issue on patent processing (2007) 3.66 
    3.6583338 = sum of:
  3.6583338 = weight(author_txt:kando in 1929) [ClassicSimilarity], result of:
    3.6583338 = fieldWeight in 1929, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.755557 = idf(docFreq=6, maxDocs=44421)
      0.375 = fieldNorm(doc=1929)
  4. Gey, F.C.; Kando, N.; Peters, C.: Cross-Language Information Retrieval : the way ahead (2005) 3.66 
    3.6583338 = sum of:
  3.6583338 = weight(author_txt:kando in 2018) [ClassicSimilarity], result of:
    3.6583338 = fieldWeight in 2018, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.755557 = idf(docFreq=6, maxDocs=44421)
      0.375 = fieldNorm(doc=2018)
  5. Kuriyama, K.; Kando, N.; Nozue, T.; Eguchi, K.: Pooling for a large-scale test collection : an analysis of the search results from the First NTCIR Workshop (2002) 3.05 
    3.0486116 = sum of:
  3.0486116 = weight(author_txt:kando in 4830) [ClassicSimilarity], result of:
    3.0486116 = fieldWeight in 4830, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.755557 = idf(docFreq=6, maxDocs=44421)
      0.3125 = fieldNorm(doc=4830)

Similar documents (content)

  1. Wacholder, N.; Kelly, D.; Kantor, P.; Rittman, R.; Sun, Y.; Bai, B.; Small, S.; Yamrom, B.; Strzalkowski, T.: ¬A model for quantitative evaluation of an end-to-end question-answering system (2007) 0.15 
    0.15481378 = sum of:
  0.15481378 = product of:
    0.48379308 = sum of:
      0.008052255 = weight(abstract_txt:with in 1435) [ClassicSimilarity], result of:
        0.008052255 = score(doc=1435,freq=1.0), product of:
          0.041291304 = queryWeight, product of:
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.016542064 = queryNorm
          0.19501092 = fieldWeight in 1435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.018781686 = weight(abstract_txt:work in 1435) [ClassicSimilarity], result of:
        0.018781686 = score(doc=1435,freq=1.0), product of:
          0.06344142 = queryWeight, product of:
            1.0120722 = boost
            3.7894108 = idf(docFreq=2729, maxDocs=44421)
            0.016542064 = queryNorm
          0.29604772 = fieldWeight in 1435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.7894108 = idf(docFreq=2729, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.062054966 = weight(abstract_txt:evaluation in 1435) [ClassicSimilarity], result of:
        0.062054966 = score(doc=1435,freq=4.0), product of:
          0.08865724 = queryWeight, product of:
            1.1964159 = boost
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.016542064 = queryNorm
          0.69994247 = fieldWeight in 1435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.0687014 = weight(abstract_txt:question in 1435) [ClassicSimilarity], result of:
        0.0687014 = score(doc=1435,freq=2.0), product of:
          0.119541004 = queryWeight, product of:
            1.3892599 = boost
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.016542064 = queryNorm
          0.5747099 = fieldWeight in 1435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.07231874 = weight(abstract_txt:assess in 1435) [ClassicSimilarity], result of:
        0.07231874 = score(doc=1435,freq=1.0), product of:
          0.15585367 = queryWeight, product of:
            1.5862937 = boost
            5.9394164 = idf(docFreq=317, maxDocs=44421)
            0.016542064 = queryNorm
          0.4640169 = fieldWeight in 1435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.9394164 = idf(docFreq=317, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.14282937 = weight(abstract_txt:answering in 1435) [ClassicSimilarity], result of:
        0.14282937 = score(doc=1435,freq=2.0), product of:
          0.19472343 = queryWeight, product of:
            1.7731038 = boost
            6.6388726 = idf(docFreq=157, maxDocs=44421)
            0.016542064 = queryNorm
          0.7334987 = fieldWeight in 1435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            6.6388726 = idf(docFreq=157, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.028852591 = weight(abstract_txt:this in 1435) [ClassicSimilarity], result of:
        0.028852591 = score(doc=1435,freq=4.0), product of:
          0.07674088 = queryWeight, product of:
            1.9279647 = boost
            2.4062347 = idf(docFreq=10885, maxDocs=44421)
            0.016542064 = queryNorm
          0.37597418 = fieldWeight in 1435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            2.4062347 = idf(docFreq=10885, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
      0.082202084 = weight(abstract_txt:systems in 1435) [ClassicSimilarity], result of:
        0.082202084 = score(doc=1435,freq=4.0), product of:
          0.15422642 = queryWeight, product of:
            2.7331603 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.016542064 = queryNorm
          0.5329961 = fieldWeight in 1435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.078125 = fieldNorm(doc=1435)
    0.32 = coord(8/25)
  2. Parapar, J.; Losada, D.E.; Presedo-Quindimil, M.A.; Barreiro, A.: Using score distributions to compare statistical significance tests for information retrieval evaluation (2020) 0.14 
    0.14191711 = sum of:
  0.14191711 = product of:
    0.5913213 = sum of:
      0.021892425 = weight(abstract_txt:current in 506) [ClassicSimilarity], result of:
        0.021892425 = score(doc=506,freq=1.0), product of:
          0.08153657 = queryWeight, product of:
            1.1473641 = boost
            4.295972 = idf(docFreq=1644, maxDocs=44421)
            0.016542064 = queryNorm
          0.26849824 = fieldWeight in 506, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.295972 = idf(docFreq=1644, maxDocs=44421)
            0.0625 = fieldNorm(doc=506)
      0.03510359 = weight(abstract_txt:evaluation in 506) [ClassicSimilarity], result of:
        0.03510359 = score(doc=506,freq=2.0), product of:
          0.08865724 = queryWeight, product of:
            1.1964159 = boost
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.016542064 = queryNorm
          0.39594725 = fieldWeight in 506, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.0625 = fieldNorm(doc=506)
      0.05785499 = weight(abstract_txt:assess in 506) [ClassicSimilarity], result of:
        0.05785499 = score(doc=506,freq=1.0), product of:
          0.15585367 = queryWeight, product of:
            1.5862937 = boost
            5.9394164 = idf(docFreq=317, maxDocs=44421)
            0.016542064 = queryNorm
          0.37121353 = fieldWeight in 506, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.9394164 = idf(docFreq=317, maxDocs=44421)
            0.0625 = fieldNorm(doc=506)
      0.019989662 = weight(abstract_txt:this in 506) [ClassicSimilarity], result of:
        0.019989662 = score(doc=506,freq=3.0), product of:
          0.07674088 = queryWeight, product of:
            1.9279647 = boost
            2.4062347 = idf(docFreq=10885, maxDocs=44421)
            0.016542064 = queryNorm
          0.26048255 = fieldWeight in 506, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            2.4062347 = idf(docFreq=10885, maxDocs=44421)
            0.0625 = fieldNorm(doc=506)
      0.04650052 = weight(abstract_txt:systems in 506) [ClassicSimilarity], result of:
        0.04650052 = score(doc=506,freq=2.0), product of:
          0.15422642 = queryWeight, product of:
            2.7331603 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.016542064 = queryNorm
          0.30150813 = fieldWeight in 506, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.0625 = fieldNorm(doc=506)
      0.40998012 = weight(abstract_txt:tests in 506) [ClassicSimilarity], result of:
        0.40998012 = score(doc=506,freq=7.0), product of:
          0.40795508 = queryWeight, product of:
            4.057899 = boost
            6.0774503 = idf(docFreq=276, maxDocs=44421)
            0.016542064 = queryNorm
          1.0049639 = fieldWeight in 506, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            6.0774503 = idf(docFreq=276, maxDocs=44421)
            0.0625 = fieldNorm(doc=506)
    0.24 = coord(6/25)
  3. Wong, W.; Thangarajah, J.T.; Padgham, L.: Contextual question answering for the health domain (2012) 0.14 
    0.14177035 = sum of:
  0.14177035 = product of:
    0.5063227 = sum of:
      0.011157532 = weight(abstract_txt:with in 1499) [ClassicSimilarity], result of:
        0.011157532 = score(doc=1499,freq=3.0), product of:
          0.041291304 = queryWeight, product of:
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.016542064 = queryNorm
          0.27021506 = fieldWeight in 1499, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
      0.0731162 = weight(abstract_txt:unable in 1499) [ClassicSimilarity], result of:
        0.0731162 = score(doc=1499,freq=1.0), product of:
          0.14459576 = queryWeight, product of:
            1.0804081 = boost
            8.090549 = idf(docFreq=36, maxDocs=44421)
            0.016542064 = queryNorm
          0.50565934 = fieldWeight in 1499, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.090549 = idf(docFreq=36, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
      0.089572266 = weight(abstract_txt:comprehend in 1499) [ClassicSimilarity], result of:
        0.089572266 = score(doc=1499,freq=1.0), product of:
          0.16554987 = queryWeight, product of:
            1.1560444 = boost
            8.656945 = idf(docFreq=20, maxDocs=44421)
            0.016542064 = queryNorm
          0.5410591 = fieldWeight in 1499, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.656945 = idf(docFreq=20, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
      0.07772677 = weight(abstract_txt:question in 1499) [ClassicSimilarity], result of:
        0.07772677 = score(doc=1499,freq=4.0), product of:
          0.119541004 = queryWeight, product of:
            1.3892599 = boost
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.016542064 = queryNorm
          0.6502101 = fieldWeight in 1499, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
      0.05785499 = weight(abstract_txt:assess in 1499) [ClassicSimilarity], result of:
        0.05785499 = score(doc=1499,freq=1.0), product of:
          0.15585367 = queryWeight, product of:
            1.5862937 = boost
            5.9394164 = idf(docFreq=317, maxDocs=44421)
            0.016542064 = queryNorm
          0.37121353 = fieldWeight in 1499, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.9394164 = idf(docFreq=317, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
      0.13994364 = weight(abstract_txt:answering in 1499) [ClassicSimilarity], result of:
        0.13994364 = score(doc=1499,freq=3.0), product of:
          0.19472343 = queryWeight, product of:
            1.7731038 = boost
            6.6388726 = idf(docFreq=157, maxDocs=44421)
            0.016542064 = queryNorm
          0.718679 = fieldWeight in 1499, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.6388726 = idf(docFreq=157, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
      0.056951273 = weight(abstract_txt:systems in 1499) [ClassicSimilarity], result of:
        0.056951273 = score(doc=1499,freq=3.0), product of:
          0.15422642 = queryWeight, product of:
            2.7331603 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.016542064 = queryNorm
          0.36927053 = fieldWeight in 1499, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.0625 = fieldNorm(doc=1499)
    0.28 = coord(7/25)
  4. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amode, D.; Sutskever, I.: Language models are unsupervised multitask learners 0.14 
    0.13548446 = sum of:
  0.13548446 = product of:
    0.48387307 = sum of:
      0.006441804 = weight(abstract_txt:with in 1872) [ClassicSimilarity], result of:
        0.006441804 = score(doc=1872,freq=1.0), product of:
          0.041291304 = queryWeight, product of:
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.016542064 = queryNorm
          0.15600874 = fieldWeight in 1872, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
      0.023132378 = weight(abstract_txt:these in 1872) [ClassicSimilarity], result of:
        0.023132378 = score(doc=1872,freq=3.0), product of:
          0.067136705 = queryWeight, product of:
            1.275119 = boost
            3.1828754 = idf(docFreq=5006, maxDocs=44421)
            0.016542064 = queryNorm
          0.34455636 = fieldWeight in 1872, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.1828754 = idf(docFreq=5006, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
      0.038863383 = weight(abstract_txt:question in 1872) [ClassicSimilarity], result of:
        0.038863383 = score(doc=1872,freq=1.0), product of:
          0.119541004 = queryWeight, product of:
            1.3892599 = boost
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.016542064 = queryNorm
          0.32510504 = fieldWeight in 1872, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
      0.0807965 = weight(abstract_txt:answering in 1872) [ClassicSimilarity], result of:
        0.0807965 = score(doc=1872,freq=1.0), product of:
          0.19472343 = queryWeight, product of:
            1.7731038 = boost
            6.6388726 = idf(docFreq=157, maxDocs=44421)
            0.016542064 = queryNorm
          0.41492954 = fieldWeight in 1872, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.6388726 = idf(docFreq=157, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
      0.04650052 = weight(abstract_txt:systems in 1872) [ClassicSimilarity], result of:
        0.04650052 = score(doc=1872,freq=2.0), product of:
          0.15422642 = queryWeight, product of:
            2.7331603 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.016542064 = queryNorm
          0.30150813 = fieldWeight in 1872, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
      0.16859035 = weight(abstract_txt:comprehension in 1872) [ClassicSimilarity], result of:
        0.16859035 = score(doc=1872,freq=1.0), product of:
          0.3639782 = queryWeight, product of:
            2.9689875 = boost
            7.4110084 = idf(docFreq=72, maxDocs=44421)
            0.016542064 = queryNorm
          0.46318802 = fieldWeight in 1872, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.4110084 = idf(docFreq=72, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
      0.11954813 = weight(abstract_txt:reading in 1872) [ClassicSimilarity], result of:
        0.11954813 = score(doc=1872,freq=1.0), product of:
          0.31856278 = queryWeight, product of:
            3.2072852 = boost
            6.004374 = idf(docFreq=297, maxDocs=44421)
            0.016542064 = queryNorm
          0.37527338 = fieldWeight in 1872, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.004374 = idf(docFreq=297, maxDocs=44421)
            0.0625 = fieldNorm(doc=1872)
    0.28 = coord(7/25)
  5. Chen, A.T.: Information use and illness representations : understanding their connection in illness coping (2015) 0.14 
    0.13503353 = sum of:
  0.13503353 = product of:
    0.4822626 = sum of:
      0.009110087 = weight(abstract_txt:with in 2640) [ClassicSimilarity], result of:
        0.009110087 = score(doc=2640,freq=2.0), product of:
          0.041291304 = queryWeight, product of:
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.016542064 = queryNorm
          0.22062966 = fieldWeight in 2640, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
      0.013355485 = weight(abstract_txt:these in 2640) [ClassicSimilarity], result of:
        0.013355485 = score(doc=2640,freq=1.0), product of:
          0.067136705 = queryWeight, product of:
            1.275119 = boost
            3.1828754 = idf(docFreq=5006, maxDocs=44421)
            0.016542064 = queryNorm
          0.19892971 = fieldWeight in 2640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.1828754 = idf(docFreq=5006, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
      0.038863383 = weight(abstract_txt:question in 2640) [ClassicSimilarity], result of:
        0.038863383 = score(doc=2640,freq=1.0), product of:
          0.119541004 = queryWeight, product of:
            1.3892599 = boost
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.016542064 = queryNorm
          0.32510504 = fieldWeight in 2640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.2016807 = idf(docFreq=664, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
      0.033105772 = weight(abstract_txt:what in 2640) [ClassicSimilarity], result of:
        0.033105772 = score(doc=2640,freq=1.0), product of:
          0.122967236 = queryWeight, product of:
            1.7257004 = boost
            4.3075895 = idf(docFreq=1625, maxDocs=44421)
            0.016542064 = queryNorm
          0.26922435 = fieldWeight in 2640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.3075895 = idf(docFreq=1625, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
      0.025806542 = weight(abstract_txt:this in 2640) [ClassicSimilarity], result of:
        0.025806542 = score(doc=2640,freq=5.0), product of:
          0.07674088 = queryWeight, product of:
            1.9279647 = boost
            2.4062347 = idf(docFreq=10885, maxDocs=44421)
            0.016542064 = queryNorm
          0.33628154 = fieldWeight in 2640, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            2.4062347 = idf(docFreq=10885, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
      0.20706342 = weight(abstract_txt:reading in 2640) [ClassicSimilarity], result of:
        0.20706342 = score(doc=2640,freq=3.0), product of:
          0.31856278 = queryWeight, product of:
            3.2072852 = boost
            6.004374 = idf(docFreq=297, maxDocs=44421)
            0.016542064 = queryNorm
          0.6499925 = fieldWeight in 2640, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.004374 = idf(docFreq=297, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
      0.15495792 = weight(abstract_txt:tests in 2640) [ClassicSimilarity], result of:
        0.15495792 = score(doc=2640,freq=1.0), product of:
          0.40795508 = queryWeight, product of:
            4.057899 = boost
            6.0774503 = idf(docFreq=276, maxDocs=44421)
            0.016542064 = queryNorm
          0.37984064 = fieldWeight in 2640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.0774503 = idf(docFreq=276, maxDocs=44421)
            0.0625 = fieldNorm(doc=2640)
    0.28 = coord(7/25)