Document (#44369)

Author

Hou, Y.
Pascale, A.
Carnerero-Cano, J.
Sattigeri, P.
Tchrakian, T.
Marinescu, R.
Daly, E.
Padhi, I.

Title

WikiContradict : a benchmark for evaluating LLMs on real-world knowledge conflicts from Wikipedia

Source

https://arxiv.org/abs/2406.13805 [DOI: 10.48550/arXiv.2406.13805]

Year

2024

Abstract

Retrieval-augmented generation (RAG) has emerged as a promising solution to mitigate the limitations of large language models (LLMs), such as hallucinations and outdated information. However, it remains unclear how LLMs handle knowledge conflicts arising from different augmented retrieved passages, especially when these passages originate from the same source and have equal trustworthiness. In this work, we conduct a comprehensive evaluation of LLM-generated answers to questions that have varying answers based on contradictory passages from Wikipedia, a dataset widely regarded as a high-quality pre-training resource for most LLMs. Specifically, we introduce WikiContradict, a benchmark consisting of 253 high-quality, human-annotated instances designed to assess LLM performance when augmented with retrieved passages containing real-world knowledge conflicts. We benchmark a diverse range of both closed and open-source LLMs under different QA scenarios, including RAG with a single passage, and RAG with 2 contradictory passages. Through rigorous human evaluations on a subset of WikiContradict instances involving 5 LLMs and over 3,500 judgements, we shed light on the behaviour and limitations of these models. For instance, when provided with two passages containing contradictory facts, all models struggle to generate answers that accurately reflect the conflicting nature of the context, especially for implicit conflicts requiring reasoning. Since human evaluation is costly, we also introduce an automated model that estimates LLM performance using a strong open-source language model, achieving an F-score of 0.8. Using this automated metric, we evaluate more than 1,500 answers from seven LLMs across all WikiContradict instances. To facilitate future work, we release WikiContradict on: https://ibm.biz/wikicontradict.

Content

Vgl.: https://www.researchgate.net/publication/381580571_WikiContradict_A_Benchmark_for_Evaluating_LLMs_on_Real-World_Knowledge_Conflicts_from_Wikipedia.

Theme

Computerlinguistik
Retrievalstudien

Object

Wikipedia

Similar documents (content)

1. Ghali, M.-K.; Farrag, A.; Won, D.; Jin, Y.: Enhancing knowledge retrieval with in-context learning and semantic search through Generative AI (2024) 0.31

   0.3109735 = sum of:
     0.3109735 = product of:
       0.77743375 = sum of:
         0.013893928 = weight(abstract_txt:open in 2367) [ClassicSimilarity], result of:
           0.013893928 = score(doc=2367,freq=1.0), product of:
             0.052720234 = queryWeight, product of:
               4.8190303 = idf(docFreq=974, maxDocs=44421)
               0.010940009 = queryNorm
             0.26354071 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8190303 = idf(docFreq=974, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.014166 = weight(abstract_txt:high in 2367) [ClassicSimilarity], result of:
           0.014166 = score(doc=2367,freq=1.0), product of:
             0.053406253 = queryWeight, product of:
               1.0064852 = boost
               4.8502827 = idf(docFreq=944, maxDocs=44421)
               0.010940009 = queryNorm
             0.26524985 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8502827 = idf(docFreq=944, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.0077234753 = weight(abstract_txt:with in 2367) [ClassicSimilarity], result of:
           0.0077234753 = score(doc=2367,freq=4.0), product of:
             0.028289534 = queryWeight, product of:
               1.035951 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.010940009 = queryNorm
             0.2730153 = fieldWeight in 2367, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.017862398 = weight(abstract_txt:limitations in 2367) [ClassicSimilarity], result of:
           0.017862398 = score(doc=2367,freq=1.0), product of:
             0.06233335 = queryWeight, product of:
               1.0873555 = boost
               5.2399993 = idf(docFreq=639, maxDocs=44421)
               0.010940009 = queryNorm
             0.28656247 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2399993 = idf(docFreq=639, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.008306112 = weight(abstract_txt:knowledge in 2367) [ClassicSimilarity], result of:
           0.008306112 = score(doc=2367,freq=1.0), product of:
             0.042827517 = queryWeight, product of:
               1.1038712 = boost
               3.5463927 = idf(docFreq=3480, maxDocs=44421)
               0.010940009 = queryNorm
             0.19394335 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.5463927 = idf(docFreq=3480, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.006521231 = weight(abstract_txt:from in 2367) [ClassicSimilarity], result of:
           0.006521231 = score(doc=2367,freq=1.0), product of:
             0.043214254 = queryWeight, product of:
               1.4315115 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.010940009 = queryNorm
             0.15090463 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.018401133 = weight(abstract_txt:models in 2367) [ClassicSimilarity], result of:
           0.018401133 = score(doc=2367,freq=1.0), product of:
             0.07278146 = queryWeight, product of:
               1.4390217 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.010940009 = queryNorm
             0.2528272 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.019104263 = weight(abstract_txt:human in 2367) [ClassicSimilarity], result of:
           0.019104263 = score(doc=2367,freq=1.0), product of:
             0.0746239 = queryWeight, product of:
               1.4571221 = boost
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.010940009 = queryNorm
             0.2560073 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.024647277 = weight(abstract_txt:source in 2367) [ClassicSimilarity], result of:
           0.024647277 = score(doc=2367,freq=1.0), product of:
             0.08843761 = queryWeight, product of:
               1.586264 = boost
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.010940009 = queryNorm
             0.27869678 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0961695 = idf(docFreq=738, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.6468079 = weight(abstract_txt:llms in 2367) [ClassicSimilarity], result of:
           0.6468079 = score(doc=2367,freq=4.0), product of:
             0.65254956 = queryWeight, product of:
               6.581913 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.010940009 = queryNorm
             0.99120116 = fieldWeight in 2367, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
       0.4 = coord(10/25)
   

2. Gao, T.; Yen, H.; Yu, J.; Chen, D.: Enabling large language models to generate text with citations (2023) 0.29

   0.29322013 = sum of:
     0.29322013 = product of:
       1.0472147 = sum of:
         0.0098686945 = weight(abstract_txt:with in 2295) [ClassicSimilarity], result of:
           0.0098686945 = score(doc=2295,freq=5.0), product of:
             0.028289534 = queryWeight, product of:
               1.035951 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.010940009 = queryNorm
             0.34884614 = fieldWeight in 2295, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
         0.0074528353 = weight(abstract_txt:from in 2295) [ClassicSimilarity], result of:
           0.0074528353 = score(doc=2295,freq=1.0), product of:
             0.043214254 = queryWeight, product of:
               1.4315115 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.010940009 = queryNorm
             0.17246243 = fieldWeight in 2295, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
         0.021029865 = weight(abstract_txt:models in 2295) [ClassicSimilarity], result of:
           0.021029865 = score(doc=2295,freq=1.0), product of:
             0.07278146 = queryWeight, product of:
               1.4390217 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.010940009 = queryNorm
             0.28894538 = fieldWeight in 2295, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
         0.030877154 = weight(abstract_txt:human in 2295) [ClassicSimilarity], result of:
           0.030877154 = score(doc=2295,freq=2.0), product of:
             0.0746239 = queryWeight, product of:
               1.4571221 = boost
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.010940009 = queryNorm
             0.41377032 = fieldWeight in 2295, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
         0.08179922 = weight(abstract_txt:benchmark in 2295) [ClassicSimilarity], result of:
           0.08179922 = score(doc=2295,freq=1.0), product of:
             0.18000966 = queryWeight, product of:
               2.2631059 = boost
               7.270651 = idf(docFreq=83, maxDocs=44421)
               0.010940009 = queryNorm
             0.45441568 = fieldWeight in 2295, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.270651 = idf(docFreq=83, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
         0.0697262 = weight(abstract_txt:answers in 2295) [ClassicSimilarity], result of:
           0.0697262 = score(doc=2295,freq=1.0), product of:
             0.17811751 = queryWeight, product of:
               2.5994391 = boost
               6.263388 = idf(docFreq=229, maxDocs=44421)
               0.010940009 = queryNorm
             0.39146176 = fieldWeight in 2295, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.263388 = idf(docFreq=229, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
         0.8264608 = weight(abstract_txt:llms in 2295) [ClassicSimilarity], result of:
           0.8264608 = score(doc=2295,freq=5.0), product of:
             0.65254956 = queryWeight, product of:
               6.581913 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.010940009 = queryNorm
             1.2665104 = fieldWeight in 2295, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0625 = fieldNorm(doc=2295)
       0.28 = coord(7/25)
   

3. Williams, S.; Huckle, J.: Easy problems that LLMs get wrong (2024) 0.26

   0.2587702 = sum of:
     0.2587702 = product of:
       0.92417926 = sum of:
         0.007801888 = weight(abstract_txt:with in 2394) [ClassicSimilarity], result of:
           0.007801888 = score(doc=2394,freq=2.0), product of:
             0.028289534 = queryWeight, product of:
               1.035951 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.010940009 = queryNorm
             0.2757871 = fieldWeight in 2394, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
         0.036087494 = weight(abstract_txt:limitations in 2394) [ClassicSimilarity], result of:
           0.036087494 = score(doc=2394,freq=2.0), product of:
             0.06233335 = queryWeight, product of:
               1.0873555 = boost
               5.2399993 = idf(docFreq=639, maxDocs=44421)
               0.010940009 = queryNorm
             0.5789436 = fieldWeight in 2394, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2399993 = idf(docFreq=639, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
         0.040538818 = weight(abstract_txt:introduce in 2394) [ClassicSimilarity], result of:
           0.040538818 = score(doc=2394,freq=1.0), product of:
             0.08486723 = queryWeight, product of:
               1.2687654 = boost
               6.114219 = idf(docFreq=266, maxDocs=44421)
               0.010940009 = queryNorm
             0.47767338 = fieldWeight in 2394, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.114219 = idf(docFreq=266, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
         0.045530993 = weight(abstract_txt:models in 2394) [ClassicSimilarity], result of:
           0.045530993 = score(doc=2394,freq=3.0), product of:
             0.07278146 = queryWeight, product of:
               1.4390217 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.010940009 = queryNorm
             0.6255851 = fieldWeight in 2394, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
         0.03859644 = weight(abstract_txt:human in 2394) [ClassicSimilarity], result of:
           0.03859644 = score(doc=2394,freq=2.0), product of:
             0.0746239 = queryWeight, product of:
               1.4571221 = boost
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.010940009 = queryNorm
             0.51721287 = fieldWeight in 2394, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
         0.10224902 = weight(abstract_txt:benchmark in 2394) [ClassicSimilarity], result of:
           0.10224902 = score(doc=2394,freq=1.0), product of:
             0.18000966 = queryWeight, product of:
               2.2631059 = boost
               7.270651 = idf(docFreq=83, maxDocs=44421)
               0.010940009 = queryNorm
             0.5680196 = fieldWeight in 2394, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.270651 = idf(docFreq=83, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
         0.6533746 = weight(abstract_txt:llms in 2394) [ClassicSimilarity], result of:
           0.6533746 = score(doc=2394,freq=2.0), product of:
             0.65254956 = queryWeight, product of:
               6.581913 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.010940009 = queryNorm
             1.0012643 = fieldWeight in 2394, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.078125 = fieldNorm(doc=2394)
       0.28 = coord(7/25)
   

4. Jha, A.: Why GPT-4 isn't all it's cracked up to be (2023) 0.20

   0.1993757 = sum of:
     0.1993757 = product of:
       0.7120561 = sum of:
         0.002758384 = weight(abstract_txt:with in 1924) [ClassicSimilarity], result of:
           0.002758384 = score(doc=1924,freq=1.0), product of:
             0.028289534 = queryWeight, product of:
               1.035951 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.010940009 = queryNorm
             0.09750546 = fieldWeight in 1924, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
         0.0059329374 = weight(abstract_txt:knowledge in 1924) [ClassicSimilarity], result of:
           0.0059329374 = score(doc=1924,freq=1.0), product of:
             0.042827517 = queryWeight, product of:
               1.1038712 = boost
               3.5463927 = idf(docFreq=3480, maxDocs=44421)
               0.010940009 = queryNorm
             0.13853097 = fieldWeight in 1924, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.5463927 = idf(docFreq=3480, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
         0.009480346 = weight(abstract_txt:when in 1924) [ClassicSimilarity], result of:
           0.009480346 = score(doc=1924,freq=1.0), product of:
             0.05853638 = queryWeight, product of:
               1.2905353 = boost
               4.1460857 = idf(docFreq=1910, maxDocs=44421)
               0.010940009 = queryNorm
             0.16195647 = fieldWeight in 1924, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1460857 = idf(docFreq=1910, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
         0.008067931 = weight(abstract_txt:from in 1924) [ClassicSimilarity], result of:
           0.008067931 = score(doc=1924,freq=3.0), product of:
             0.043214254 = queryWeight, product of:
               1.4315115 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.010940009 = queryNorm
             0.18669605 = fieldWeight in 1924, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
         0.013143667 = weight(abstract_txt:models in 1924) [ClassicSimilarity], result of:
           0.013143667 = score(doc=1924,freq=1.0), product of:
             0.07278146 = queryWeight, product of:
               1.4390217 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.010940009 = queryNorm
             0.18059087 = fieldWeight in 1924, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
         0.01929822 = weight(abstract_txt:human in 1924) [ClassicSimilarity], result of:
           0.01929822 = score(doc=1924,freq=2.0), product of:
             0.0746239 = queryWeight, product of:
               1.4571221 = boost
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.010940009 = queryNorm
             0.25860643 = fieldWeight in 1924, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.681277 = idf(docFreq=1118, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
         0.6533746 = weight(abstract_txt:llms in 1924) [ClassicSimilarity], result of:
           0.6533746 = score(doc=1924,freq=8.0), product of:
             0.65254956 = queryWeight, product of:
               6.581913 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.010940009 = queryNorm
             1.0012643 = fieldWeight in 1924, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0390625 = fieldNorm(doc=1924)
       0.28 = coord(7/25)
   

5. El Hamdani, R.; Bonald, T.; Malliaros, F.; Suchanek, F.; Holzenberger, N.: ¬The factuality of Large Language Models in the legal domain (2024) 0.18

   0.17500323 = sum of:
     0.17500323 = product of:
       0.72918016 = sum of:
         0.005516768 = weight(abstract_txt:with in 2383) [ClassicSimilarity], result of:
           0.005516768 = score(doc=2383,freq=1.0), product of:
             0.028289534 = queryWeight, product of:
               1.035951 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.010940009 = queryNorm
             0.19501092 = fieldWeight in 2383, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.078125 = fieldNorm(doc=2383)
         0.011865875 = weight(abstract_txt:knowledge in 2383) [ClassicSimilarity], result of:
           0.011865875 = score(doc=2383,freq=1.0), product of:
             0.042827517 = queryWeight, product of:
               1.1038712 = boost
               3.5463927 = idf(docFreq=3480, maxDocs=44421)
               0.010940009 = queryNorm
             0.27706194 = fieldWeight in 2383, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.5463927 = idf(docFreq=3480, maxDocs=44421)
               0.078125 = fieldNorm(doc=2383)
         0.018960692 = weight(abstract_txt:when in 2383) [ClassicSimilarity], result of:
           0.018960692 = score(doc=2383,freq=1.0), product of:
             0.05853638 = queryWeight, product of:
               1.2905353 = boost
               4.1460857 = idf(docFreq=1910, maxDocs=44421)
               0.010940009 = queryNorm
             0.32391295 = fieldWeight in 2383, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1460857 = idf(docFreq=1910, maxDocs=44421)
               0.078125 = fieldNorm(doc=2383)
         0.013174876 = weight(abstract_txt:from in 2383) [ClassicSimilarity], result of:
           0.013174876 = score(doc=2383,freq=2.0), product of:
             0.043214254 = queryWeight, product of:
               1.4315115 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.010940009 = queryNorm
             0.30487338 = fieldWeight in 2383, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.078125 = fieldNorm(doc=2383)
         0.026287334 = weight(abstract_txt:models in 2383) [ClassicSimilarity], result of:
           0.026287334 = score(doc=2383,freq=1.0), product of:
             0.07278146 = queryWeight, product of:
               1.4390217 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.010940009 = queryNorm
             0.36118174 = fieldWeight in 2383, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.078125 = fieldNorm(doc=2383)
         0.6533746 = weight(abstract_txt:llms in 2383) [ClassicSimilarity], result of:
           0.6533746 = score(doc=2383,freq=2.0), product of:
             0.65254956 = queryWeight, product of:
               6.581913 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.010940009 = queryNorm
             1.0012643 = fieldWeight in 2383, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.078125 = fieldNorm(doc=2383)
       0.24 = coord(6/25)