Document (#44350)

Author

Liu, N.F.
Paranjape, A.
Bevilacqua, M.
Petroni, F.
Lin, K.
Hewitt, J.
Liang, P.

Title

Lost in the middle : how language models use long contexts

Source

Transactions of the Association for Compurational Linguistics. 12(2024) no.5, S.157-173

Year

2024

Abstract

While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.

Content

Vgl.: https://www.researchgate.net/publication/378284067_Lost_in_the_Middle_How_Language_Models_Use_Long_Contexts.

Theme

Computerlinguistik

Similar documents (author)

1. Liang, D.F.: Mathematical journals : an annotated guide (1992) 5.19

   5.187669 = sum of:
     5.187669 = weight(author_txt:liang in 247) [ClassicSimilarity], result of:
       5.187669 = fieldWeight in 247, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.30027 = idf(docFreq=29, maxDocs=44421)
         0.625 = fieldNorm(doc=247)
   

2. Liang, L.: R-Sequences : relative indicators for the rhythm of science (2005) 5.19

   5.187669 = sum of:
     5.187669 = weight(author_txt:liang in 4877) [ClassicSimilarity], result of:
       5.187669 = fieldWeight in 4877, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.30027 = idf(docFreq=29, maxDocs=44421)
         0.625 = fieldNorm(doc=4877)
   

3. Liang, T.-Y.: ¬The basic entity model : a fundamental theoretical model of information and information processing (1994) 4.15

   4.150135 = sum of:
     4.150135 = weight(author_txt:liang in 82) [ClassicSimilarity], result of:
       4.150135 = fieldWeight in 82, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.30027 = idf(docFreq=29, maxDocs=44421)
         0.5 = fieldNorm(doc=82)
   

4. Liang, T.-Y.: ¬The basic entity model : a theoretical model of information processing, decision making and information systems (1996) 4.15

   4.150135 = sum of:
     4.150135 = weight(author_txt:liang in 5476) [ClassicSimilarity], result of:
       4.150135 = fieldWeight in 5476, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.30027 = idf(docFreq=29, maxDocs=44421)
         0.5 = fieldNorm(doc=5476)
   

5. Liang, T.-Y.: ¬The basic entity model (1997) 4.15

   4.150135 = sum of:
     4.150135 = weight(author_txt:liang in 826) [ClassicSimilarity], result of:
       4.150135 = fieldWeight in 826, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.30027 = idf(docFreq=29, maxDocs=44421)
         0.5 = fieldNorm(doc=826)
   

Similar documents (content)

1. Wu, H.C.; Luk, R.W.P.; Wong, K.F,; Kwok, K.L.: ¬A retrospective study of a hybrid document-context based retrieval model (2007) 0.21

   0.2066769 = sum of:
     0.2066769 = product of:
       0.6458653 = sum of:
         0.039653484 = weight(abstract_txt:explicitly in 1936) [ClassicSimilarity], result of:
           0.039653484 = score(doc=1936,freq=1.0), product of:
             0.10474663 = queryWeight, product of:
               1.0819268 = boost
               6.922344 = idf(docFreq=118, maxDocs=44421)
               0.013985854 = queryNorm
             0.3785657 = fieldWeight in 1936, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.922344 = idf(docFreq=118, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.015492271 = weight(abstract_txt:that in 1936) [ClassicSimilarity], result of:
           0.015492271 = score(doc=1936,freq=6.0), product of:
             0.04890262 = queryWeight, product of:
               1.4785088 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013985854 = queryNorm
             0.3167984 = fieldWeight in 1936, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.011963555 = weight(abstract_txt:information in 1936) [ClassicSimilarity], result of:
           0.011963555 = score(doc=1936,freq=2.0), product of:
             0.06394982 = queryWeight, product of:
               1.8903079 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.013985854 = queryNorm
             0.18707721 = fieldWeight in 1936, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.06711351 = weight(abstract_txt:relevant in 1936) [ClassicSimilarity], result of:
           0.06711351 = score(doc=1936,freq=2.0), product of:
             0.1874286 = queryWeight, product of:
               2.8945148 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.013985854 = queryNorm
             0.35807508 = fieldWeight in 1936, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.04862911 = weight(abstract_txt:context in 1936) [ClassicSimilarity], result of:
           0.04862911 = score(doc=1936,freq=1.0), product of:
             0.20521387 = queryWeight, product of:
               3.3862283 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.013985854 = queryNorm
             0.23696794 = fieldWeight in 1936, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.05204628 = weight(abstract_txt:language in 1936) [ClassicSimilarity], result of:
           0.05204628 = score(doc=1936,freq=1.0), product of:
             0.2281722 = queryWeight, product of:
               3.9114234 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.013985854 = queryNorm
             0.22810088 = fieldWeight in 1936, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.2773279 = weight(abstract_txt:contexts in 1936) [ClassicSimilarity], result of:
           0.2773279 = score(doc=1936,freq=6.0), product of:
             0.3604855 = queryWeight, product of:
               4.4880404 = boost
               5.743043 = idf(docFreq=386, maxDocs=44421)
               0.013985854 = queryNorm
             0.76931775 = fieldWeight in 1936, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.743043 = idf(docFreq=386, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
         0.13363922 = weight(abstract_txt:models in 1936) [ClassicSimilarity], result of:
           0.13363922 = score(doc=1936,freq=2.0), product of:
             0.37376198 = queryWeight, product of:
               5.7805667 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013985854 = queryNorm
             0.35755166 = fieldWeight in 1936, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1936)
       0.32 = coord(8/25)
   

2. Yu, M.; Sun, A.: Dataset versus reality : understanding model performance from the perspective of information need (2023) 0.20

   0.20309275 = sum of:
     0.20309275 = product of:
       0.6346649 = sum of:
         0.049467474 = weight(abstract_txt:answering in 2075) [ClassicSimilarity], result of:
           0.049467474 = score(doc=2075,freq=2.0), product of:
             0.09634349 = queryWeight, product of:
               1.0376217 = boost
               6.6388726 = idf(docFreq=157, maxDocs=44421)
               0.013985854 = queryNorm
             0.5134491 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6388726 = idf(docFreq=157, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.008944466 = weight(abstract_txt:that in 2075) [ClassicSimilarity], result of:
           0.008944466 = score(doc=2075,freq=2.0), product of:
             0.04890262 = queryWeight, product of:
               1.4785088 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013985854 = queryNorm
             0.18290362 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.026751323 = weight(abstract_txt:information in 2075) [ClassicSimilarity], result of:
           0.026751323 = score(doc=2075,freq=10.0), product of:
             0.06394982 = queryWeight, product of:
               1.8903079 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.013985854 = queryNorm
             0.41831738 = fieldWeight in 2075, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.025559822 = weight(abstract_txt:when in 2075) [ClassicSimilarity], result of:
           0.025559822 = score(doc=2075,freq=1.0), product of:
             0.11272792 = queryWeight, product of:
               1.9440358 = boost
               4.1460857 = idf(docFreq=1910, maxDocs=44421)
               0.013985854 = queryNorm
             0.22673906 = fieldWeight in 2075, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1460857 = idf(docFreq=1910, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.08422808 = weight(abstract_txt:context in 2075) [ClassicSimilarity], result of:
           0.08422808 = score(doc=2075,freq=3.0), product of:
             0.20521387 = queryWeight, product of:
               3.3862283 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.013985854 = queryNorm
             0.4104405 = fieldWeight in 2075, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.11321864 = weight(abstract_txt:contexts in 2075) [ClassicSimilarity], result of:
           0.11321864 = score(doc=2075,freq=1.0), product of:
             0.3604855 = queryWeight, product of:
               4.4880404 = boost
               5.743043 = idf(docFreq=386, maxDocs=44421)
               0.013985854 = queryNorm
             0.31407267 = fieldWeight in 2075, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.743043 = idf(docFreq=386, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.1928559 = weight(abstract_txt:input in 2075) [ClassicSimilarity], result of:
           0.1928559 = score(doc=2075,freq=2.0), product of:
             0.40808862 = queryWeight, product of:
               4.775184 = boost
               6.110481 = idf(docFreq=267, maxDocs=44421)
               0.013985854 = queryNorm
             0.47258335 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.110481 = idf(docFreq=267, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.13363922 = weight(abstract_txt:models in 2075) [ClassicSimilarity], result of:
           0.13363922 = score(doc=2075,freq=2.0), product of:
             0.37376198 = queryWeight, product of:
               5.7805667 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013985854 = queryNorm
             0.35755166 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
       0.32 = coord(8/25)
   

3. Soricut, R.; Marcu, D.: Abstractive headline generation using WIDL-expressions (2007) 0.19

   0.19003984 = sum of:
     0.19003984 = product of:
       0.67871374 = sum of:
         0.009035275 = weight(abstract_txt:that in 1943) [ClassicSimilarity], result of:
           0.009035275 = score(doc=1943,freq=1.0), product of:
             0.04890262 = queryWeight, product of:
               1.4785088 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013985854 = queryNorm
             0.18476056 = fieldWeight in 1943, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
         0.012085016 = weight(abstract_txt:information in 1943) [ClassicSimilarity], result of:
           0.012085016 = score(doc=1943,freq=1.0), product of:
             0.06394982 = queryWeight, product of:
               1.8903079 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.013985854 = queryNorm
             0.18897653 = fieldWeight in 1943, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
         0.05051298 = weight(abstract_txt:performance in 1943) [ClassicSimilarity], result of:
           0.05051298 = score(doc=1943,freq=1.0), product of:
             0.13995668 = queryWeight, product of:
               2.1661339 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.013985854 = queryNorm
             0.36091867 = fieldWeight in 1943, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
         0.06779489 = weight(abstract_txt:relevant in 1943) [ClassicSimilarity], result of:
           0.06779489 = score(doc=1943,freq=1.0), product of:
             0.1874286 = queryWeight, product of:
               2.8945148 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.013985854 = queryNorm
             0.3617105 = fieldWeight in 1943, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
         0.12878114 = weight(abstract_txt:language in 1943) [ClassicSimilarity], result of:
           0.12878114 = score(doc=1943,freq=3.0), product of:
             0.2281722 = queryWeight, product of:
               3.9114234 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.013985854 = queryNorm
             0.5644033 = fieldWeight in 1943, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
         0.27550843 = weight(abstract_txt:input in 1943) [ClassicSimilarity], result of:
           0.27550843 = score(doc=1943,freq=2.0), product of:
             0.40808862 = queryWeight, product of:
               4.775184 = boost
               6.110481 = idf(docFreq=267, maxDocs=44421)
               0.013985854 = queryNorm
             0.6751191 = fieldWeight in 1943, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.110481 = idf(docFreq=267, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
         0.134996 = weight(abstract_txt:models in 1943) [ClassicSimilarity], result of:
           0.134996 = score(doc=1943,freq=1.0), product of:
             0.37376198 = queryWeight, product of:
               5.7805667 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013985854 = queryNorm
             0.36118174 = fieldWeight in 1943, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.078125 = fieldNorm(doc=1943)
       0.28 = coord(7/25)
   

4. Liu, X.; Zheng, W.; Fang, H.: ¬An exploration of ranking models and feedback method for related entity finding (2013) 0.18

   0.18188125 = sum of:
     0.18188125 = product of:
       0.6495759 = sum of:
         0.010222247 = weight(abstract_txt:that in 3714) [ClassicSimilarity], result of:
           0.010222247 = score(doc=3714,freq=2.0), product of:
             0.04890262 = queryWeight, product of:
               1.4785088 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013985854 = queryNorm
             0.20903271 = fieldWeight in 3714, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
         0.009668012 = weight(abstract_txt:information in 3714) [ClassicSimilarity], result of:
           0.009668012 = score(doc=3714,freq=1.0), product of:
             0.06394982 = queryWeight, product of:
               1.8903079 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.013985854 = queryNorm
             0.15118122 = fieldWeight in 3714, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
         0.040410385 = weight(abstract_txt:performance in 3714) [ClassicSimilarity], result of:
           0.040410385 = score(doc=3714,freq=1.0), product of:
             0.13995668 = queryWeight, product of:
               2.1661339 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.013985854 = queryNorm
             0.28873494 = fieldWeight in 3714, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
         0.07670116 = weight(abstract_txt:relevant in 3714) [ClassicSimilarity], result of:
           0.07670116 = score(doc=3714,freq=2.0), product of:
             0.1874286 = queryWeight, product of:
               2.8945148 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.013985854 = queryNorm
             0.40922868 = fieldWeight in 3714, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
         0.055576123 = weight(abstract_txt:context in 3714) [ClassicSimilarity], result of:
           0.055576123 = score(doc=3714,freq=1.0), product of:
             0.20521387 = queryWeight, product of:
               3.3862283 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.013985854 = queryNorm
             0.2708205 = fieldWeight in 3714, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
         0.26994202 = weight(abstract_txt:input in 3714) [ClassicSimilarity], result of:
           0.26994202 = score(doc=3714,freq=3.0), product of:
             0.40808862 = queryWeight, product of:
               4.775184 = boost
               6.110481 = idf(docFreq=267, maxDocs=44421)
               0.013985854 = queryNorm
             0.66147894 = fieldWeight in 3714, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.110481 = idf(docFreq=267, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
         0.18705595 = weight(abstract_txt:models in 3714) [ClassicSimilarity], result of:
           0.18705595 = score(doc=3714,freq=3.0), product of:
             0.37376198 = queryWeight, product of:
               5.7805667 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013985854 = queryNorm
             0.5004681 = fieldWeight in 3714, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=3714)
       0.28 = coord(7/25)
   

5. Varadarajan, U.; Dutta, B.: Models for narrative information : a study (2022) 0.17

   0.17083763 = sum of:
     0.17083763 = product of:
       0.61013436 = sum of:
         0.025513094 = weight(abstract_txt:provides in 2104) [ClassicSimilarity], result of:
           0.025513094 = score(doc=2104,freq=1.0), product of:
             0.07754193 = queryWeight, product of:
               1.3164706 = boost
               4.211497 = idf(docFreq=1789, maxDocs=44421)
               0.013985854 = queryNorm
             0.32902318 = fieldWeight in 2104, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.211497 = idf(docFreq=1789, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
         0.01277781 = weight(abstract_txt:that in 2104) [ClassicSimilarity], result of:
           0.01277781 = score(doc=2104,freq=2.0), product of:
             0.04890262 = queryWeight, product of:
               1.4785088 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.013985854 = queryNorm
             0.2612909 = fieldWeight in 2104, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
         0.05648076 = weight(abstract_txt:significantly in 2104) [ClassicSimilarity], result of:
           0.05648076 = score(doc=2104,freq=1.0), product of:
             0.13171288 = queryWeight, product of:
               1.7157618 = boost
               5.4888616 = idf(docFreq=498, maxDocs=44421)
               0.013985854 = queryNorm
             0.4288173 = fieldWeight in 2104, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4888616 = idf(docFreq=498, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
         0.02093186 = weight(abstract_txt:information in 2104) [ClassicSimilarity], result of:
           0.02093186 = score(doc=2104,freq=3.0), product of:
             0.06394982 = queryWeight, product of:
               1.8903079 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.013985854 = queryNorm
             0.32731694 = fieldWeight in 2104, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
         0.06779489 = weight(abstract_txt:relevant in 2104) [ClassicSimilarity], result of:
           0.06779489 = score(doc=2104,freq=1.0), product of:
             0.1874286 = queryWeight, product of:
               2.8945148 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.013985854 = queryNorm
             0.3617105 = fieldWeight in 2104, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
         0.06947015 = weight(abstract_txt:context in 2104) [ClassicSimilarity], result of:
           0.06947015 = score(doc=2104,freq=1.0), product of:
             0.20521387 = queryWeight, product of:
               3.3862283 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.013985854 = queryNorm
             0.33852562 = fieldWeight in 2104, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
         0.35716584 = weight(abstract_txt:models in 2104) [ClassicSimilarity], result of:
           0.35716584 = score(doc=2104,freq=7.0), product of:
             0.37376198 = queryWeight, product of:
               5.7805667 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013985854 = queryNorm
             0.95559704 = fieldWeight in 2104, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.078125 = fieldNorm(doc=2104)
       0.28 = coord(7/25)