Document (#44337)

Author

Kau, A.
He, X.
Nambissan, A.
Astudillo, A.
Yin, H.
Aryani, A.

Title

Combining graphs and Large Language Models

Source

arXiv:2407.06564v1, [cs.CL] 9 Jul 2024

Year

2024

Abstract

In recent years, Natural Language Processing (NLP) has played a significant role in various Artificial Intelligence (AI) applications such as chatbots, text generation, and language translation. The emergence of large language models (LLMs) has greatly improved the performance of these applications, showing astonishing results in language understanding and generation. However, they still show some disadvantages, such as hallucinations and lack of domain-specific knowledge, that affect their performance in real-world tasks. These issues can be effectively mitigated by incorporating knowledge graphs (KGs), which organise information in structured formats that capture relationships between entities in a versatile and interpretable fashion. Likewise, the construction and validation of KGs present challenges that LLMs can help resolve. The complementary relationship between LLMs and KGs has led to a trend that combines these technologies to achieve trustworthy results. This work collected 28 papers outlining methods for KG-powered LLMs, LLM-based KGs, and LLM-KG hybrid approaches. We systematically analysed and compared these approaches to provide a comprehensive overview highlighting key trends, innovative techniques, and common challenges. This synthesis will benefit researchers new to the field and those seeking to deepen their understanding of how KGs and LLMs can be effectively combined to enhance AI applications capabilities.

Content

Vgl.: https://www.researchgate.net/publication/382111840_Combining_Knowledge_Graphs_and_Large_Language_Models.

Theme

Computerlinguistik
Wissensrepräsentation

Similar documents (content)

1. Khorashadizadeh, H.; Amara, F.Z.; Ezzabady, M.; Ieng, F.; Tiwari, S.; Mihindukulasooriya, N.; Groppe, J.; Sahri, S.; Benamara, F.; Groppe, S.: Research trends for the interplay between Large Language Models and Knowledge Graphs (2024) 0.54

   0.53631413 = sum of:
     0.53631413 = product of:
       1.4897615 = sum of:
         0.025958126 = weight(abstract_txt:large in 2335) [ClassicSimilarity], result of:
           0.025958126 = score(doc=2335,freq=1.0), product of:
             0.07479463 = queryWeight, product of:
               1.1601228 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.014512879 = queryNorm
             0.3470587 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.027327185 = weight(abstract_txt:understanding in 2335) [ClassicSimilarity], result of:
           0.027327185 = score(doc=2335,freq=1.0), product of:
             0.07740187 = queryWeight, product of:
               1.1801697 = boost
               4.5191154 = idf(docFreq=1315, maxDocs=44421)
               0.014512879 = queryNorm
             0.3530559 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5191154 = idf(docFreq=1315, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.029257813 = weight(abstract_txt:models in 2335) [ClassicSimilarity], result of:
           0.029257813 = score(doc=2335,freq=1.0), product of:
             0.0810058 = queryWeight, product of:
               1.2073321 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.014512879 = queryNorm
             0.36118174 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.007832896 = weight(abstract_txt:that in 2335) [ClassicSimilarity], result of:
           0.007832896 = score(doc=2335,freq=1.0), product of:
             0.04239485 = queryWeight, product of:
               1.2352085 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.014512879 = queryNorm
             0.18476056 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.052346278 = weight(abstract_txt:generation in 2335) [ClassicSimilarity], result of:
           0.052346278 = score(doc=2335,freq=1.0), product of:
             0.1193838 = queryWeight, product of:
               1.4656876 = boost
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.014512879 = queryNorm
             0.43847054 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.10492746 = weight(abstract_txt:graphs in 2335) [ClassicSimilarity], result of:
           0.10492746 = score(doc=2335,freq=1.0), product of:
             0.18979333 = queryWeight, product of:
               1.8480308 = boost
               7.0764947 = idf(docFreq=101, maxDocs=44421)
               0.014512879 = queryNorm
             0.55285114 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0764947 = idf(docFreq=101, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.047191903 = weight(abstract_txt:applications in 2335) [ClassicSimilarity], result of:
           0.047191903 = score(doc=2335,freq=1.0), product of:
             0.12753525 = queryWeight, product of:
               1.8553655 = boost
               4.7363873 = idf(docFreq=1058, maxDocs=44421)
               0.014512879 = queryNorm
             0.37003025 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7363873 = idf(docFreq=1058, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.0930362 = weight(abstract_txt:language in 2335) [ClassicSimilarity], result of:
           0.0930362 = score(doc=2335,freq=3.0), product of:
             0.16483992 = queryWeight, product of:
               2.7231398 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.014512879 = queryNorm
             0.5644033 = fieldWeight in 2335, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         1.1018836 = weight(abstract_txt:llms in 2335) [ClassicSimilarity], result of:
           1.1018836 = score(doc=2335,freq=4.0), product of:
             0.7781656 = queryWeight, product of:
               5.9166346 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.014512879 = queryNorm
             1.4160016 = fieldWeight in 2335, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
       0.36 = coord(9/25)
   

2. Yang, L.; Chen, H.; Li, Z.; Ding, X.; Wu, X.: Give us the facts : enhancing Large Language Models with knowledge graphs for fact-aware language modeling (2024) 0.39

   0.38650566 = sum of:
     0.38650566 = product of:
       1.2078302 = sum of:
         0.029368266 = weight(abstract_txt:large in 2337) [ClassicSimilarity], result of:
           0.029368266 = score(doc=2337,freq=2.0), product of:
             0.07479463 = queryWeight, product of:
               1.1601228 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.014512879 = queryNorm
             0.3926521 = fieldWeight in 2337, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.023355147 = weight(abstract_txt:performance in 2337) [ClassicSimilarity], result of:
           0.023355147 = score(doc=2337,freq=1.0), product of:
             0.08088785 = queryWeight, product of:
               1.2064528 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.014512879 = queryNorm
             0.28873494 = fieldWeight in 2337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.040540814 = weight(abstract_txt:models in 2337) [ClassicSimilarity], result of:
           0.040540814 = score(doc=2337,freq=3.0), product of:
             0.0810058 = queryWeight, product of:
               1.2073321 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.014512879 = queryNorm
             0.5004681 = fieldWeight in 2337, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.015276137 = weight(abstract_txt:these in 2337) [ClassicSimilarity], result of:
           0.015276137 = score(doc=2337,freq=1.0), product of:
             0.07679163 = queryWeight, product of:
               1.6624197 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.014512879 = queryNorm
             0.19892971 = fieldWeight in 2337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.08394197 = weight(abstract_txt:graphs in 2337) [ClassicSimilarity], result of:
           0.08394197 = score(doc=2337,freq=1.0), product of:
             0.18979333 = queryWeight, product of:
               1.8480308 = boost
               7.0764947 = idf(docFreq=101, maxDocs=44421)
               0.014512879 = queryNorm
             0.44228092 = fieldWeight in 2337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0764947 = idf(docFreq=101, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.037753522 = weight(abstract_txt:applications in 2337) [ClassicSimilarity], result of:
           0.037753522 = score(doc=2337,freq=1.0), product of:
             0.12753525 = queryWeight, product of:
               1.8553655 = boost
               4.7363873 = idf(docFreq=1058, maxDocs=44421)
               0.014512879 = queryNorm
             0.2960242 = fieldWeight in 2337, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7363873 = idf(docFreq=1058, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.096087374 = weight(abstract_txt:language in 2337) [ClassicSimilarity], result of:
           0.096087374 = score(doc=2337,freq=5.0), product of:
             0.16483992 = queryWeight, product of:
               2.7231398 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.014512879 = queryNorm
             0.5829132 = fieldWeight in 2337, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
         0.881507 = weight(abstract_txt:llms in 2337) [ClassicSimilarity], result of:
           0.881507 = score(doc=2337,freq=4.0), product of:
             0.7781656 = queryWeight, product of:
               5.9166346 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.014512879 = queryNorm
             1.1328013 = fieldWeight in 2337, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0625 = fieldNorm(doc=2337)
       0.32 = coord(8/25)
   

3. Luo, L.; Ju, J.; Li, Y.-F.; Haffari, G.; Xiong, B.; Pan, S.: ChatRule: mining logical rules with large language models for knowledge graph reasoning (2023) 0.38

   0.3808371 = sum of:
     0.3808371 = product of:
       1.190116 = sum of:
         0.041533 = weight(abstract_txt:large in 2173) [ClassicSimilarity], result of:
           0.041533 = score(doc=2173,freq=4.0), product of:
             0.07479463 = queryWeight, product of:
               1.1601228 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.014512879 = queryNorm
             0.5552939 = fieldWeight in 2173, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.08815071 = weight(abstract_txt:interpretable in 2173) [ClassicSimilarity], result of:
           0.08815071 = score(doc=2173,freq=1.0), product of:
             0.15563314 = queryWeight, product of:
               1.183327 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.014512879 = queryNorm
             0.56640065 = fieldWeight in 2173, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.033029165 = weight(abstract_txt:performance in 2173) [ClassicSimilarity], result of:
           0.033029165 = score(doc=2173,freq=2.0), product of:
             0.08088785 = queryWeight, product of:
               1.2064528 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.014512879 = queryNorm
             0.40833285 = fieldWeight in 2173, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.033101436 = weight(abstract_txt:models in 2173) [ClassicSimilarity], result of:
           0.033101436 = score(doc=2173,freq=2.0), product of:
             0.0810058 = queryWeight, product of:
               1.2073321 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.014512879 = queryNorm
             0.40863046 = fieldWeight in 2173, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.11871187 = weight(abstract_txt:graphs in 2173) [ClassicSimilarity], result of:
           0.11871187 = score(doc=2173,freq=2.0), product of:
             0.18979333 = queryWeight, product of:
               1.8480308 = boost
               7.0764947 = idf(docFreq=101, maxDocs=44421)
               0.014512879 = queryNorm
             0.62547964 = fieldWeight in 2173, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.0764947 = idf(docFreq=101, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.037753522 = weight(abstract_txt:applications in 2173) [ClassicSimilarity], result of:
           0.037753522 = score(doc=2173,freq=1.0), product of:
             0.12753525 = queryWeight, product of:
               1.8553655 = boost
               4.7363873 = idf(docFreq=1058, maxDocs=44421)
               0.014512879 = queryNorm
             0.2960242 = fieldWeight in 2173, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7363873 = idf(docFreq=1058, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.07442896 = weight(abstract_txt:language in 2173) [ClassicSimilarity], result of:
           0.07442896 = score(doc=2173,freq=3.0), product of:
             0.16483992 = queryWeight, product of:
               2.7231398 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.014512879 = queryNorm
             0.45152265 = fieldWeight in 2173, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
         0.7634074 = weight(abstract_txt:llms in 2173) [ClassicSimilarity], result of:
           0.7634074 = score(doc=2173,freq=3.0), product of:
             0.7781656 = queryWeight, product of:
               5.9166346 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.014512879 = queryNorm
             0.9810347 = fieldWeight in 2173, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0625 = fieldNorm(doc=2173)
       0.32 = coord(8/25)
   

4. Hou, Y.; Pascale, A.; Carnerero-Cano, J.; Sattigeri, P.; Tchrakian, T.; Marinescu, R.; Daly, E.; Padhi, I.: WikiContradict : a benchmark for evaluating LLMs on real-world knowledge conflicts from Wikipedia (2024) 0.37

   0.36653763 = sum of:
     0.36653763 = product of:
       1.1454301 = sum of:
         0.018170688 = weight(abstract_txt:large in 2368) [ClassicSimilarity], result of:
           0.018170688 = score(doc=2368,freq=1.0), product of:
             0.07479463 = queryWeight, product of:
               1.1601228 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.014512879 = queryNorm
             0.24294108 = fieldWeight in 2368, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.02890052 = weight(abstract_txt:performance in 2368) [ClassicSimilarity], result of:
           0.02890052 = score(doc=2368,freq=2.0), product of:
             0.08088785 = queryWeight, product of:
               1.2064528 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.014512879 = queryNorm
             0.35729125 = fieldWeight in 2368, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.035473213 = weight(abstract_txt:models in 2368) [ClassicSimilarity], result of:
           0.035473213 = score(doc=2368,freq=3.0), product of:
             0.0810058 = queryWeight, product of:
               1.2073321 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.014512879 = queryNorm
             0.43790957 = fieldWeight in 2368, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.009496882 = weight(abstract_txt:that in 2368) [ClassicSimilarity], result of:
           0.009496882 = score(doc=2368,freq=3.0), product of:
             0.04239485 = queryWeight, product of:
               1.2352085 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.014512879 = queryNorm
             0.22401026 = fieldWeight in 2368, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.036642395 = weight(abstract_txt:generation in 2368) [ClassicSimilarity], result of:
           0.036642395 = score(doc=2368,freq=1.0), product of:
             0.1193838 = queryWeight, product of:
               1.4656876 = boost
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.014512879 = queryNorm
             0.30692938 = fieldWeight in 2368, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.018903254 = weight(abstract_txt:these in 2368) [ClassicSimilarity], result of:
           0.018903254 = score(doc=2368,freq=2.0), product of:
             0.07679163 = queryWeight, product of:
               1.6624197 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.014512879 = queryNorm
             0.24616295 = fieldWeight in 2368, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.05317462 = weight(abstract_txt:language in 2368) [ClassicSimilarity], result of:
           0.05317462 = score(doc=2368,freq=2.0), product of:
             0.16483992 = queryWeight, product of:
               2.7231398 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.014512879 = queryNorm
             0.32258338 = fieldWeight in 2368, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
         0.94466853 = weight(abstract_txt:llms in 2368) [ClassicSimilarity], result of:
           0.94466853 = score(doc=2368,freq=6.0), product of:
             0.7781656 = queryWeight, product of:
               5.9166346 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.014512879 = queryNorm
             1.2139685 = fieldWeight in 2368, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2368)
       0.32 = coord(8/25)
   

5. Ghali, M.-K.; Farrag, A.; Won, D.; Jin, Y.: Enhancing knowledge retrieval with in-context learning and semantic search through Generative AI (2024) 0.30

   0.30377796 = sum of:
     0.30377796 = product of:
       0.9493062 = sum of:
         0.031472553 = weight(abstract_txt:large in 2367) [ClassicSimilarity], result of:
           0.031472553 = score(doc=2367,freq=3.0), product of:
             0.07479463 = queryWeight, product of:
               1.1601228 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.014512879 = queryNorm
             0.4207863 = fieldWeight in 2367, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.020435754 = weight(abstract_txt:performance in 2367) [ClassicSimilarity], result of:
           0.020435754 = score(doc=2367,freq=1.0), product of:
             0.08088785 = queryWeight, product of:
               1.2064528 = boost
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.014512879 = queryNorm
             0.25264308 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.619759 = idf(docFreq=1189, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.020480469 = weight(abstract_txt:models in 2367) [ClassicSimilarity], result of:
           0.020480469 = score(doc=2367,freq=1.0), product of:
             0.0810058 = queryWeight, product of:
               1.2073321 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.014512879 = queryNorm
             0.2528272 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.0054830275 = weight(abstract_txt:that in 2367) [ClassicSimilarity], result of:
           0.0054830275 = score(doc=2367,freq=1.0), product of:
             0.04239485 = queryWeight, product of:
               1.2352085 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.014512879 = queryNorm
             0.1293324 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.02803789 = weight(abstract_txt:challenges in 2367) [ClassicSimilarity], result of:
           0.02803789 = score(doc=2367,freq=1.0), product of:
             0.09987414 = queryWeight, product of:
               1.3405877 = boost
               5.1333895 = idf(docFreq=711, maxDocs=44421)
               0.014512879 = queryNorm
             0.28073224 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1333895 = idf(docFreq=711, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.018903254 = weight(abstract_txt:these in 2367) [ClassicSimilarity], result of:
           0.018903254 = score(doc=2367,freq=2.0), product of:
             0.07679163 = queryWeight, product of:
               1.6624197 = boost
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.014512879 = queryNorm
             0.24616295 = fieldWeight in 2367, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1828754 = idf(docFreq=5006, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.05317462 = weight(abstract_txt:language in 2367) [ClassicSimilarity], result of:
           0.05317462 = score(doc=2367,freq=2.0), product of:
             0.16483992 = queryWeight, product of:
               2.7231398 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.014512879 = queryNorm
             0.32258338 = fieldWeight in 2367, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.7713186 = weight(abstract_txt:llms in 2367) [ClassicSimilarity], result of:
           0.7713186 = score(doc=2367,freq=4.0), product of:
             0.7781656 = queryWeight, product of:
               5.9166346 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.014512879 = queryNorm
             0.99120116 = fieldWeight in 2367, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
       0.32 = coord(8/25)