Document (#44296)

Author
Gao, T.
Yen, H.
Yu, J.
Chen, D.
Title
Enabling large language models to generate text with citations
Source
https://arxiv.org/abs/2305.14627
Year
2023
Abstract
Large language models (LLMs) have emerged as a widely-used tool for information seeking, but their generated outputs are prone to hallucination. In this work, we aim to enable LLMs to generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search engines and human evaluation, making it challenging to reproduce and compare with different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs' Citation Evaluation. ALCE collects a diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve supporting evidence and generate answers with citations. We build automatic metrics along three dimensions -- fluency, correctness, and citation quality -- and demonstrate their strong correlation with human judgements. Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvements -- for example, on the ELI5 dataset, even the best model has 49% of its generations lacking complete citation support. Our extensive analyses further highlight promising future directions, including developing better retrievers, advancing long-context LLMs, and improving the ability to synthesize information from multiple sources.
Content
Vgl. https://www.researchgate.net/publication/371009581_Enabling_Large_Language_Models_to_Generate_Text_with_Citations.

Similar documents (author)

  1. Chen, Y.N.; Chen, S.J.: ¬A metadata practice of the OFLA FRBR model : a case study for the National Palace Museum in Taipai (2004) 4.34 
    4.3394766 = sum of:
  4.3394766 = weight(author_txt:chen in 4384) [ClassicSimilarity], result of:
    4.3394766 = score(doc=4384,freq=2.0), product of:
      0.99999994 = queryWeight, product of:
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.16294746 = queryNorm
      4.339477 = fieldWeight in 4384, product of:
        1.4142135 = tf(freq=2.0), with freq of:
          2.0 = termFreq=2.0
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.5 = fieldNorm(doc=4384)
  2. Chen, C.C.; Chen, H.H.; Chen, K.H.: ¬The design of the XML/Metadata management system (2000) 3.99 
    3.9860637 = sum of:
  3.9860637 = weight(author_txt:chen in 5633) [ClassicSimilarity], result of:
    3.9860637 = score(doc=5633,freq=3.0), product of:
      0.99999994 = queryWeight, product of:
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.16294746 = queryNorm
      3.986064 = fieldWeight in 5633, product of:
        1.7320508 = tf(freq=3.0), with freq of:
          3.0 = termFreq=3.0
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.375 = fieldNorm(doc=5633)
  3. Chen, W.Y.: Observations on cataloguing and classification (1991) 3.84 
    3.8355918 = sum of:
  3.8355918 = weight(author_txt:chen in 4183) [ClassicSimilarity], result of:
    3.8355918 = score(doc=4183,freq=1.0), product of:
      0.99999994 = queryWeight, product of:
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.16294746 = queryNorm
      3.835592 = fieldWeight in 4183, product of:
        1.0 = tf(freq=1.0), with freq of:
          1.0 = termFreq=1.0
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.625 = fieldNorm(doc=4183)
  4. Chen, H.: Knowledge-based document retrieval : framework and design (1992) 3.84 
    3.8355918 = sum of:
  3.8355918 = weight(author_txt:chen in 5282) [ClassicSimilarity], result of:
    3.8355918 = score(doc=5282,freq=1.0), product of:
      0.99999994 = queryWeight, product of:
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.16294746 = queryNorm
      3.835592 = fieldWeight in 5282, product of:
        1.0 = tf(freq=1.0), with freq of:
          1.0 = termFreq=1.0
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.625 = fieldNorm(doc=5282)
  5. Chen, P.S.: On inference rules of logic-based information retrieval systems (1994) 3.84 
    3.8355918 = sum of:
  3.8355918 = weight(author_txt:chen in 6730) [ClassicSimilarity], result of:
    3.8355918 = score(doc=6730,freq=1.0), product of:
      0.99999994 = queryWeight, product of:
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.16294746 = queryNorm
      3.835592 = fieldWeight in 6730, product of:
        1.0 = tf(freq=1.0), with freq of:
          1.0 = termFreq=1.0
        6.136947 = idf(docFreq=260, maxDocs=44421)
        0.625 = fieldNorm(doc=6730)

Similar documents (content)

  1. Khorashadizadeh, H.; Amara, F.Z.; Ezzabady, M.; Ieng, F.; Tiwari, S.; Mihindukulasooriya, N.; Groppe, J.; Sahri, S.; Benamara, F.; Groppe, S.: Research trends for the interplay between Large Language Models and Knowledge Graphs (2024) 0.38 
    0.3752194 = sum of:
  0.3752194 = product of:
    1.3400693 = sum of:
      0.08114478 = weight(abstract_txt:advancing in 2335) [ClassicSimilarity], result of:
        0.08114478 = score(doc=2335,freq=1.0), product of:
          0.12658845 = queryWeight, product of:
            1.0785891 = boost
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.014304135 = queryNorm
          0.6410125 = fieldWeight in 2335, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
      0.03692653 = weight(abstract_txt:language in 2335) [ClassicSimilarity], result of:
        0.03692653 = score(doc=2335,freq=3.0), product of:
          0.06542579 = queryWeight, product of:
            1.0966005 = boost
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.014304135 = queryNorm
          0.5644033 = fieldWeight in 2335, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
      0.025757274 = weight(abstract_txt:large in 2335) [ClassicSimilarity], result of:
        0.025757274 = score(doc=2335,freq=1.0), product of:
          0.0742159 = queryWeight, product of:
            1.1679451 = boost
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.014304135 = queryNorm
          0.3470587 = fieldWeight in 2335, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
      0.02903143 = weight(abstract_txt:models in 2335) [ClassicSimilarity], result of:
        0.02903143 = score(doc=2335,freq=1.0), product of:
          0.08037901 = queryWeight, product of:
            1.2154729 = boost
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.014304135 = queryNorm
          0.36118174 = fieldWeight in 2335, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
      0.013806119 = weight(abstract_txt:their in 2335) [ClassicSimilarity], result of:
        0.013806119 = score(doc=2335,freq=1.0), product of:
          0.05605855 = queryWeight, product of:
            1.2431986 = boost
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.014304135 = queryNorm
          0.24628034 = fieldWeight in 2335, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
      0.060045335 = weight(abstract_txt:improving in 2335) [ClassicSimilarity], result of:
        0.060045335 = score(doc=2335,freq=1.0), product of:
          0.13048178 = queryWeight, product of:
            1.5486345 = boost
            5.8903265 = idf(docFreq=333, maxDocs=44421)
            0.014304135 = queryNorm
          0.46018177 = fieldWeight in 2335, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.8903265 = idf(docFreq=333, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
      1.0933578 = weight(abstract_txt:llms in 2335) [ClassicSimilarity], result of:
        1.0933578 = score(doc=2335,freq=4.0), product of:
          0.7721445 = queryWeight, product of:
            5.956529 = boost
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.014304135 = queryNorm
          1.4160016 = fieldWeight in 2335, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.078125 = fieldNorm(doc=2335)
    0.28 = coord(7/25)
  2. Hou, Y.; Pascale, A.; Carnerero-Cano, J.; Sattigeri, P.; Tchrakian, T.; Marinescu, R.; Daly, E.; Padhi, I.: WikiContradict : a benchmark for evaluating LLMs on real-world knowledge conflicts from Wikipedia (2024) 0.37 
    0.3715686 = sum of:
  0.3715686 = product of:
    1.1611519 = sum of:
      0.021105273 = weight(abstract_txt:language in 2368) [ClassicSimilarity], result of:
        0.021105273 = score(doc=2368,freq=2.0), product of:
          0.06542579 = queryWeight, product of:
            1.0966005 = boost
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.014304135 = queryNorm
          0.32258338 = fieldWeight in 2368, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.01803009 = weight(abstract_txt:large in 2368) [ClassicSimilarity], result of:
        0.01803009 = score(doc=2368,freq=1.0), product of:
          0.0742159 = queryWeight, product of:
            1.1679451 = boost
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.014304135 = queryNorm
          0.24294108 = fieldWeight in 2368, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.026145754 = weight(abstract_txt:evaluation in 2368) [ClassicSimilarity], result of:
        0.026145754 = score(doc=2368,freq=2.0), product of:
          0.075466774 = queryWeight, product of:
            1.1777467 = boost
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.014304135 = queryNorm
          0.34645385 = fieldWeight in 2368, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.035198737 = weight(abstract_txt:models in 2368) [ClassicSimilarity], result of:
        0.035198737 = score(doc=2368,freq=3.0), product of:
          0.08037901 = queryWeight, product of:
            1.2154729 = boost
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.014304135 = queryNorm
          0.43790957 = fieldWeight in 2368, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.036543723 = weight(abstract_txt:human in 2368) [ClassicSimilarity], result of:
        0.036543723 = score(doc=2368,freq=3.0), product of:
          0.08241377 = queryWeight, product of:
            1.2307614 = boost
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.014304135 = queryNorm
          0.44341767 = fieldWeight in 2368, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.019191865 = weight(abstract_txt:with in 2368) [ClassicSimilarity], result of:
        0.019191865 = score(doc=2368,freq=4.0), product of:
          0.07029594 = queryWeight, product of:
            1.9687915 = boost
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.014304135 = queryNorm
          0.2730153 = fieldWeight in 2368, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.06757739 = weight(abstract_txt:generate in 2368) [ClassicSimilarity], result of:
        0.06757739 = score(doc=2368,freq=1.0), product of:
          0.20498861 = queryWeight, product of:
            2.3773026 = boost
            6.0281444 = idf(docFreq=290, maxDocs=44421)
            0.014304135 = queryNorm
          0.32966414 = fieldWeight in 2368, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.0281444 = idf(docFreq=290, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
      0.9373591 = weight(abstract_txt:llms in 2368) [ClassicSimilarity], result of:
        0.9373591 = score(doc=2368,freq=6.0), product of:
          0.7721445 = queryWeight, product of:
            5.956529 = boost
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.014304135 = queryNorm
          1.2139685 = fieldWeight in 2368, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2368)
    0.32 = coord(8/25)
  3. Luo, L.; Ju, J.; Li, Y.-F.; Haffari, G.; Xiong, B.; Pan, S.: ChatRule: mining logical rules with large language models for knowledge graph reasoning (2023) 0.35 
    0.34529868 = sum of:
  0.34529868 = product of:
    1.0790584 = sum of:
      0.029541226 = weight(abstract_txt:language in 2173) [ClassicSimilarity], result of:
        0.029541226 = score(doc=2173,freq=3.0), product of:
          0.06542579 = queryWeight, product of:
            1.0966005 = boost
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.014304135 = queryNorm
          0.45152265 = fieldWeight in 2173, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.041211635 = weight(abstract_txt:large in 2173) [ClassicSimilarity], result of:
        0.041211635 = score(doc=2173,freq=4.0), product of:
          0.0742159 = queryWeight, product of:
            1.1679451 = boost
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.014304135 = queryNorm
          0.5552939 = fieldWeight in 2173, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.03284531 = weight(abstract_txt:models in 2173) [ClassicSimilarity], result of:
        0.03284531 = score(doc=2173,freq=2.0), product of:
          0.08037901 = queryWeight, product of:
            1.2154729 = boost
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.014304135 = queryNorm
          0.40863046 = fieldWeight in 2173, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.011044894 = weight(abstract_txt:their in 2173) [ClassicSimilarity], result of:
        0.011044894 = score(doc=2173,freq=1.0), product of:
          0.05605855 = queryWeight, product of:
            1.2431986 = boost
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.014304135 = queryNorm
          0.19702427 = fieldWeight in 2173, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.010966781 = weight(abstract_txt:with in 2173) [ClassicSimilarity], result of:
        0.010966781 = score(doc=2173,freq=1.0), product of:
          0.07029594 = queryWeight, product of:
            1.9687915 = boost
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.014304135 = queryNorm
          0.15600874 = fieldWeight in 2173, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.118716724 = weight(abstract_txt:correctness in 2173) [ClassicSimilarity], result of:
        0.118716724 = score(doc=2173,freq=1.0), product of:
          0.23851278 = queryWeight, product of:
            2.093774 = boost
            7.963798 = idf(docFreq=41, maxDocs=44421)
            0.014304135 = queryNorm
          0.49773738 = fieldWeight in 2173, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.963798 = idf(docFreq=41, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.07723131 = weight(abstract_txt:generate in 2173) [ClassicSimilarity], result of:
        0.07723131 = score(doc=2173,freq=1.0), product of:
          0.20498861 = queryWeight, product of:
            2.3773026 = boost
            6.0281444 = idf(docFreq=290, maxDocs=44421)
            0.014304135 = queryNorm
          0.37675902 = fieldWeight in 2173, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.0281444 = idf(docFreq=290, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
      0.7575005 = weight(abstract_txt:llms in 2173) [ClassicSimilarity], result of:
        0.7575005 = score(doc=2173,freq=3.0), product of:
          0.7721445 = queryWeight, product of:
            5.956529 = boost
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.014304135 = queryNorm
          0.9810347 = fieldWeight in 2173, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.0625 = fieldNorm(doc=2173)
    0.32 = coord(8/25)
  4. Ghali, M.-K.; Farrag, A.; Won, D.; Jin, Y.: Enhancing knowledge retrieval with in-context learning and semantic search through Generative AI (2024) 0.34 
    0.34325916 = sum of:
  0.34325916 = product of:
    0.9534976 = sum of:
      0.023504358 = weight(abstract_txt:text in 2367) [ClassicSimilarity], result of:
        0.023504358 = score(doc=2367,freq=3.0), product of:
          0.061407797 = queryWeight, product of:
            1.0623944 = boost
            4.040882 = idf(docFreq=2122, maxDocs=44421)
            0.014304135 = queryNorm
          0.38275853 = fieldWeight in 2367, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.040882 = idf(docFreq=2122, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.021105273 = weight(abstract_txt:language in 2367) [ClassicSimilarity], result of:
        0.021105273 = score(doc=2367,freq=2.0), product of:
          0.06542579 = queryWeight, product of:
            1.0966005 = boost
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.014304135 = queryNorm
          0.32258338 = fieldWeight in 2367, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.031229032 = weight(abstract_txt:large in 2367) [ClassicSimilarity], result of:
        0.031229032 = score(doc=2367,freq=3.0), product of:
          0.0742159 = queryWeight, product of:
            1.1679451 = boost
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.014304135 = queryNorm
          0.4207863 = fieldWeight in 2367, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.020321999 = weight(abstract_txt:models in 2367) [ClassicSimilarity], result of:
        0.020321999 = score(doc=2367,freq=1.0), product of:
          0.08037901 = queryWeight, product of:
            1.2154729 = boost
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.014304135 = queryNorm
          0.2528272 = fieldWeight in 2367, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.021098528 = weight(abstract_txt:human in 2367) [ClassicSimilarity], result of:
        0.021098528 = score(doc=2367,freq=1.0), product of:
          0.08241377 = queryWeight, product of:
            1.2307614 = boost
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.014304135 = queryNorm
          0.2560073 = fieldWeight in 2367, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.009664283 = weight(abstract_txt:their in 2367) [ClassicSimilarity], result of:
        0.009664283 = score(doc=2367,freq=1.0), product of:
          0.05605855 = queryWeight, product of:
            1.2431986 = boost
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.014304135 = queryNorm
          0.17239624 = fieldWeight in 2367, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.04203173 = weight(abstract_txt:improving in 2367) [ClassicSimilarity], result of:
        0.04203173 = score(doc=2367,freq=1.0), product of:
          0.13048178 = queryWeight, product of:
            1.5486345 = boost
            5.8903265 = idf(docFreq=333, maxDocs=44421)
            0.014304135 = queryNorm
          0.32212722 = fieldWeight in 2367, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.8903265 = idf(docFreq=333, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.019191865 = weight(abstract_txt:with in 2367) [ClassicSimilarity], result of:
        0.019191865 = score(doc=2367,freq=4.0), product of:
          0.07029594 = queryWeight, product of:
            1.9687915 = boost
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.014304135 = queryNorm
          0.2730153 = fieldWeight in 2367, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
      0.7653505 = weight(abstract_txt:llms in 2367) [ClassicSimilarity], result of:
        0.7653505 = score(doc=2367,freq=4.0), product of:
          0.7721445 = queryWeight, product of:
            5.956529 = boost
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.014304135 = queryNorm
          0.99120116 = fieldWeight in 2367, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.0546875 = fieldNorm(doc=2367)
    0.36 = coord(9/25)
  5. Yang, L.; Chen, H.; Li, Z.; Ding, X.; Wu, X.: Give us the facts : enhancing Large Language Models with knowledge graphs for fact-aware language modeling (2024) 0.30 
    0.2999618 = sum of:
  0.2999618 = product of:
    1.0712922 = sum of:
      0.03813756 = weight(abstract_txt:language in 2337) [ClassicSimilarity], result of:
        0.03813756 = score(doc=2337,freq=5.0), product of:
          0.06542579 = queryWeight, product of:
            1.0966005 = boost
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.014304135 = queryNorm
          0.5829132 = fieldWeight in 2337, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            4.1709876 = idf(docFreq=1863, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
      0.029141027 = weight(abstract_txt:large in 2337) [ClassicSimilarity], result of:
        0.029141027 = score(doc=2337,freq=2.0), product of:
          0.0742159 = queryWeight, product of:
            1.1679451 = boost
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.014304135 = queryNorm
          0.3926521 = fieldWeight in 2337, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.4423513 = idf(docFreq=1420, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
      0.040227126 = weight(abstract_txt:models in 2337) [ClassicSimilarity], result of:
        0.040227126 = score(doc=2337,freq=3.0), product of:
          0.08037901 = queryWeight, product of:
            1.2154729 = boost
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.014304135 = queryNorm
          0.5004681 = fieldWeight in 2337, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.623126 = idf(docFreq=1185, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
      0.01913032 = weight(abstract_txt:their in 2337) [ClassicSimilarity], result of:
        0.01913032 = score(doc=2337,freq=3.0), product of:
          0.05605855 = queryWeight, product of:
            1.2431986 = boost
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.014304135 = queryNorm
          0.34125605 = fieldWeight in 2337, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.1523883 = idf(docFreq=5161, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
      0.048036266 = weight(abstract_txt:improving in 2337) [ClassicSimilarity], result of:
        0.048036266 = score(doc=2337,freq=1.0), product of:
          0.13048178 = queryWeight, product of:
            1.5486345 = boost
            5.8903265 = idf(docFreq=333, maxDocs=44421)
            0.014304135 = queryNorm
          0.3681454 = fieldWeight in 2337, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.8903265 = idf(docFreq=333, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
      0.021933561 = weight(abstract_txt:with in 2337) [ClassicSimilarity], result of:
        0.021933561 = score(doc=2337,freq=4.0), product of:
          0.07029594 = queryWeight, product of:
            1.9687915 = boost
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.014304135 = queryNorm
          0.31201747 = fieldWeight in 2337, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            2.4961398 = idf(docFreq=9949, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
      0.8746863 = weight(abstract_txt:llms in 2337) [ClassicSimilarity], result of:
        0.8746863 = score(doc=2337,freq=4.0), product of:
          0.7721445 = queryWeight, product of:
            5.956529 = boost
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.014304135 = queryNorm
          1.1328013 = fieldWeight in 2337, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            9.06241 = idf(docFreq=13, maxDocs=44421)
            0.0625 = fieldNorm(doc=2337)
    0.28 = coord(7/25)