Document (#44404)

Author

Harrow, J.
Wickersham, L.
Rotherham, S.
Ella Farnsworth, E.
McElhenny, G.

Title

Contextual depth projection in Large Language Models through semantic lattice frameworks

Source

DOI: 10.31219/osf.io/k59aj

Year

2024

Abstract

Semantic representation within computational linguistics requires innovative methodologies to address limitations in contextual understanding and hierarchical reasoning. The semantic lattice framework introduced here leverages graph-based structures to redefine contextual relationships, facilitating dynamic adjustments that significantly enhance linguistic comprehension. Through its integration into transformer-based architectures, the framework achieves improved accuracy and efficiency across diverse natural language tasks, including classification, summarization, and cross-referential analysis. Experimental results demonstrate substantial improvements in coherence and adaptability, particularly in processing extended narratives and highly structured textual datasets. The proposed framework incorporates weighted node-edge dynamics to capture local and global dependencies, addressing challenges associated with ambiguity and non-linear semantic constructs. Quantitative metrics reveal notable gains in performance, while qualitative observations highlight advancements in semantic alignment and contextual depth. The modularity of the framework ensures seamless integration without necessitating extensive architectural modifications, making it a scalable solution for modern computational demands. Robustness to input variability further reinforces its applicability to real-world scenarios, where data inconsistencies are common. Detailed analyses of inference times and energy consumption demonstrate its computational efficiency, balancing enhanced capabilities with resource constraints. Case studies across domains such as legal, medical, and literary datasets emphasize the framework's ability to generalize and maintain high accuracy in specialized applications. The findings demonstrate the transformative potential of graph-enhanced methodologies in achieving unprecedented levels of linguistic precision and contextual awareness. This approach establishes a foundational shift toward more sophisticated representations in large-scale language models, ensuring both scalability and accuracy in processing complex textual data.

Content

Vgl.: https://www.researchgate.net/publication/386983932_Contextual_Depth_Projection_in_Large_Language_Models_Through_Semantic_Lattice_Frameworks.

Theme

Computerlinguistik
Wissensrepräsentation

Similar documents (content)

1. Melucci, M.: Contextual search : a computational framework (2012) 0.29

   0.28890234 = sum of:
     0.28890234 = product of:
       1.0317941 = sum of:
         0.023922106 = weight(abstract_txt:large in 913) [ClassicSimilarity], result of:
           0.023922106 = score(doc=913,freq=1.0), product of:
             0.08616016 = queryWeight, product of:
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.01939517 = queryNorm
             0.27764696 = fieldWeight in 913, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
         0.046701256 = weight(abstract_txt:models in 913) [ClassicSimilarity], result of:
           0.046701256 = score(doc=913,freq=3.0), product of:
             0.093315154 = queryWeight, product of:
               1.0406935 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.01939517 = queryNorm
             0.5004681 = fieldWeight in 913, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
         0.032596353 = weight(abstract_txt:processing in 913) [ClassicSimilarity], result of:
           0.032596353 = score(doc=913,freq=1.0), product of:
             0.105897635 = queryWeight, product of:
               1.1086383 = boost
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.01939517 = queryNorm
             0.30781004 = fieldWeight in 913, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
         0.0742864 = weight(abstract_txt:methodologies in 913) [ClassicSimilarity], result of:
           0.0742864 = score(doc=913,freq=1.0), product of:
             0.18339173 = queryWeight, product of:
               1.4589373 = boost
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.01939517 = queryNorm
             0.4050695 = fieldWeight in 913, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.481112 = idf(docFreq=184, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
         0.08871824 = weight(abstract_txt:framework in 913) [ClassicSimilarity], result of:
           0.08871824 = score(doc=913,freq=3.0), product of:
             0.1803371 = queryWeight, product of:
               2.0459936 = boost
               4.5445113 = idf(docFreq=1282, maxDocs=44421)
               0.01939517 = queryNorm
             0.49195778 = fieldWeight in 913, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.5445113 = idf(docFreq=1282, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
         0.20956706 = weight(abstract_txt:computational in 913) [ClassicSimilarity], result of:
           0.20956706 = score(doc=913,freq=4.0), product of:
             0.26403773 = queryWeight, product of:
               2.144003 = boost
               6.3496094 = idf(docFreq=210, maxDocs=44421)
               0.01939517 = queryNorm
             0.7937012 = fieldWeight in 913, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.3496094 = idf(docFreq=210, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
         0.5560027 = weight(abstract_txt:contextual in 913) [ClassicSimilarity], result of:
           0.5560027 = score(doc=913,freq=10.0), product of:
             0.44205007 = queryWeight, product of:
               3.581397 = boost
               6.3639297 = idf(docFreq=207, maxDocs=44421)
               0.01939517 = queryNorm
             1.2577821 = fieldWeight in 913, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               6.3639297 = idf(docFreq=207, maxDocs=44421)
               0.0625 = fieldNorm(doc=913)
       0.28 = coord(7/25)
   

2. Multilingual information management : current levels and future abilities. A report Commissioned by the US National Science Foundation and also delivered to the European Commission's Language Engineering Office and the US Defense Advanced Research Projects Agency, April 1999 (1999) 0.15

   0.14994395 = sum of:
     0.14994395 = product of:
       0.41651094 = sum of:
         0.023592608 = weight(abstract_txt:models in 68) [ClassicSimilarity], result of:
           0.023592608 = score(doc=68,freq=1.0), product of:
             0.093315154 = queryWeight, product of:
               1.0406935 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.01939517 = queryNorm
             0.2528272 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.02852181 = weight(abstract_txt:processing in 68) [ClassicSimilarity], result of:
           0.02852181 = score(doc=68,freq=1.0), product of:
             0.105897635 = queryWeight, product of:
               1.1086383 = boost
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.01939517 = queryNorm
             0.26933378 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.0313269 = weight(abstract_txt:across in 68) [ClassicSimilarity], result of:
           0.0313269 = score(doc=68,freq=1.0), product of:
             0.11273181 = queryWeight, product of:
               1.1438524 = boost
               5.081394 = idf(docFreq=749, maxDocs=44421)
               0.01939517 = queryNorm
             0.27788875 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.081394 = idf(docFreq=749, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.035404574 = weight(abstract_txt:integration in 68) [ClassicSimilarity], result of:
           0.035404574 = score(doc=68,freq=1.0), product of:
             0.12231351 = queryWeight, product of:
               1.1914723 = boost
               5.2929387 = idf(docFreq=606, maxDocs=44421)
               0.01939517 = queryNorm
             0.2894576 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2929387 = idf(docFreq=606, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.055709083 = weight(abstract_txt:enhanced in 68) [ClassicSimilarity], result of:
           0.055709083 = score(doc=68,freq=1.0), product of:
             0.16546984 = queryWeight, product of:
               1.3858179 = boost
               6.15629 = idf(docFreq=255, maxDocs=44421)
               0.01939517 = queryNorm
             0.33667213 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.15629 = idf(docFreq=255, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.051976465 = weight(abstract_txt:language in 68) [ClassicSimilarity], result of:
           0.051976465 = score(doc=68,freq=4.0), product of:
             0.113933064 = queryWeight, product of:
               1.4083716 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01939517 = queryNorm
             0.45620176 = fieldWeight in 68, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.04481881 = weight(abstract_txt:framework in 68) [ClassicSimilarity], result of:
           0.04481881 = score(doc=68,freq=1.0), product of:
             0.1803371 = queryWeight, product of:
               2.0459936 = boost
               4.5445113 = idf(docFreq=1282, maxDocs=44421)
               0.01939517 = queryNorm
             0.24852796 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5445113 = idf(docFreq=1282, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.091685586 = weight(abstract_txt:computational in 68) [ClassicSimilarity], result of:
           0.091685586 = score(doc=68,freq=1.0), product of:
             0.26403773 = queryWeight, product of:
               2.144003 = boost
               6.3496094 = idf(docFreq=210, maxDocs=44421)
               0.01939517 = queryNorm
             0.34724426 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.3496094 = idf(docFreq=210, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.053475086 = weight(abstract_txt:semantic in 68) [ClassicSimilarity], result of:
           0.053475086 = score(doc=68,freq=1.0), product of:
             0.2185324 = queryWeight, product of:
               2.5181098 = boost
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.01939517 = queryNorm
             0.24470095 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
       0.36 = coord(9/25)
   

3. Van Valin, R.D. Jr.: Semantic macroroles and language processing (2006) 0.13

   0.1331605 = sum of:
     0.1331605 = product of:
       0.55483544 = sum of:
         0.047185216 = weight(abstract_txt:models in 664) [ClassicSimilarity], result of:
           0.047185216 = score(doc=664,freq=4.0), product of:
             0.093315154 = queryWeight, product of:
               1.0406935 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.01939517 = queryNorm
             0.5056544 = fieldWeight in 664, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=664)
         0.06377671 = weight(abstract_txt:processing in 664) [ClassicSimilarity], result of:
           0.06377671 = score(doc=664,freq=5.0), product of:
             0.105897635 = queryWeight, product of:
               1.1086383 = boost
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.01939517 = queryNorm
             0.60224867 = fieldWeight in 664, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.0546875 = fieldNorm(doc=664)
         0.10484864 = weight(abstract_txt:linguistic in 664) [ClassicSimilarity], result of:
           0.10484864 = score(doc=664,freq=5.0), product of:
             0.14750952 = queryWeight, product of:
               1.3084489 = boost
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.01939517 = queryNorm
             0.71079236 = fieldWeight in 664, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.0546875 = fieldNorm(doc=664)
         0.05811145 = weight(abstract_txt:language in 664) [ClassicSimilarity], result of:
           0.05811145 = score(doc=664,freq=5.0), product of:
             0.113933064 = queryWeight, product of:
               1.4083716 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01939517 = queryNorm
             0.51004905 = fieldWeight in 664, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=664)
         0.129663 = weight(abstract_txt:computational in 664) [ClassicSimilarity], result of:
           0.129663 = score(doc=664,freq=2.0), product of:
             0.26403773 = queryWeight, product of:
               2.144003 = boost
               6.3496094 = idf(docFreq=210, maxDocs=44421)
               0.01939517 = queryNorm
             0.49107754 = fieldWeight in 664, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.3496094 = idf(docFreq=210, maxDocs=44421)
               0.0546875 = fieldNorm(doc=664)
         0.15125039 = weight(abstract_txt:semantic in 664) [ClassicSimilarity], result of:
           0.15125039 = score(doc=664,freq=8.0), product of:
             0.2185324 = queryWeight, product of:
               2.5181098 = boost
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.01939517 = queryNorm
             0.6921188 = fieldWeight in 664, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.4745317 = idf(docFreq=1375, maxDocs=44421)
               0.0546875 = fieldNorm(doc=664)
       0.24 = coord(6/25)
   

4. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amode, D.; Sutskever, I.: Language models are unsupervised multitask learners 0.13

   0.13173535 = sum of:
     0.13173535 = product of:
       0.4704834 = sum of:
         0.026962982 = weight(abstract_txt:models in 1872) [ClassicSimilarity], result of:
           0.026962982 = score(doc=1872,freq=1.0), product of:
             0.093315154 = queryWeight, product of:
               1.0406935 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.01939517 = queryNorm
             0.28894538 = fieldWeight in 1872, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
         0.117134094 = weight(abstract_txt:transformer in 1872) [ClassicSimilarity], result of:
           0.117134094 = score(doc=1872,freq=1.0), product of:
             0.1971904 = queryWeight, product of:
               1.0697311 = boost
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.01939517 = queryNorm
             0.5940152 = fieldWeight in 1872, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.504243 = idf(docFreq=8, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
         0.046098206 = weight(abstract_txt:processing in 1872) [ClassicSimilarity], result of:
           0.046098206 = score(doc=1872,freq=2.0), product of:
             0.105897635 = queryWeight, product of:
               1.1086383 = boost
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.01939517 = queryNorm
             0.4353091 = fieldWeight in 1872, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
         0.03580217 = weight(abstract_txt:across in 1872) [ClassicSimilarity], result of:
           0.03580217 = score(doc=1872,freq=1.0), product of:
             0.11273181 = queryWeight, product of:
               1.1438524 = boost
               5.081394 = idf(docFreq=749, maxDocs=44421)
               0.01939517 = queryNorm
             0.31758714 = fieldWeight in 1872, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.081394 = idf(docFreq=749, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
         0.0727519 = weight(abstract_txt:language in 1872) [ClassicSimilarity], result of:
           0.0727519 = score(doc=1872,freq=6.0), product of:
             0.113933064 = queryWeight, product of:
               1.4083716 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01939517 = queryNorm
             0.6385495 = fieldWeight in 1872, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
         0.10835194 = weight(abstract_txt:datasets in 1872) [ClassicSimilarity], result of:
           0.10835194 = score(doc=1872,freq=2.0), product of:
             0.18720669 = queryWeight, product of:
               1.4740337 = boost
               6.548176 = idf(docFreq=172, maxDocs=44421)
               0.01939517 = queryNorm
             0.57878244 = fieldWeight in 1872, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.548176 = idf(docFreq=172, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
         0.06338208 = weight(abstract_txt:demonstrate in 1872) [ClassicSimilarity], result of:
           0.06338208 = score(doc=1872,freq=1.0), product of:
             0.18884912 = queryWeight, product of:
               1.8132174 = boost
               5.3699656 = idf(docFreq=561, maxDocs=44421)
               0.01939517 = queryNorm
             0.33562285 = fieldWeight in 1872, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3699656 = idf(docFreq=561, maxDocs=44421)
               0.0625 = fieldNorm(doc=1872)
       0.28 = coord(7/25)
   

5. Cavalcante Dourado, Í.; Galante, R.; Gonçalves, M.A.; Silva Torres, R. de: Bag of textual graphs (BoTG) : a general graph-based text representation model (2019) 0.13

   0.13044919 = sum of:
     0.13044919 = product of:
       0.46588996 = sum of:
         0.023922106 = weight(abstract_txt:large in 291) [ClassicSimilarity], result of:
           0.023922106 = score(doc=291,freq=1.0), product of:
             0.08616016 = queryWeight, product of:
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.01939517 = queryNorm
             0.27764696 = fieldWeight in 291, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
         0.038131416 = weight(abstract_txt:models in 291) [ClassicSimilarity], result of:
           0.038131416 = score(doc=291,freq=2.0), product of:
             0.093315154 = queryWeight, product of:
               1.0406935 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.01939517 = queryNorm
             0.40863046 = fieldWeight in 291, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
         0.115824156 = weight(abstract_txt:textual in 291) [ClassicSimilarity], result of:
           0.115824156 = score(doc=291,freq=4.0), product of:
             0.15534107 = queryWeight, product of:
               1.3427337 = boost
               5.9648952 = idf(docFreq=309, maxDocs=44421)
               0.01939517 = queryNorm
             0.7456119 = fieldWeight in 291, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.9648952 = idf(docFreq=309, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
         0.06136207 = weight(abstract_txt:efficiency in 291) [ClassicSimilarity], result of:
           0.06136207 = score(doc=291,freq=1.0), product of:
             0.1614508 = queryWeight, product of:
               1.3688847 = boost
               6.0810666 = idf(docFreq=275, maxDocs=44421)
               0.01939517 = queryNorm
             0.38006666 = fieldWeight in 291, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0810666 = idf(docFreq=275, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
         0.07682007 = weight(abstract_txt:graph in 291) [ClassicSimilarity], result of:
           0.07682007 = score(doc=291,freq=1.0), product of:
             0.18753831 = queryWeight, product of:
               1.4753388 = boost
               6.553973 = idf(docFreq=171, maxDocs=44421)
               0.01939517 = queryNorm
             0.40962332 = fieldWeight in 291, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.553973 = idf(docFreq=171, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
         0.06338208 = weight(abstract_txt:demonstrate in 291) [ClassicSimilarity], result of:
           0.06338208 = score(doc=291,freq=1.0), product of:
             0.18884912 = queryWeight, product of:
               1.8132174 = boost
               5.3699656 = idf(docFreq=561, maxDocs=44421)
               0.01939517 = queryNorm
             0.33562285 = fieldWeight in 291, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3699656 = idf(docFreq=561, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
         0.08644804 = weight(abstract_txt:accuracy in 291) [ClassicSimilarity], result of:
           0.08644804 = score(doc=291,freq=1.0), product of:
             0.23225982 = queryWeight, product of:
               2.0108488 = boost
               5.9552646 = idf(docFreq=312, maxDocs=44421)
               0.01939517 = queryNorm
             0.37220404 = fieldWeight in 291, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9552646 = idf(docFreq=312, maxDocs=44421)
               0.0625 = fieldNorm(doc=291)
       0.28 = coord(7/25)