Document (#43801)

Author
Schaer, P.
Title
Sprachmodelle und neuronale Netze im Information Retrieval
Source
Grundlagen der Informationswissenschaft. Hrsg.: Rainer Kuhlen, Dirk Lewandowski, Wolfgang Semar und Christa Womser-Hacker. 7., völlig neu gefasste Ausg
Imprint
Berlin : DeGruyter
Year
2023
Pages
S.455-466
Abstract
In den letzten Jahren haben Sprachmodelltechnologien unterschiedlichster Ausprägungen in der Informationswissenschaft Einzug gehalten. Diesen Sprachmodellen, die unter den Bezeichnungen GPT, ELMo oder BERT bekannt sind, ist gemein, dass sie dank sehr großer Webkorpora auf eine Datenbasis zurückgreifen, die bei vorherigen Sprachmodellansätzen undenkbar war. Gleichzeitig setzen diese Modelle auf neuere Entwicklungen des maschinellen Lernens, insbesondere auf künstliche neuronale Netze. Diese Technologien haben auch im Information Retrieval (IR) Fuß gefasst und bereits kurz nach ihrer Einführung sprunghafte, substantielle Leistungssteigerungen erzielt. Neuronale Netze haben in Kombination mit großen vortrainierten Sprachmodellen und kontextualisierten Worteinbettungen geführt. Wurde in vergangenen Jahren immer wieder eine stagnierende Retrievalleistung beklagt, die Leistungssteigerungen nur gegenüber "schwachen Baselines" aufwies, so konnten mit diesen technischen und methodischen Innovationen beeindruckende Leistungssteigerungen in Aufgaben wie dem klassischen Ad-hoc-Retrieval, der maschinellen Übersetzung oder auch dem Question Answering erzielt werden. In diesem Kapitel soll ein kurzer Überblick über die Grundlagen der Sprachmodelle und der NN gegeben werden, um die prinzipiellen Bausteine zu verstehen, die hinter aktuellen Technologien wie ELMo oder BERT stecken, die die Welt des NLP und IR im Moment beherrschen.
Footnote
Vgl.: https://doi.org/10.1515/9783110769043.
Theme
Computerlinguistik
Object
GPT-3

Similar documents (author)

  1. Schaer, P.: Integration von Open-Access-Repositorien in Fachportale (2010) 5.54 
    5.5426593 = sum of:
  5.5426593 = weight(author_txt:schaer in 3320) [ClassicSimilarity], result of:
    5.5426593 = fieldWeight in 3320, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.868255 = idf(docFreq=16, maxDocs=44421)
      0.625 = fieldNorm(doc=3320)
  2. Munkelt, J.; Schaer, P.: Towards an IR test collection for the German National Library (2018) 4.43 
    4.4341273 = sum of:
  4.4341273 = weight(author_txt:schaer in 780) [ClassicSimilarity], result of:
    4.4341273 = fieldWeight in 780, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.868255 = idf(docFreq=16, maxDocs=44421)
      0.5 = fieldNorm(doc=780)
  3. Mayr, P.; Schaer, P.; Mutschke, P.: ¬A science model driven retrieval prototype (2011) 3.33 
    3.3255954 = sum of:
  3.3255954 = weight(author_txt:schaer in 1649) [ClassicSimilarity], result of:
    3.3255954 = fieldWeight in 1649, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.868255 = idf(docFreq=16, maxDocs=44421)
      0.375 = fieldNorm(doc=1649)
  4. Neumann, M.; Steinberg, J.; Schaer, P.: Web-ccraping for non-programmers : introducing OXPath for digital library metadata harvesting (2017) 3.33 
    3.3255954 = sum of:
  3.3255954 = weight(author_txt:schaer in 4895) [ClassicSimilarity], result of:
    3.3255954 = fieldWeight in 4895, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.868255 = idf(docFreq=16, maxDocs=44421)
      0.375 = fieldNorm(doc=4895)
  5. Munkelt, J.; Schaer, P.; Lepsky, K.: Towards an IR test collection for the German National Library (2018) 3.33 
    3.3255954 = sum of:
  3.3255954 = weight(author_txt:schaer in 311) [ClassicSimilarity], result of:
    3.3255954 = fieldWeight in 311, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.868255 = idf(docFreq=16, maxDocs=44421)
      0.375 = fieldNorm(doc=311)

Similar documents (content)

  1. Matt, A.; Schaber, E.; Violet, B.: Vielfältige Formate und dynamische Umsetzung : Mathematik-Kommunikation zu Künstlicher Intelligenz bei IMAGINARY (2023) 0.15 
    0.14621226 = sum of:
  0.14621226 = product of:
    0.73106134 = sum of:
      0.029160755 = weight(abstract_txt:diese in 2307) [ClassicSimilarity], result of:
        0.029160755 = score(doc=2307,freq=1.0), product of:
          0.07176281 = queryWeight, product of:
            4.3343906 = idf(docFreq=1582, maxDocs=44421)
            0.016556608 = queryNorm
          0.40634912 = fieldWeight in 2307, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.3343906 = idf(docFreq=1582, maxDocs=44421)
            0.09375 = fieldNorm(doc=2307)
      0.040990245 = weight(abstract_txt:oder in 2307) [ClassicSimilarity], result of:
        0.040990245 = score(doc=2307,freq=1.0), product of:
          0.10308236 = queryWeight, product of:
            1.4678718 = boost
            4.241553 = idf(docFreq=1736, maxDocs=44421)
            0.016556608 = queryNorm
          0.3976456 = fieldWeight in 2307, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.241553 = idf(docFreq=1736, maxDocs=44421)
            0.09375 = fieldNorm(doc=2307)
      0.15462959 = weight(abstract_txt:maschinellen in 2307) [ClassicSimilarity], result of:
        0.15462959 = score(doc=2307,freq=1.0), product of:
          0.21822038 = queryWeight, product of:
            1.7438052 = boost
            7.558333 = idf(docFreq=62, maxDocs=44421)
            0.016556608 = queryNorm
          0.7085937 = fieldWeight in 2307, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.558333 = idf(docFreq=62, maxDocs=44421)
            0.09375 = fieldNorm(doc=2307)
      0.2095687 = weight(abstract_txt:netze in 2307) [ClassicSimilarity], result of:
        0.2095687 = score(doc=2307,freq=1.0), product of:
          0.30592498 = queryWeight, product of:
            2.5287354 = boost
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.016556608 = queryNorm
          0.68503296 = fieldWeight in 2307, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.09375 = fieldNorm(doc=2307)
      0.29671207 = weight(abstract_txt:neuronale in 2307) [ClassicSimilarity], result of:
        0.29671207 = score(doc=2307,freq=1.0), product of:
          0.3857336 = queryWeight, product of:
            2.8394856 = boost
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.016556608 = queryNorm
          0.769215 = fieldWeight in 2307, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.09375 = fieldNorm(doc=2307)
    0.2 = coord(5/25)
  2. Bischoff, M.: KI lernt die Sprache der Mathematik (2020) 0.11 
    0.11214041 = sum of:
  0.11214041 = product of:
    0.93450344 = sum of:
      0.090702124 = weight(abstract_txt:haben in 904) [ClassicSimilarity], result of:
        0.090702124 = score(doc=904,freq=1.0), product of:
          0.12452115 = queryWeight, product of:
            1.6133088 = boost
            4.661807 = idf(docFreq=1140, maxDocs=44421)
            0.016556608 = queryNorm
          0.7284074 = fieldWeight in 904, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.661807 = idf(docFreq=1140, maxDocs=44421)
            0.15625 = fieldNorm(doc=904)
      0.34928116 = weight(abstract_txt:netze in 904) [ClassicSimilarity], result of:
        0.34928116 = score(doc=904,freq=1.0), product of:
          0.30592498 = queryWeight, product of:
            2.5287354 = boost
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.016556608 = queryNorm
          1.1417216 = fieldWeight in 904, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.15625 = fieldNorm(doc=904)
      0.49452013 = weight(abstract_txt:neuronale in 904) [ClassicSimilarity], result of:
        0.49452013 = score(doc=904,freq=1.0), product of:
          0.3857336 = queryWeight, product of:
            2.8394856 = boost
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.016556608 = queryNorm
          1.282025 = fieldWeight in 904, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.15625 = fieldNorm(doc=904)
    0.12 = coord(3/25)
  3. Angerer, C.: Neuronale Netze : Revolution für die Wissenschaft? (2018) 0.11 
    0.110944666 = sum of:
  0.110944666 = product of:
    0.9245389 = sum of:
      0.08073762 = weight(abstract_txt:jahren in 23) [ClassicSimilarity], result of:
        0.08073762 = score(doc=23,freq=1.0), product of:
          0.1006588 = queryWeight, product of:
            1.1843394 = boost
            5.1333895 = idf(docFreq=711, maxDocs=44421)
            0.016556608 = queryNorm
          0.8020921 = fieldWeight in 23, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.1333895 = idf(docFreq=711, maxDocs=44421)
            0.15625 = fieldNorm(doc=23)
      0.34928116 = weight(abstract_txt:netze in 23) [ClassicSimilarity], result of:
        0.34928116 = score(doc=23,freq=1.0), product of:
          0.30592498 = queryWeight, product of:
            2.5287354 = boost
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.016556608 = queryNorm
          1.1417216 = fieldWeight in 23, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.15625 = fieldNorm(doc=23)
      0.49452013 = weight(abstract_txt:neuronale in 23) [ClassicSimilarity], result of:
        0.49452013 = score(doc=23,freq=1.0), product of:
          0.3857336 = queryWeight, product of:
            2.8394856 = boost
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.016556608 = queryNorm
          1.282025 = fieldWeight in 23, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.15625 = fieldNorm(doc=23)
    0.12 = coord(3/25)
  4. Lämmel, U.; Cleve, J.: Künstliche Intelligenz : mit 50 Tabellen, 43 Beispielen, 208 Aufgaben, 89 Kontrollfragen und Referatsthemen (2008) 0.11 
    0.1057822 = sum of:
  0.1057822 = product of:
    0.6611388 = sum of:
      0.01701044 = weight(abstract_txt:diese in 1642) [ClassicSimilarity], result of:
        0.01701044 = score(doc=1642,freq=1.0), product of:
          0.07176281 = queryWeight, product of:
            4.3343906 = idf(docFreq=1582, maxDocs=44421)
            0.016556608 = queryNorm
          0.23703699 = fieldWeight in 1642, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.3343906 = idf(docFreq=1582, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1642)
      0.04489526 = weight(abstract_txt:haben in 1642) [ClassicSimilarity], result of:
        0.04489526 = score(doc=1642,freq=2.0), product of:
          0.12452115 = queryWeight, product of:
            1.6133088 = boost
            4.661807 = idf(docFreq=1140, maxDocs=44421)
            0.016556608 = queryNorm
          0.36054325 = fieldWeight in 1642, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.661807 = idf(docFreq=1140, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1642)
      0.2994462 = weight(abstract_txt:netze in 1642) [ClassicSimilarity], result of:
        0.2994462 = score(doc=1642,freq=6.0), product of:
          0.30592498 = queryWeight, product of:
            2.5287354 = boost
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.016556608 = queryNorm
          0.97882235 = fieldWeight in 1642, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            7.3070183 = idf(docFreq=80, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1642)
      0.29978687 = weight(abstract_txt:neuronale in 1642) [ClassicSimilarity], result of:
        0.29978687 = score(doc=1642,freq=3.0), product of:
          0.3857336 = queryWeight, product of:
            2.8394856 = boost
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.016556608 = queryNorm
          0.7771863 = fieldWeight in 1642, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            8.20496 = idf(docFreq=32, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1642)
    0.16 = coord(4/25)
  5. Assfalg, R.: Metadaten (2023) 0.09 
    0.093235016 = sum of:
  0.093235016 = product of:
    0.46617508 = sum of:
      0.05892589 = weight(abstract_txt:diese in 1788) [ClassicSimilarity], result of:
        0.05892589 = score(doc=1788,freq=3.0), product of:
          0.07176281 = queryWeight, product of:
            4.3343906 = idf(docFreq=1582, maxDocs=44421)
            0.016556608 = queryNorm
          0.8211202 = fieldWeight in 1788, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.3343906 = idf(docFreq=1582, maxDocs=44421)
            0.109375 = fieldNorm(doc=1788)
      0.13783127 = weight(abstract_txt:ausprägungen in 1788) [ClassicSimilarity], result of:
        0.13783127 = score(doc=1788,freq=1.0), product of:
          0.14475189 = queryWeight, product of:
            1.0042629 = boost
            8.705735 = idf(docFreq=19, maxDocs=44421)
            0.016556608 = queryNorm
          0.9521898 = fieldWeight in 1788, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.705735 = idf(docFreq=19, maxDocs=44421)
            0.109375 = fieldNorm(doc=1788)
      0.14870383 = weight(abstract_txt:gefasst in 1788) [ClassicSimilarity], result of:
        0.14870383 = score(doc=1788,freq=1.0), product of:
          0.15226749 = queryWeight, product of:
            1.0300039 = boost
            8.928879 = idf(docFreq=15, maxDocs=44421)
            0.016556608 = queryNorm
          0.9765961 = fieldWeight in 1788, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.928879 = idf(docFreq=15, maxDocs=44421)
            0.109375 = fieldNorm(doc=1788)
      0.0728921 = weight(abstract_txt:diesen in 1788) [ClassicSimilarity], result of:
        0.0728921 = score(doc=1788,freq=1.0), product of:
          0.11926766 = queryWeight, product of:
            1.2891743 = boost
            5.5877852 = idf(docFreq=451, maxDocs=44421)
            0.016556608 = queryNorm
          0.61116403 = fieldWeight in 1788, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.5877852 = idf(docFreq=451, maxDocs=44421)
            0.109375 = fieldNorm(doc=1788)
      0.047821954 = weight(abstract_txt:oder in 1788) [ClassicSimilarity], result of:
        0.047821954 = score(doc=1788,freq=1.0), product of:
          0.10308236 = queryWeight, product of:
            1.4678718 = boost
            4.241553 = idf(docFreq=1736, maxDocs=44421)
            0.016556608 = queryNorm
          0.46391985 = fieldWeight in 1788, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.241553 = idf(docFreq=1736, maxDocs=44421)
            0.109375 = fieldNorm(doc=1788)
    0.2 = coord(5/25)