Document (#43161)

Author

Lee, G.E.
Sun, A.

Title

Understanding the stability of medical concept embeddings

Source

Journal of the Association for Information Science and Technology. 72(2021) no.3, S.346-356

Year

2021

Abstract

Frequency is one of the major factors for training quality word embeddings. Several studies have recently discussed the stability of word embeddings in general domain and suggested factors influencing the stability. In this work, we conduct a detailed analysis on the stability of concept embeddings in medical domain, particularly in relations with concept frequency. The analysis reveals the surprising high stability of low-frequency concepts: low-frequency (<100) concepts have the same high stability as high-frequency (>1,000) concepts. To develop a deeper understanding of this finding, we propose a new factor, the noisiness of context words, which influences the stability of medical concept embeddings regardless of high or low frequency. We evaluate the proposed factor by showing the linear correlation with the stability of medical concept embeddings. The correlations are clear and consistent with various groups of medical concepts. Based on the linear relations, we make suggestions on ways to adjust the noisiness of context words for the improvement of stability. Finally, we demonstrate that the linear relation of the proposed factor extends to the word embedding stability in general domain.

Content

Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24411.

Theme

Computerlinguistik

Field

Medizin

Similar documents (content)

1. Li, G.; Siddharth, L.; Luo, J.: Embedding knowledge graph of patent metadata to measure knowledge proximity (2023) 0.18

   0.18069576 = sum of:
     0.18069576 = product of:
       0.9034788 = sum of:
         0.06024053 = weight(abstract_txt:embedding in 1921) [ClassicSimilarity], result of:
           0.06024053 = score(doc=1921,freq=2.0), product of:
             0.069445014 = queryWeight, product of:
               1.0767992 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.00821417 = queryNorm
             0.8674565 = fieldWeight in 1921, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.078125 = fieldNorm(doc=1921)
         0.014321229 = weight(abstract_txt:context in 1921) [ClassicSimilarity], result of:
           0.014321229 = score(doc=1921,freq=1.0), product of:
             0.042304713 = queryWeight, product of:
               1.1885667 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.00821417 = queryNorm
             0.33852562 = fieldWeight in 1921, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.078125 = fieldNorm(doc=1921)
         0.030002693 = weight(abstract_txt:relations in 1921) [ClassicSimilarity], result of:
           0.030002693 = score(doc=1921,freq=1.0), product of:
             0.06926418 = queryWeight, product of:
               1.5208399 = boost
               5.5444884 = idf(docFreq=471, maxDocs=44421)
               0.00821417 = queryNorm
             0.43316317 = fieldWeight in 1921, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5444884 = idf(docFreq=471, maxDocs=44421)
               0.078125 = fieldNorm(doc=1921)
         0.039486784 = weight(abstract_txt:domain in 1921) [ClassicSimilarity], result of:
           0.039486784 = score(doc=1921,freq=2.0), product of:
             0.07557709 = queryWeight, product of:
               1.9456731 = boost
               4.7288613 = idf(docFreq=1066, maxDocs=44421)
               0.00821417 = queryNorm
             0.5224703 = fieldWeight in 1921, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7288613 = idf(docFreq=1066, maxDocs=44421)
               0.078125 = fieldNorm(doc=1921)
         0.75942755 = weight(abstract_txt:embeddings in 1921) [ClassicSimilarity], result of:
           0.75942755 = score(doc=1921,freq=3.0), product of:
             0.597117 = queryWeight, product of:
               7.734273 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.00821417 = queryNorm
             1.2718236 = fieldWeight in 1921, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.078125 = fieldNorm(doc=1921)
       0.2 = coord(5/25)
   

2. Lee, Y.-Y.; Ke, H.; Yen, T.-Y.; Huang, H.-H.; Chen, H.-H.: Combining and learning word embedding with WordNet for semantic relatedness and similarity measurement (2020) 0.15

   0.15336472 = sum of:
     0.15336472 = product of:
       0.63901967 = sum of:
         0.035627343 = weight(abstract_txt:correlations in 871) [ClassicSimilarity], result of:
           0.035627343 = score(doc=871,freq=1.0), product of:
             0.061647315 = queryWeight, product of:
               1.0145448 = boost
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.00821417 = queryNorm
             0.57792205 = fieldWeight in 871, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3974023 = idf(docFreq=73, maxDocs=44421)
               0.078125 = fieldNorm(doc=871)
         0.0041065183 = weight(abstract_txt:with in 871) [ClassicSimilarity], result of:
           0.0041065183 = score(doc=871,freq=1.0), product of:
             0.021057889 = queryWeight, product of:
               1.0270278 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.00821417 = queryNorm
             0.19501092 = fieldWeight in 871, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.078125 = fieldNorm(doc=871)
         0.06024053 = weight(abstract_txt:embedding in 871) [ClassicSimilarity], result of:
           0.06024053 = score(doc=871,freq=2.0), product of:
             0.069445014 = queryWeight, product of:
               1.0767992 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.00821417 = queryNorm
             0.8674565 = fieldWeight in 871, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.078125 = fieldNorm(doc=871)
         0.027043095 = weight(abstract_txt:words in 871) [ClassicSimilarity], result of:
           0.027043095 = score(doc=871,freq=1.0), product of:
             0.06463079 = queryWeight, product of:
               1.4690917 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.00821417 = queryNorm
             0.4184243 = fieldWeight in 871, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.078125 = fieldNorm(doc=871)
         0.07354648 = weight(abstract_txt:word in 871) [ClassicSimilarity], result of:
           0.07354648 = score(doc=871,freq=3.0), product of:
             0.09994624 = queryWeight, product of:
               2.2374742 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.00821417 = queryNorm
             0.73586035 = fieldWeight in 871, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.078125 = fieldNorm(doc=871)
         0.4384557 = weight(abstract_txt:embeddings in 871) [ClassicSimilarity], result of:
           0.4384557 = score(doc=871,freq=1.0), product of:
             0.597117 = queryWeight, product of:
               7.734273 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.00821417 = queryNorm
             0.73428774 = fieldWeight in 871, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.078125 = fieldNorm(doc=871)
       0.24 = coord(6/25)
   

3. Safder, I.; Ali, M.; Aljohani, N.R.; Nawaz, R.; Hassan, S.-U.: Neural machine translation for in-text citation classification (2023) 0.15

   0.15096092 = sum of:
     0.15096092 = product of:
       0.6290039 = sum of:
         0.004645995 = weight(abstract_txt:with in 2055) [ClassicSimilarity], result of:
           0.004645995 = score(doc=2055,freq=2.0), product of:
             0.021057889 = queryWeight, product of:
               1.0270278 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.00821417 = queryNorm
             0.22062966 = fieldWeight in 2055, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=2055)
         0.03407719 = weight(abstract_txt:embedding in 2055) [ClassicSimilarity], result of:
           0.03407719 = score(doc=2055,freq=1.0), product of:
             0.069445014 = queryWeight, product of:
               1.0767992 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.00821417 = queryNorm
             0.4907075 = fieldWeight in 2055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.0625 = fieldNorm(doc=2055)
         0.032405242 = weight(abstract_txt:context in 2055) [ClassicSimilarity], result of:
           0.032405242 = score(doc=2055,freq=8.0), product of:
             0.042304713 = queryWeight, product of:
               1.1885667 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.00821417 = queryNorm
             0.76599604 = fieldWeight in 2055, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.0625 = fieldNorm(doc=2055)
         0.013779107 = weight(abstract_txt:proposed in 2055) [ClassicSimilarity], result of:
           0.013779107 = score(doc=2055,freq=1.0), product of:
             0.047843464 = queryWeight, product of:
               1.2639807 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.00821417 = queryNorm
             0.28800395 = fieldWeight in 2055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.0625 = fieldNorm(doc=2055)
         0.048040353 = weight(abstract_txt:word in 2055) [ClassicSimilarity], result of:
           0.048040353 = score(doc=2055,freq=2.0), product of:
             0.09994624 = queryWeight, product of:
               2.2374742 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.00821417 = queryNorm
             0.48066196 = fieldWeight in 2055, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.0625 = fieldNorm(doc=2055)
         0.496056 = weight(abstract_txt:embeddings in 2055) [ClassicSimilarity], result of:
           0.496056 = score(doc=2055,freq=2.0), product of:
             0.597117 = queryWeight, product of:
               7.734273 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.00821417 = queryNorm
             0.8307517 = fieldWeight in 2055, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.0625 = fieldNorm(doc=2055)
       0.24 = coord(6/25)
   

4. Sun, Q.; Shaw, D.; Davis, C.H.: ¬A model for estimating the occurence of same-frequency words and the boundary between high- and low-frequency words in texts (1999) 0.14

   0.13852352 = sum of:
     0.13852352 = product of:
       0.57718134 = sum of:
         0.004927822 = weight(abstract_txt:with in 4063) [ClassicSimilarity], result of:
           0.004927822 = score(doc=4063,freq=1.0), product of:
             0.021057889 = queryWeight, product of:
               1.0270278 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.00821417 = queryNorm
             0.23401311 = fieldWeight in 4063, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.09375 = fieldNorm(doc=4063)
         0.020668661 = weight(abstract_txt:proposed in 4063) [ClassicSimilarity], result of:
           0.020668661 = score(doc=4063,freq=1.0), product of:
             0.047843464 = queryWeight, product of:
               1.2639807 = boost
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.00821417 = queryNorm
             0.43200594 = fieldWeight in 4063, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.608063 = idf(docFreq=1203, maxDocs=44421)
               0.09375 = fieldNorm(doc=4063)
         0.07949014 = weight(abstract_txt:words in 4063) [ClassicSimilarity], result of:
           0.07949014 = score(doc=4063,freq=6.0), product of:
             0.06463079 = queryWeight, product of:
               1.4690917 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.00821417 = queryNorm
             1.2299113 = fieldWeight in 4063, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.09375 = fieldNorm(doc=4063)
         0.050954495 = weight(abstract_txt:word in 4063) [ClassicSimilarity], result of:
           0.050954495 = score(doc=4063,freq=1.0), product of:
             0.09994624 = queryWeight, product of:
               2.2374742 = boost
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.00821417 = queryNorm
             0.50981903 = fieldWeight in 4063, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4380693 = idf(docFreq=524, maxDocs=44421)
               0.09375 = fieldNorm(doc=4063)
         0.068171546 = weight(abstract_txt:high in 4063) [ClassicSimilarity], result of:
           0.068171546 = score(doc=4063,freq=2.0), product of:
             0.10601073 = queryWeight, product of:
               2.660842 = boost
               4.8502827 = idf(docFreq=944, maxDocs=44421)
               0.00821417 = queryNorm
             0.6430627 = fieldWeight in 4063, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.8502827 = idf(docFreq=944, maxDocs=44421)
               0.09375 = fieldNorm(doc=4063)
         0.3529687 = weight(abstract_txt:frequency in 4063) [ClassicSimilarity], result of:
           0.3529687 = score(doc=4063,freq=7.0), product of:
             0.23921014 = queryWeight, product of:
               4.8953032 = boost
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.00821417 = queryNorm
             1.475559 = fieldWeight in 4063, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.948895 = idf(docFreq=314, maxDocs=44421)
               0.09375 = fieldNorm(doc=4063)
       0.24 = coord(6/25)
   

5. Maaten, L. van den: Accelerating t-SNE using Tree-Based Algorithms (2014) 0.14

   0.13573314 = sum of:
     0.13573314 = product of:
       0.84833217 = sum of:
         0.004927822 = weight(abstract_txt:with in 4886) [ClassicSimilarity], result of:
           0.004927822 = score(doc=4886,freq=1.0), product of:
             0.021057889 = queryWeight, product of:
               1.0270278 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.00821417 = queryNorm
             0.23401311 = fieldWeight in 4886, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.09375 = fieldNorm(doc=4886)
         0.05111578 = weight(abstract_txt:embedding in 4886) [ClassicSimilarity], result of:
           0.05111578 = score(doc=4886,freq=1.0), product of:
             0.069445014 = queryWeight, product of:
               1.0767992 = boost
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.00821417 = queryNorm
             0.7360612 = fieldWeight in 4886, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.85132 = idf(docFreq=46, maxDocs=44421)
               0.09375 = fieldNorm(doc=4886)
         0.048204564 = weight(abstract_txt:high in 4886) [ClassicSimilarity], result of:
           0.048204564 = score(doc=4886,freq=1.0), product of:
             0.10601073 = queryWeight, product of:
               2.660842 = boost
               4.8502827 = idf(docFreq=944, maxDocs=44421)
               0.00821417 = queryNorm
             0.454714 = fieldWeight in 4886, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8502827 = idf(docFreq=944, maxDocs=44421)
               0.09375 = fieldNorm(doc=4886)
         0.744084 = weight(abstract_txt:embeddings in 4886) [ClassicSimilarity], result of:
           0.744084 = score(doc=4886,freq=2.0), product of:
             0.597117 = queryWeight, product of:
               7.734273 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.00821417 = queryNorm
             1.2461276 = fieldWeight in 4886, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.09375 = fieldNorm(doc=4886)
       0.16 = coord(4/25)