Document (#21252)

Author

Charniak, E.

Title

Statistical techniques for natural language parsing

Source

AI magazine. 18(1997) no.4, S.33-43

Year

1997

Abstract

Reviews statistical work on syntactic parsing and considers part-of-speech tagging, which was the 1st syntactic problem to be successfully be attacked by statistical techniques and discusses statistical parsing. Considers both the simplified case in which the input string is a string of parts of speech and the more interesting case in which the parser is guided by statistical information about the particular words in the sentence

Footnote

Contribution to a special section reviewing recent research in empirical methods in speech recognition, syntactic parsing, semantic processing, information extraction and machine translation

Similar documents (content)

1. Losee, R.M.: Learning syntactic rules and tags with genetic algorithms for information retrieval and filtering : an empirical basis for grammatical rules (1996) 0.26

   0.25737876 = sum of:
     0.25737876 = product of:
       0.80430865 = sum of:
         0.0167615 = weight(abstract_txt:language in 4136) [ClassicSimilarity], result of:
           0.0167615 = score(doc=4136,freq=1.0), product of:
             0.051437985 = queryWeight, product of:
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.012332328 = queryNorm
             0.3258584 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.019690137 = weight(abstract_txt:particular in 4136) [ClassicSimilarity], result of:
           0.019690137 = score(doc=4136,freq=1.0), product of:
             0.057267454 = queryWeight, product of:
               1.0551445 = boost
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.012332328 = queryNorm
             0.34382772 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.400995 = idf(docFreq=1480, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.022206685 = weight(abstract_txt:part in 4136) [ClassicSimilarity], result of:
           0.022206685 = score(doc=4136,freq=1.0), product of:
             0.062048484 = queryWeight, product of:
               1.0983067 = boost
               4.581023 = idf(docFreq=1236, maxDocs=44421)
               0.012332328 = queryNorm
             0.35789245 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.581023 = idf(docFreq=1236, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.030094193 = weight(abstract_txt:natural in 4136) [ClassicSimilarity], result of:
           0.030094193 = score(doc=4136,freq=1.0), product of:
             0.07598546 = queryWeight, product of:
               1.2154113 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.012332328 = queryNorm
             0.396052 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.052130602 = weight(abstract_txt:parts in 4136) [ClassicSimilarity], result of:
           0.052130602 = score(doc=4136,freq=1.0), product of:
             0.109598316 = queryWeight, product of:
               1.4596877 = boost
               6.0883393 = idf(docFreq=273, maxDocs=44421)
               0.012332328 = queryNorm
             0.4756515 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0883393 = idf(docFreq=273, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.12869507 = weight(abstract_txt:syntactic in 4136) [ClassicSimilarity], result of:
           0.12869507 = score(doc=4136,freq=1.0), product of:
             0.252228 = queryWeight, product of:
               3.1316245 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.012332328 = queryNorm
             0.5102331 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.21213353 = weight(abstract_txt:speech in 4136) [ClassicSimilarity], result of:
           0.21213353 = score(doc=4136,freq=2.0), product of:
             0.2793497 = queryWeight, product of:
               3.295696 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.012332328 = queryNorm
             0.7593834 = fieldWeight in 4136, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
         0.32259694 = weight(abstract_txt:parsing in 4136) [ClassicSimilarity], result of:
           0.32259694 = score(doc=4136,freq=1.0), product of:
             0.5327898 = queryWeight, product of:
               5.5743804 = boost
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.012332328 = queryNorm
             0.6054863 = fieldWeight in 4136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.078125 = fieldNorm(doc=4136)
       0.32 = coord(8/25)
   

2. Jacquemin, C.: What is the tree that we see through the window : a linguistic approach to windowing and term variation (1996) 0.24

   0.24307303 = sum of:
     0.24307303 = product of:
       0.868118 = sum of:
         0.0201138 = weight(abstract_txt:language in 5646) [ClassicSimilarity], result of:
           0.0201138 = score(doc=5646,freq=1.0), product of:
             0.051437985 = queryWeight, product of:
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.012332328 = queryNorm
             0.39103007 = fieldWeight in 5646, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
         0.03611303 = weight(abstract_txt:natural in 5646) [ClassicSimilarity], result of:
           0.03611303 = score(doc=5646,freq=1.0), product of:
             0.07598546 = queryWeight, product of:
               1.2154113 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.012332328 = queryNorm
             0.4752624 = fieldWeight in 5646, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
         0.060224477 = weight(abstract_txt:words in 5646) [ClassicSimilarity], result of:
           0.060224477 = score(doc=5646,freq=2.0), product of:
             0.08481251 = queryWeight, product of:
               1.2840679 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.012332328 = queryNorm
             0.71008956 = fieldWeight in 5646, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
         0.15850905 = weight(abstract_txt:parser in 5646) [ClassicSimilarity], result of:
           0.15850905 = score(doc=5646,freq=1.0), product of:
             0.20369978 = queryWeight, product of:
               1.9900011 = boost
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.012332328 = queryNorm
             0.7781503 = fieldWeight in 5646, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
         0.05160727 = weight(abstract_txt:techniques in 5646) [ClassicSimilarity], result of:
           0.05160727 = score(doc=5646,freq=1.0), product of:
             0.12146145 = queryWeight, product of:
               2.1731627 = boost
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.012332328 = queryNorm
             0.424886 = fieldWeight in 5646, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
         0.15443408 = weight(abstract_txt:syntactic in 5646) [ClassicSimilarity], result of:
           0.15443408 = score(doc=5646,freq=1.0), product of:
             0.252228 = queryWeight, product of:
               3.1316245 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.012332328 = queryNorm
             0.6122797 = fieldWeight in 5646, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
         0.38711628 = weight(abstract_txt:parsing in 5646) [ClassicSimilarity], result of:
           0.38711628 = score(doc=5646,freq=1.0), product of:
             0.5327898 = queryWeight, product of:
               5.5743804 = boost
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.012332328 = queryNorm
             0.7265835 = fieldWeight in 5646, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.09375 = fieldNorm(doc=5646)
       0.28 = coord(7/25)
   

3. Chowdhury, G.G.: Natural language processing and information retrieval : pt.1: basic issues; pt.2: major applications (1991) 0.22

   0.22418046 = sum of:
     0.22418046 = product of:
       1.1209023 = sum of:
         0.058063537 = weight(abstract_txt:language in 3312) [ClassicSimilarity], result of:
           0.058063537 = score(doc=3312,freq=3.0), product of:
             0.051437985 = queryWeight, product of:
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.012332328 = queryNorm
             1.1288066 = fieldWeight in 3312, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.15625 = fieldNorm(doc=3312)
         0.056005474 = weight(abstract_txt:reviews in 3312) [ClassicSimilarity], result of:
           0.056005474 = score(doc=3312,freq=1.0), product of:
             0.07242286 = queryWeight, product of:
               1.1865768 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.012332328 = queryNorm
             0.7733121 = fieldWeight in 3312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.15625 = fieldNorm(doc=3312)
         0.10424934 = weight(abstract_txt:natural in 3312) [ClassicSimilarity], result of:
           0.10424934 = score(doc=3312,freq=3.0), product of:
             0.07598546 = queryWeight, product of:
               1.2154113 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.012332328 = queryNorm
             1.3719643 = fieldWeight in 3312, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.15625 = fieldNorm(doc=3312)
         0.25739014 = weight(abstract_txt:syntactic in 3312) [ClassicSimilarity], result of:
           0.25739014 = score(doc=3312,freq=1.0), product of:
             0.252228 = queryWeight, product of:
               3.1316245 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.012332328 = queryNorm
             1.0204662 = fieldWeight in 3312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.15625 = fieldNorm(doc=3312)
         0.6451939 = weight(abstract_txt:parsing in 3312) [ClassicSimilarity], result of:
           0.6451939 = score(doc=3312,freq=1.0), product of:
             0.5327898 = queryWeight, product of:
               5.5743804 = boost
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.012332328 = queryNorm
             1.2109725 = fieldWeight in 3312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.15625 = fieldNorm(doc=3312)
       0.2 = coord(5/25)
   

4. Multilingual information management : current levels and future abilities. A report Commissioned by the US National Science Foundation and also delivered to the European Commission's Language Engineering Office and the US Defense Advanced Research Projects Agency, April 1999 (1999) 0.22

   0.21544646 = sum of:
     0.21544646 = product of:
       0.7694516 = sum of:
         0.023466099 = weight(abstract_txt:language in 68) [ClassicSimilarity], result of:
           0.023466099 = score(doc=68,freq=4.0), product of:
             0.051437985 = queryWeight, product of:
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.012332328 = queryNorm
             0.45620176 = fieldWeight in 68, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.021065935 = weight(abstract_txt:natural in 68) [ClassicSimilarity], result of:
           0.021065935 = score(doc=68,freq=1.0), product of:
             0.07598546 = queryWeight, product of:
               1.2154113 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.012332328 = queryNorm
             0.2772364 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.0120002385 = weight(abstract_txt:which in 68) [ClassicSimilarity], result of:
           0.0120002385 = score(doc=68,freq=1.0), product of:
             0.07530844 = queryWeight, product of:
               2.0957549 = boost
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.012332328 = queryNorm
             0.15934785 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9137893 = idf(docFreq=6552, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.07374003 = weight(abstract_txt:techniques in 68) [ClassicSimilarity], result of:
           0.07374003 = score(doc=68,freq=6.0), product of:
             0.12146145 = queryWeight, product of:
               2.1731627 = boost
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.012332328 = queryNorm
             0.60710645 = fieldWeight in 68, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.5321174 = idf(docFreq=1298, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.18186663 = weight(abstract_txt:speech in 68) [ClassicSimilarity], result of:
           0.18186663 = score(doc=68,freq=3.0), product of:
             0.2793497 = queryWeight, product of:
               3.295696 = boost
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.012332328 = queryNorm
             0.65103567 = fieldWeight in 68, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.8731537 = idf(docFreq=124, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.31935465 = weight(abstract_txt:parsing in 68) [ClassicSimilarity], result of:
           0.31935465 = score(doc=68,freq=2.0), product of:
             0.5327898 = queryWeight, product of:
               5.5743804 = boost
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.012332328 = queryNorm
             0.5994008 = fieldWeight in 68, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
         0.13795805 = weight(abstract_txt:statistical in 68) [ClassicSimilarity], result of:
           0.13795805 = score(doc=68,freq=1.0), product of:
             0.45481142 = queryWeight, product of:
               6.6490364 = boost
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.012332328 = queryNorm
             0.3033302 = fieldWeight in 68, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.0546875 = fieldNorm(doc=68)
       0.28 = coord(7/25)
   

5. Sikkel, K.: Parsing schemata : a framework for specification and analysis of parsing algorithms (1996) 0.21

   0.21465565 = sum of:
     0.21465565 = product of:
       0.89439857 = sum of:
         0.0167615 = weight(abstract_txt:language in 1685) [ClassicSimilarity], result of:
           0.0167615 = score(doc=1685,freq=1.0), product of:
             0.051437985 = queryWeight, product of:
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.012332328 = queryNorm
             0.3258584 = fieldWeight in 1685, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.078125 = fieldNorm(doc=1685)
         0.028002737 = weight(abstract_txt:reviews in 1685) [ClassicSimilarity], result of:
           0.028002737 = score(doc=1685,freq=1.0), product of:
             0.07242286 = queryWeight, product of:
               1.1865768 = boost
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.012332328 = queryNorm
             0.38665605 = fieldWeight in 1685, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9491973 = idf(docFreq=855, maxDocs=44421)
               0.078125 = fieldNorm(doc=1685)
         0.030094193 = weight(abstract_txt:natural in 1685) [ClassicSimilarity], result of:
           0.030094193 = score(doc=1685,freq=1.0), product of:
             0.07598546 = queryWeight, product of:
               1.2154113 = boost
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.012332328 = queryNorm
             0.396052 = fieldWeight in 1685, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0694656 = idf(docFreq=758, maxDocs=44421)
               0.078125 = fieldNorm(doc=1685)
         0.13209088 = weight(abstract_txt:parser in 1685) [ClassicSimilarity], result of:
           0.13209088 = score(doc=1685,freq=1.0), product of:
             0.20369978 = queryWeight, product of:
               1.9900011 = boost
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.012332328 = queryNorm
             0.6484586 = fieldWeight in 1685, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.078125 = fieldNorm(doc=1685)
         0.12869507 = weight(abstract_txt:syntactic in 1685) [ClassicSimilarity], result of:
           0.12869507 = score(doc=1685,freq=1.0), product of:
             0.252228 = queryWeight, product of:
               3.1316245 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.012332328 = queryNorm
             0.5102331 = fieldWeight in 1685, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.078125 = fieldNorm(doc=1685)
         0.5587542 = weight(abstract_txt:parsing in 1685) [ClassicSimilarity], result of:
           0.5587542 = score(doc=1685,freq=3.0), product of:
             0.5327898 = queryWeight, product of:
               5.5743804 = boost
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.012332328 = queryNorm
             1.0487329 = fieldWeight in 1685, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.750224 = idf(docFreq=51, maxDocs=44421)
               0.078125 = fieldNorm(doc=1685)
       0.24 = coord(6/25)