Document (#35458)

Author

Al-Shawakfa, E.
Al-Badarneh, A.
Shatnawi, S.
Al-Rabab'ah, K.
Bani-Ismail, B.

Title

¬A comparison study of some Arabic root finding algorithms

Source

Journal of the American Society for Information Science and Technology. 61(2010) no.5, S.1015-1024

Year

2010

Abstract

Arabic has a complex structure, which makes it difficult to apply natural language processing (NLP). Much research on Arabic NLP (ANLP) does exist; however, it is not as mature as that of other languages. Finding Arabic roots is an important step toward conducting effective research on most of ANLP applications. The authors have studied and compared six root-finding algorithms with success rates of over 90%. All algorithms of this study did not use the same testing corpus and/or benchmarking measures. They unified the testing process by implementing their own algorithm descriptions and building a corpus out of 3823 triliteral roots, applying 73 triliteral patterns, and with 18 affixes, producing around 27.6 million words. They tested the algorithms with the generated corpus and have obtained interesting results; they offer to share the corpus freely for benchmarking and ANLP research.

Theme

Computerlinguistik

Similar documents (content)

1. Hmeidi, I.I.; Al-Shalabi, R.F.; Al-Taani, A.T.; Najadat, H.; Al-Hazaimeh, S.A.: ¬A novel approach to the extraction of roots from Arabic words using bigrams (2010) 0.40

   0.39620346 = sum of:
     0.39620346 = product of:
       1.2381358 = sum of:
         0.009425823 = weight(abstract_txt:have in 413) [ClassicSimilarity], result of:
           0.009425823 = score(doc=413,freq=1.0), product of:
             0.04713813 = queryWeight, product of:
               1.0528718 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.013993615 = queryNorm
             0.19996175 = fieldWeight in 413, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.016213665 = weight(abstract_txt:study in 413) [ClassicSimilarity], result of:
           0.016213665 = score(doc=413,freq=2.0), product of:
             0.053711873 = queryWeight, product of:
               1.1238917 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.013993615 = queryNorm
             0.3018637 = fieldWeight in 413, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.011629931 = weight(abstract_txt:with in 413) [ClassicSimilarity], result of:
           0.011629931 = score(doc=413,freq=3.0), product of:
             0.043039534 = queryWeight, product of:
               1.2321646 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.013993615 = queryNorm
             0.27021506 = fieldWeight in 413, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.21525696 = weight(abstract_txt:roots in 413) [ClassicSimilarity], result of:
           0.21525696 = score(doc=413,freq=3.0), product of:
             0.26308164 = queryWeight, product of:
               2.4873369 = boost
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.013993615 = queryNorm
             0.8182135 = fieldWeight in 413, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.14537148 = weight(abstract_txt:root in 413) [ClassicSimilarity], result of:
           0.14537148 = score(doc=413,freq=1.0), product of:
             0.29206464 = queryWeight, product of:
               2.6207695 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013993615 = queryNorm
             0.49773738 = fieldWeight in 413, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.15076485 = weight(abstract_txt:algorithms in 413) [ClassicSimilarity], result of:
           0.15076485 = score(doc=413,freq=2.0), product of:
             0.29924455 = queryWeight, product of:
               3.751608 = boost
               5.7000527 = idf(docFreq=403, maxDocs=44421)
               0.013993615 = queryNorm
             0.5038182 = fieldWeight in 413, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7000527 = idf(docFreq=403, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.13014495 = weight(abstract_txt:corpus in 413) [ClassicSimilarity], result of:
           0.13014495 = score(doc=413,freq=1.0), product of:
             0.3418123 = queryWeight, product of:
               4.0095735 = boost
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.013993615 = queryNorm
             0.38074973 = fieldWeight in 413, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
         0.55932826 = weight(abstract_txt:arabic in 413) [ClassicSimilarity], result of:
           0.55932826 = score(doc=413,freq=5.0), product of:
             0.5283933 = queryWeight, product of:
               4.9852047 = boost
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.013993615 = queryNorm
             1.0585452 = fieldWeight in 413, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.0625 = fieldNorm(doc=413)
       0.32 = coord(8/25)
   

2. Kanaan, G.; Al-Shalabi, R.; Ghwanmeh, S.; Al-Ma'adeed, H.: ¬A comparison of text-classification techniques applied to Arabic text (2009) 0.26

   0.2606766 = sum of:
     0.2606766 = product of:
       1.303383 = sum of:
         0.014138733 = weight(abstract_txt:have in 83) [ClassicSimilarity], result of:
           0.014138733 = score(doc=83,freq=1.0), product of:
             0.04713813 = queryWeight, product of:
               1.0528718 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.013993615 = queryNorm
             0.2999426 = fieldWeight in 83, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.09375 = fieldNorm(doc=83)
         0.028887162 = weight(abstract_txt:research in 83) [ClassicSimilarity], result of:
           0.028887162 = score(doc=83,freq=2.0), product of:
             0.068958685 = queryWeight, product of:
               1.5596583 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013993615 = queryNorm
             0.41890535 = fieldWeight in 83, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.09375 = fieldNorm(doc=83)
         0.22614728 = weight(abstract_txt:algorithms in 83) [ClassicSimilarity], result of:
           0.22614728 = score(doc=83,freq=2.0), product of:
             0.29924455 = queryWeight, product of:
               3.751608 = boost
               5.7000527 = idf(docFreq=403, maxDocs=44421)
               0.013993615 = queryNorm
             0.7557273 = fieldWeight in 83, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7000527 = idf(docFreq=403, maxDocs=44421)
               0.09375 = fieldNorm(doc=83)
         0.19521743 = weight(abstract_txt:corpus in 83) [ClassicSimilarity], result of:
           0.19521743 = score(doc=83,freq=1.0), product of:
             0.3418123 = queryWeight, product of:
               4.0095735 = boost
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.013993615 = queryNorm
             0.5711246 = fieldWeight in 83, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.09375 = fieldNorm(doc=83)
         0.8389924 = weight(abstract_txt:arabic in 83) [ClassicSimilarity], result of:
           0.8389924 = score(doc=83,freq=5.0), product of:
             0.5283933 = queryWeight, product of:
               4.9852047 = boost
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.013993615 = queryNorm
             1.5878179 = fieldWeight in 83, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.09375 = fieldNorm(doc=83)
       0.2 = coord(5/25)
   

3. Hmeidi, I.; Kanaan, G.; Evens, M.: Design and implementation of automatic indexing for information retrieval with Arabic documents (1997) 0.19

   0.19129382 = sum of:
     0.19129382 = product of:
       0.95646906 = sum of:
         0.014537413 = weight(abstract_txt:with in 2660) [ClassicSimilarity], result of:
           0.014537413 = score(doc=2660,freq=3.0), product of:
             0.043039534 = queryWeight, product of:
               1.2321646 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.013993615 = queryNorm
             0.33776882 = fieldWeight in 2660, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.078125 = fieldNorm(doc=2660)
         0.15534833 = weight(abstract_txt:roots in 2660) [ClassicSimilarity], result of:
           0.15534833 = score(doc=2660,freq=1.0), product of:
             0.26308164 = queryWeight, product of:
               2.4873369 = boost
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.013993615 = queryNorm
             0.59049475 = fieldWeight in 2660, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.078125 = fieldNorm(doc=2660)
         0.18171436 = weight(abstract_txt:root in 2660) [ClassicSimilarity], result of:
           0.18171436 = score(doc=2660,freq=1.0), product of:
             0.29206464 = queryWeight, product of:
               2.6207695 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013993615 = queryNorm
             0.6221717 = fieldWeight in 2660, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.078125 = fieldNorm(doc=2660)
         0.16268118 = weight(abstract_txt:corpus in 2660) [ClassicSimilarity], result of:
           0.16268118 = score(doc=2660,freq=1.0), product of:
             0.3418123 = queryWeight, product of:
               4.0095735 = boost
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.013993615 = queryNorm
             0.47593716 = fieldWeight in 2660, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.078125 = fieldNorm(doc=2660)
         0.4421878 = weight(abstract_txt:arabic in 2660) [ClassicSimilarity], result of:
           0.4421878 = score(doc=2660,freq=2.0), product of:
             0.5283933 = queryWeight, product of:
               4.9852047 = boost
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.013993615 = queryNorm
             0.83685344 = fieldWeight in 2660, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.078125 = fieldNorm(doc=2660)
       0.2 = coord(5/25)
   

4. Abdelali, A.: Localization in modern standard Arabic (2004) 0.18

   0.18094097 = sum of:
     0.18094097 = product of:
       0.90470487 = sum of:
         0.016662657 = weight(abstract_txt:have in 3066) [ClassicSimilarity], result of:
           0.016662657 = score(doc=3066,freq=2.0), product of:
             0.04713813 = queryWeight, product of:
               1.0528718 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.013993615 = queryNorm
             0.35348576 = fieldWeight in 3066, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.078125 = fieldNorm(doc=3066)
         0.014330992 = weight(abstract_txt:study in 3066) [ClassicSimilarity], result of:
           0.014330992 = score(doc=3066,freq=1.0), product of:
             0.053711873 = queryWeight, product of:
               1.1238917 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.013993615 = queryNorm
             0.26681235 = fieldWeight in 3066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.078125 = fieldNorm(doc=3066)
         0.011869748 = weight(abstract_txt:with in 3066) [ClassicSimilarity], result of:
           0.011869748 = score(doc=3066,freq=2.0), product of:
             0.043039534 = queryWeight, product of:
               1.2321646 = boost
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.013993615 = queryNorm
             0.2757871 = fieldWeight in 3066, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4961398 = idf(docFreq=9949, maxDocs=44421)
               0.078125 = fieldNorm(doc=3066)
         0.16268118 = weight(abstract_txt:corpus in 3066) [ClassicSimilarity], result of:
           0.16268118 = score(doc=3066,freq=1.0), product of:
             0.3418123 = queryWeight, product of:
               4.0095735 = boost
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.013993615 = queryNorm
             0.47593716 = fieldWeight in 3066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.078125 = fieldNorm(doc=3066)
         0.6991603 = weight(abstract_txt:arabic in 3066) [ClassicSimilarity], result of:
           0.6991603 = score(doc=3066,freq=5.0), product of:
             0.5283933 = queryWeight, product of:
               4.9852047 = boost
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.013993615 = queryNorm
             1.3231815 = fieldWeight in 3066, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.078125 = fieldNorm(doc=3066)
       0.2 = coord(5/25)
   

5. Rushdi-Saleh, M.; Martín-Valdivia, M.T.; Ureña-López, L.A.; Perea-Ortega, J.M.: OCA: Opinion corpus for Arabic (2011) 0.18

   0.17861466 = sum of:
     0.17861466 = product of:
       0.8930733 = sum of:
         0.011782278 = weight(abstract_txt:have in 360) [ClassicSimilarity], result of:
           0.011782278 = score(doc=360,freq=1.0), product of:
             0.04713813 = queryWeight, product of:
               1.0528718 = boost
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.013993615 = queryNorm
             0.2499522 = fieldWeight in 360, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.199388 = idf(docFreq=4924, maxDocs=44421)
               0.078125 = fieldNorm(doc=360)
         0.024072634 = weight(abstract_txt:research in 360) [ClassicSimilarity], result of:
           0.024072634 = score(doc=360,freq=2.0), product of:
             0.068958685 = queryWeight, product of:
               1.5596583 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013993615 = queryNorm
             0.34908777 = fieldWeight in 360, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.078125 = fieldNorm(doc=360)
         0.13325857 = weight(abstract_txt:algorithms in 360) [ClassicSimilarity], result of:
           0.13325857 = score(doc=360,freq=1.0), product of:
             0.29924455 = queryWeight, product of:
               3.751608 = boost
               5.7000527 = idf(docFreq=403, maxDocs=44421)
               0.013993615 = queryNorm
             0.4453166 = fieldWeight in 360, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7000527 = idf(docFreq=403, maxDocs=44421)
               0.078125 = fieldNorm(doc=360)
         0.28177208 = weight(abstract_txt:corpus in 360) [ClassicSimilarity], result of:
           0.28177208 = score(doc=360,freq=3.0), product of:
             0.3418123 = queryWeight, product of:
               4.0095735 = boost
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.013993615 = queryNorm
             0.8243474 = fieldWeight in 360, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.0919957 = idf(docFreq=272, maxDocs=44421)
               0.078125 = fieldNorm(doc=360)
         0.4421878 = weight(abstract_txt:arabic in 360) [ClassicSimilarity], result of:
           0.4421878 = score(doc=360,freq=2.0), product of:
             0.5283933 = queryWeight, product of:
               4.9852047 = boost
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.013993615 = queryNorm
             0.83685344 = fieldWeight in 360, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.574333 = idf(docFreq=61, maxDocs=44421)
               0.078125 = fieldNorm(doc=360)
       0.2 = coord(5/25)