Document (#35007)

Author

Malak, P.

Title

Is the Artificial Intelligence applicable for the libraries purposes?

Source

Librarianship in the information age: Proceedings of the 13th BOBCATSSS Symposium, 31 January - 2 February 2005 in Budapest, Hungary. Eds.: Marte Langeland u.a

Imprint

Budapest : ELTE

Year

2005

Pages

S.232-237

Abstract

Artificial Intelligence (well introduced in the "Matrix" movie) is a high level tool and technology. Sophisticated methods, genetics algorithms, neural networks, self-learning systems, and so on, are the leading fields of the AI research. What are potential relations between very classical science and service - libraries and very futuristic one - AI? If we consider library as a service - still developing one, we can use AI achievements for improving the service. Some parts of Al research are concerned with texts analyses (neural networks), documents classification - either for information classification, as well as for (mostly) user information needs meeting in searching systems. Another possibility is self-learning systems designed for information value estimation or documents abstracts or summaries automatic preparation. Combining Al tools and methods and linguistic rules and principles we can design a virtual librarian - searching agent, that works not only with the contents of documents (as we can meet already), but also, and even mainly with the information as it is - evaluating its value and assigning document as valuable or not for further use. One of used linguistic tools - a very simple one - is frequencies, and their distribution within certain natural language. The basic frequencies computation is a sufficient base for further, more extensive linguistic studies. Knowing the current frequencies of the signals (words) in the document one can evaluate the probability of particular word occurrence and, basing on the probabilities, evaluate entropy. The entropy calculation for the document allows the information value estimation. The relations between number of words and their frequencies, from one side, and the entropy on the other side, for different scientific, web available, documents have been studied. The research resulted with some interesting observations and facts concerning contemporary scientific language, and information accessibility on the Internet. The results can be useful for intelligent information retrieval systems, enabling them to proper fulfilling and meeting user information needs, and strict, proper user queries answer, as well as documents acquisition for research libraries. For each discipline there are different words being used in documents. This concern especially so-called characteristic vocabulary, i.e. the frequent, but limited in occurrence to a specific type of publications entries (e.g.: technical, biological, mathematical, sociological, and so on, terms).

Similar documents (content)

1. Lan, K.C.; Ho, K.S.; Luk, R.W.P.; Leong, H.V.: Dialogue act recognition using maximum entropy (2008) 0.15

   0.15079603 = sum of:
     0.15079603 = product of:
       0.62831676 = sum of:
         0.0826859 = weight(abstract_txt:occurrence in 2717) [ClassicSimilarity], result of:
           0.0826859 = score(doc=2717,freq=1.0), product of:
             0.19550402 = queryWeight, product of:
               1.2256154 = boost
               6.7669935 = idf(docFreq=138, maxDocs=44421)
               0.023572505 = queryNorm
             0.4229371 = fieldWeight in 2717, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7669935 = idf(docFreq=138, maxDocs=44421)
               0.0625 = fieldNorm(doc=2717)
         0.021182938 = weight(abstract_txt:systems in 2717) [ClassicSimilarity], result of:
           0.021182938 = score(doc=2717,freq=1.0), product of:
             0.09935784 = queryWeight, product of:
               1.2356414 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.023572505 = queryNorm
             0.21319844 = fieldWeight in 2717, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=2717)
         0.06752366 = weight(abstract_txt:very in 2717) [ClassicSimilarity], result of:
           0.06752366 = score(doc=2717,freq=2.0), product of:
             0.15518835 = queryWeight, product of:
               1.3373699 = boost
               4.922683 = idf(docFreq=878, maxDocs=44421)
               0.023572505 = queryNorm
             0.4351078 = fieldWeight in 2717, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.922683 = idf(docFreq=878, maxDocs=44421)
               0.0625 = fieldNorm(doc=2717)
         0.021363515 = weight(abstract_txt:information in 2717) [ClassicSimilarity], result of:
           0.021363515 = score(doc=2717,freq=2.0), product of:
             0.09992171 = queryWeight, product of:
               1.7524122 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023572505 = queryNorm
             0.21380253 = fieldWeight in 2717, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=2717)
         0.20216043 = weight(abstract_txt:entropy in 2717) [ClassicSimilarity], result of:
           0.20216043 = score(doc=2717,freq=1.0), product of:
             0.40615883 = queryWeight, product of:
               2.163565 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.023572505 = queryNorm
             0.49773738 = fieldWeight in 2717, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.0625 = fieldNorm(doc=2717)
         0.23340033 = weight(abstract_txt:frequencies in 2717) [ClassicSimilarity], result of:
           0.23340033 = score(doc=2717,freq=1.0), product of:
             0.49197802 = queryWeight, product of:
               2.7495663 = boost
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.023572505 = queryNorm
             0.4744121 = fieldWeight in 2717, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.0625 = fieldNorm(doc=2717)
       0.24 = coord(6/25)
   

2. Wang, F.L.; Yang, C.C.: Mining Web data for Chinese segmentation (2007) 0.15

   0.14965336 = sum of:
     0.14965336 = product of:
       0.62355566 = sum of:
         0.1053977 = weight(abstract_txt:proper in 1604) [ClassicSimilarity], result of:
           0.1053977 = score(doc=1604,freq=2.0), product of:
             0.18242267 = queryWeight, product of:
               1.183902 = boost
               6.5366817 = idf(docFreq=174, maxDocs=44421)
               0.023572505 = queryNorm
             0.5777665 = fieldWeight in 1604, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.5366817 = idf(docFreq=174, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.021182938 = weight(abstract_txt:systems in 1604) [ClassicSimilarity], result of:
           0.021182938 = score(doc=1604,freq=1.0), product of:
             0.09935784 = queryWeight, product of:
               1.2356414 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.023572505 = queryNorm
             0.21319844 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.12298328 = weight(abstract_txt:words in 1604) [ClassicSimilarity], result of:
           0.12298328 = score(doc=1604,freq=4.0), product of:
             0.18370001 = queryWeight, product of:
               1.4550455 = boost
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.023572505 = queryNorm
             0.6694789 = fieldWeight in 1604, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.355831 = idf(docFreq=569, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.015106286 = weight(abstract_txt:information in 1604) [ClassicSimilarity], result of:
           0.015106286 = score(doc=1604,freq=1.0), product of:
             0.09992171 = queryWeight, product of:
               1.7524122 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023572505 = queryNorm
             0.15118122 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.12548514 = weight(abstract_txt:documents in 1604) [ClassicSimilarity], result of:
           0.12548514 = score(doc=1604,freq=5.0), product of:
             0.21776094 = queryWeight, product of:
               2.2404072 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.023572505 = queryNorm
             0.5762518 = fieldWeight in 1604, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
         0.23340033 = weight(abstract_txt:frequencies in 1604) [ClassicSimilarity], result of:
           0.23340033 = score(doc=1604,freq=1.0), product of:
             0.49197802 = queryWeight, product of:
               2.7495663 = boost
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.023572505 = queryNorm
             0.4744121 = fieldWeight in 1604, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.590594 = idf(docFreq=60, maxDocs=44421)
               0.0625 = fieldNorm(doc=1604)
       0.24 = coord(6/25)
   

3. Deerwester, S.; Dumais, S.; Landauer, T.; Furnass, G.; Beck, L.: Improving information retrieval with latent semantic indexing (1988) 0.15

   0.14861669 = sum of:
     0.14861669 = product of:
       0.61923623 = sum of:
         0.06822128 = weight(abstract_txt:relations in 3396) [ClassicSimilarity], result of:
           0.06822128 = score(doc=3396,freq=1.0), product of:
             0.13124631 = queryWeight, product of:
               1.0041993 = boost
               5.5444884 = idf(docFreq=471, maxDocs=44421)
               0.023572505 = queryNorm
             0.5197958 = fieldWeight in 3396, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5444884 = idf(docFreq=471, maxDocs=44421)
               0.09375 = fieldNorm(doc=3396)
         0.047539905 = weight(abstract_txt:document in 3396) [ClassicSimilarity], result of:
           0.047539905 = score(doc=3396,freq=1.0), product of:
             0.1180891 = queryWeight, product of:
               1.1666132 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.023572505 = queryNorm
             0.40257657 = fieldWeight in 3396, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.09375 = fieldNorm(doc=3396)
         0.031774405 = weight(abstract_txt:systems in 3396) [ClassicSimilarity], result of:
           0.031774405 = score(doc=3396,freq=1.0), product of:
             0.09935784 = queryWeight, product of:
               1.2356414 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.023572505 = queryNorm
             0.31979766 = fieldWeight in 3396, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.09375 = fieldNorm(doc=3396)
         0.02265943 = weight(abstract_txt:information in 3396) [ClassicSimilarity], result of:
           0.02265943 = score(doc=3396,freq=1.0), product of:
             0.09992171 = queryWeight, product of:
               1.7524122 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023572505 = queryNorm
             0.22677183 = fieldWeight in 3396, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.09375 = fieldNorm(doc=3396)
         0.30324066 = weight(abstract_txt:entropy in 3396) [ClassicSimilarity], result of:
           0.30324066 = score(doc=3396,freq=1.0), product of:
             0.40615883 = queryWeight, product of:
               2.163565 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.023572505 = queryNorm
             0.74660605 = fieldWeight in 3396, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.09375 = fieldNorm(doc=3396)
         0.14580053 = weight(abstract_txt:documents in 3396) [ClassicSimilarity], result of:
           0.14580053 = score(doc=3396,freq=3.0), product of:
             0.21776094 = queryWeight, product of:
               2.2404072 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.023572505 = queryNorm
             0.66954404 = fieldWeight in 3396, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.09375 = fieldNorm(doc=3396)
       0.24 = coord(6/25)
   

4. Hutchins, W.J.; Somers, H.L.: ¬An introduction to machine translation (1992) 0.15

   0.1480184 = sum of:
     0.1480184 = product of:
       0.5286372 = sum of:
         0.11229001 = weight(abstract_txt:intelligence in 5512) [ClassicSimilarity], result of:
           0.11229001 = score(doc=5512,freq=2.0), product of:
             0.1452194 = queryWeight, product of:
               1.0563034 = boost
               5.8321705 = idf(docFreq=353, maxDocs=44421)
               0.023572505 = queryNorm
             0.77324384 = fieldWeight in 5512, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8321705 = idf(docFreq=353, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
         0.035683453 = weight(abstract_txt:well in 5512) [ClassicSimilarity], result of:
           0.035683453 = score(doc=5512,freq=1.0), product of:
             0.09753239 = queryWeight, product of:
               1.0602211 = boost
               3.9025342 = idf(docFreq=2437, maxDocs=44421)
               0.023572505 = queryNorm
             0.36586258 = fieldWeight in 5512, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9025342 = idf(docFreq=2437, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
         0.12485026 = weight(abstract_txt:artificial in 5512) [ClassicSimilarity], result of:
           0.12485026 = score(doc=5512,freq=2.0), product of:
             0.155856 = queryWeight, product of:
               1.0943043 = boost
               6.0419855 = idf(docFreq=286, maxDocs=44421)
               0.023572505 = queryNorm
             0.80106163 = fieldWeight in 5512, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.0419855 = idf(docFreq=286, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
         0.02524959 = weight(abstract_txt:research in 5512) [ClassicSimilarity], result of:
           0.02524959 = score(doc=5512,freq=1.0), product of:
             0.08524196 = queryWeight, product of:
               1.144506 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.023572505 = queryNorm
             0.2962108 = fieldWeight in 5512, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
         0.031774405 = weight(abstract_txt:systems in 5512) [ClassicSimilarity], result of:
           0.031774405 = score(doc=5512,freq=1.0), product of:
             0.09935784 = queryWeight, product of:
               1.2356414 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.023572505 = queryNorm
             0.31979766 = fieldWeight in 5512, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
         0.16674419 = weight(abstract_txt:linguistic in 5512) [ClassicSimilarity], result of:
           0.16674419 = score(doc=5512,freq=2.0), product of:
             0.21636887 = queryWeight, product of:
               1.5791354 = boost
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.023572505 = queryNorm
             0.77064776 = fieldWeight in 5512, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8125896 = idf(docFreq=360, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
         0.03204527 = weight(abstract_txt:information in 5512) [ClassicSimilarity], result of:
           0.03204527 = score(doc=5512,freq=2.0), product of:
             0.09992171 = queryWeight, product of:
               1.7524122 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023572505 = queryNorm
             0.3207038 = fieldWeight in 5512, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.09375 = fieldNorm(doc=5512)
       0.28 = coord(7/25)
   

5. Saarikoski, J.; Laurikkala, J.; Järvelin, K.; Juhola, M.: ¬A study of the use of self-organising maps in information retrieval (2009) 0.14

   0.14081267 = sum of:
     0.14081267 = product of:
       0.5029024 = sum of:
         0.11153279 = weight(abstract_txt:self in 3836) [ClassicSimilarity], result of:
           0.11153279 = score(doc=3836,freq=6.0), product of:
             0.13134675 = queryWeight, product of:
               1.0045835 = boost
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.023572505 = queryNorm
             0.84914774 = fieldWeight in 3836, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.5466094 = idf(docFreq=470, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
         0.01683306 = weight(abstract_txt:research in 3836) [ClassicSimilarity], result of:
           0.01683306 = score(doc=3836,freq=1.0), product of:
             0.08524196 = queryWeight, product of:
               1.144506 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.023572505 = queryNorm
             0.19747387 = fieldWeight in 3836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
         0.07763234 = weight(abstract_txt:document in 3836) [ClassicSimilarity], result of:
           0.07763234 = score(doc=3836,freq=6.0), product of:
             0.1180891 = queryWeight, product of:
               1.1666132 = boost
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.023572505 = queryNorm
             0.6574048 = fieldWeight in 3836, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.29415 = idf(docFreq=1647, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
         0.033426926 = weight(abstract_txt:value in 3836) [ClassicSimilarity], result of:
           0.033426926 = score(doc=3836,freq=1.0), product of:
             0.122357175 = queryWeight, product of:
               1.1875085 = boost
               4.3710623 = idf(docFreq=1525, maxDocs=44421)
               0.023572505 = queryNorm
             0.2731914 = fieldWeight in 3836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.3710623 = idf(docFreq=1525, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
         0.088836424 = weight(abstract_txt:neural in 3836) [ClassicSimilarity], result of:
           0.088836424 = score(doc=3836,freq=1.0), product of:
             0.20508258 = queryWeight, product of:
               1.2552804 = boost
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.023572505 = queryNorm
             0.43317392 = fieldWeight in 3836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.930783 = idf(docFreq=117, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
         0.026164854 = weight(abstract_txt:information in 3836) [ClassicSimilarity], result of:
           0.026164854 = score(doc=3836,freq=3.0), product of:
             0.09992171 = queryWeight, product of:
               1.7524122 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.023572505 = queryNorm
             0.26185355 = fieldWeight in 3836, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
         0.148476 = weight(abstract_txt:documents in 3836) [ClassicSimilarity], result of:
           0.148476 = score(doc=3836,freq=7.0), product of:
             0.21776094 = queryWeight, product of:
               2.2404072 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.023572505 = queryNorm
             0.6818303 = fieldWeight in 3836, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=3836)
       0.28 = coord(7/25)