Document (#18872)

Author

Chen, H.
Chung, Y.-M.
Ramsey, M.
Yang, C.C.

Title

¬A smart itsy bitsy spider for the Web

Source

Journal of the American Society for Information Science. 49(1998) no.7, S.604-618

Year

1998

Abstract

As part of the ongoing Illinois Digital Library Initiative project, this research proposes an intelligent agent approach to Web searching. In this experiment, we developed 2 Web personal spiders based on best first search and genetic algorithm techniques, respectively. These personal spiders can dynamically take a user's selected starting homepages and search for the most closely related homepages in the Web, based on the links and keyword indexing. A graphical, dynamic, Jav-based interface was developed and is available for Web access. A system architecture for implementing such an agent-spider is presented, followed by deteiled discussions of benchmark testing and user evaluation results. In benchmark testing, although the genetic algorithm spider did not outperform the best first search spider, we found both results to be comparable and complementary. In user evaluation, the genetic algorithm spider obtained significantly higher recall value than that of the best first search spider. However, their precision values were not statistically different. The mutation process introduced in genetic algorithms allows users to find other potential relevant homepages that cannot be explored via a conventional local search process. In addition, we found the Java-based interface to be a necessary component for design of a truly interactive and dynamic Web agent

Theme

Web-Agenten

Similar documents (author)

1. Yang, C.C.; Chung, A.: ¬A personal agent for Chinese financial news on the Web (2002) 3.30

   3.30234 = sum of:
     3.30234 = product of:
       4.95351 = sum of:
         1.9950042 = weight(author_txt:yang in 1205) [ClassicSimilarity], result of:
           1.9950042 = score(doc=1205,freq=1.0), product of:
             0.5565744 = queryWeight, product of:
               1.1681489 = boost
               7.168868 = idf(docFreq=92, maxDocs=44421)
               0.06646216 = queryNorm
             3.584434 = fieldWeight in 1205, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.168868 = idf(docFreq=92, maxDocs=44421)
               0.5 = fieldNorm(doc=1205)
         2.9585056 = weight(author_txt:chung in 1205) [ClassicSimilarity], result of:
           2.9585056 = score(doc=1205,freq=1.0), product of:
             0.72378397 = queryWeight, product of:
               1.332113 = boost
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.06646216 = queryNorm
             4.0875535 = fieldWeight in 1205, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.5 = fieldNorm(doc=1205)
       0.6666667 = coord(2/3)
   

2. Chung, W.; Chen, H.: Browsing the underdeveloped Web : an experiment on the Arabic Medical Web Directory (2009) 2.81

   2.806706 = sum of:
     2.806706 = product of:
       4.2100587 = sum of:
         1.2515529 = weight(author_txt:chen in 3733) [ClassicSimilarity], result of:
           1.2515529 = score(doc=3733,freq=1.0), product of:
             0.40787476 = queryWeight, product of:
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.06646216 = queryNorm
             3.0684736 = fieldWeight in 3733, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.5 = fieldNorm(doc=3733)
         2.9585056 = weight(author_txt:chung in 3733) [ClassicSimilarity], result of:
           2.9585056 = score(doc=3733,freq=1.0), product of:
             0.72378397 = queryWeight, product of:
               1.332113 = boost
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.06646216 = queryNorm
             4.0875535 = fieldWeight in 3733, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.5 = fieldNorm(doc=3733)
       0.6666667 = coord(2/3)
   

3. Huang, Z.; Chung, Z.W.; Chen, H.: ¬A graph model for e-commerce recommender systems (2004) 2.11

   2.1050296 = sum of:
     2.1050296 = product of:
       3.1575441 = sum of:
         0.9386647 = weight(author_txt:chen in 1501) [ClassicSimilarity], result of:
           0.9386647 = score(doc=1501,freq=1.0), product of:
             0.40787476 = queryWeight, product of:
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.06646216 = queryNorm
             2.3013551 = fieldWeight in 1501, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.375 = fieldNorm(doc=1501)
         2.2188795 = weight(author_txt:chung in 1501) [ClassicSimilarity], result of:
           2.2188795 = score(doc=1501,freq=1.0), product of:
             0.72378397 = queryWeight, product of:
               1.332113 = boost
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.06646216 = queryNorm
             3.0656652 = fieldWeight in 1501, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.375 = fieldNorm(doc=1501)
       0.6666667 = coord(2/3)
   

4. Chung, W.; Chen, H.; Reid, E.: Business stakeholder analyzer : an experiment of classifying stakeholders on the Web (2009) 2.11

   2.1050296 = sum of:
     2.1050296 = product of:
       3.1575441 = sum of:
         0.9386647 = weight(author_txt:chen in 3699) [ClassicSimilarity], result of:
           0.9386647 = score(doc=3699,freq=1.0), product of:
             0.40787476 = queryWeight, product of:
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.06646216 = queryNorm
             2.3013551 = fieldWeight in 3699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.375 = fieldNorm(doc=3699)
         2.2188795 = weight(author_txt:chung in 3699) [ClassicSimilarity], result of:
           2.2188795 = score(doc=3699,freq=1.0), product of:
             0.72378397 = queryWeight, product of:
               1.332113 = boost
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.06646216 = queryNorm
             3.0656652 = fieldWeight in 3699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.375 = fieldNorm(doc=3699)
       0.6666667 = coord(2/3)
   

5. Cho, H.; Chen, M.-H.; Chung, S.: Testing an integrative theoretical model of knowledge-sharing behavior in the context of Wikipedia (2010) 2.11

   2.1050296 = sum of:
     2.1050296 = product of:
       3.1575441 = sum of:
         0.9386647 = weight(author_txt:chen in 447) [ClassicSimilarity], result of:
           0.9386647 = score(doc=447,freq=1.0), product of:
             0.40787476 = queryWeight, product of:
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.06646216 = queryNorm
             2.3013551 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.136947 = idf(docFreq=260, maxDocs=44421)
               0.375 = fieldNorm(doc=447)
         2.2188795 = weight(author_txt:chung in 447) [ClassicSimilarity], result of:
           2.2188795 = score(doc=447,freq=1.0), product of:
             0.72378397 = queryWeight, product of:
               1.332113 = boost
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.06646216 = queryNorm
             3.0656652 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.375 = fieldNorm(doc=447)
       0.6666667 = coord(2/3)
   

Similar documents (content)

1. Martin-Bautista, M.J.; Vila, M.-A.; Larsen, H.L.: ¬A fuzzy genetic algorithm approach to an adaptive information retrieval agent (1999) 0.12

   0.11898376 = sum of:
     0.11898376 = product of:
       0.4957657 = sum of:
         0.015756182 = weight(abstract_txt:developed in 4914) [ClassicSimilarity], result of:
           0.015756182 = score(doc=4914,freq=1.0), product of:
             0.0480649 = queryWeight, product of:
               1.1365079 = boost
               4.1959753 = idf(docFreq=1817, maxDocs=44421)
               0.010079121 = queryNorm
             0.3278106 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1959753 = idf(docFreq=1817, maxDocs=44421)
               0.078125 = fieldNorm(doc=4914)
         0.01917252 = weight(abstract_txt:evaluation in 4914) [ClassicSimilarity], result of:
           0.01917252 = score(doc=4914,freq=1.0), product of:
             0.054783132 = queryWeight, product of:
               1.2133381 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.010079121 = queryNorm
             0.34997123 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=4914)
         0.013756973 = weight(abstract_txt:based in 4914) [ClassicSimilarity], result of:
           0.013756973 = score(doc=4914,freq=1.0), product of:
             0.055320483 = queryWeight, product of:
               1.7243142 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.010079121 = queryNorm
             0.24867775 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.078125 = fieldNorm(doc=4914)
         0.059403766 = weight(abstract_txt:algorithm in 4914) [ClassicSimilarity], result of:
           0.059403766 = score(doc=4914,freq=1.0), product of:
             0.13328065 = queryWeight, product of:
               2.3178623 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.010079121 = queryNorm
             0.44570434 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.078125 = fieldNorm(doc=4914)
         0.11290173 = weight(abstract_txt:agent in 4914) [ClassicSimilarity], result of:
           0.11290173 = score(doc=4914,freq=1.0), product of:
             0.20449916 = queryWeight, product of:
               2.87111 = boost
               7.0667386 = idf(docFreq=102, maxDocs=44421)
               0.010079121 = queryNorm
             0.552089 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0667386 = idf(docFreq=102, maxDocs=44421)
               0.078125 = fieldNorm(doc=4914)
         0.27477452 = weight(abstract_txt:genetic in 4914) [ClassicSimilarity], result of:
           0.27477452 = score(doc=4914,freq=2.0), product of:
             0.32323003 = queryWeight, product of:
               4.168015 = boost
               7.694134 = idf(docFreq=54, maxDocs=44421)
               0.010079121 = queryNorm
             0.8500897 = fieldWeight in 4914, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.694134 = idf(docFreq=54, maxDocs=44421)
               0.078125 = fieldNorm(doc=4914)
       0.24 = coord(6/25)
   

2. Blake, P.: Searching out and assessing Web sites (1996) 0.11

   0.106032774 = sum of:
     0.106032774 = product of:
       0.8836065 = sum of:
         0.013756973 = weight(abstract_txt:based in 4163) [ClassicSimilarity], result of:
           0.013756973 = score(doc=4163,freq=1.0), product of:
             0.055320483 = queryWeight, product of:
               1.7243142 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.010079121 = queryNorm
             0.24867775 = fieldWeight in 4163, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.078125 = fieldNorm(doc=4163)
         0.06375095 = weight(abstract_txt:search in 4163) [ClassicSimilarity], result of:
           0.06375095 = score(doc=4163,freq=6.0), product of:
             0.091155194 = queryWeight, product of:
               2.4746804 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010079121 = queryNorm
             0.6993672 = fieldWeight in 4163, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.078125 = fieldNorm(doc=4163)
         0.8060986 = weight(abstract_txt:spider in 4163) [ClassicSimilarity], result of:
           0.8060986 = score(doc=4163,freq=2.0), product of:
             0.75825727 = queryWeight, product of:
               7.81857 = boost
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.010079121 = queryNorm
             1.0630938 = fieldWeight in 4163, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.078125 = fieldNorm(doc=4163)
       0.12 = coord(3/25)
   

3. Lopez-Pujalte, C.; Guerrero Bote, V.P.; Moya-Anegón, F. de: Evaluation of the application of genetic algorithms to relevance feedback (2003) 0.10

   0.10478013 = sum of:
     0.10478013 = product of:
       0.5239006 = sum of:
         0.04389639 = weight(abstract_txt:truly in 3756) [ClassicSimilarity], result of:
           0.04389639 = score(doc=3756,freq=1.0), product of:
             0.07553297 = queryWeight, product of:
               1.0074227 = boost
               7.438788 = idf(docFreq=70, maxDocs=44421)
               0.010079121 = queryNorm
             0.5811553 = fieldWeight in 3756, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.438788 = idf(docFreq=70, maxDocs=44421)
               0.078125 = fieldNorm(doc=3756)
         0.019069944 = weight(abstract_txt:found in 3756) [ClassicSimilarity], result of:
           0.019069944 = score(doc=3756,freq=1.0), product of:
             0.054587558 = queryWeight, product of:
               1.2111704 = boost
               4.4716287 = idf(docFreq=1379, maxDocs=44421)
               0.010079121 = queryNorm
             0.34934598 = fieldWeight in 3756, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4716287 = idf(docFreq=1379, maxDocs=44421)
               0.078125 = fieldNorm(doc=3756)
         0.040395997 = weight(abstract_txt:best in 3756) [ClassicSimilarity], result of:
           0.040395997 = score(doc=3756,freq=1.0), product of:
             0.1030663 = queryWeight, product of:
               2.0382733 = boost
               5.0168557 = idf(docFreq=799, maxDocs=44421)
               0.010079121 = queryNorm
             0.39194185 = fieldWeight in 3756, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0168557 = idf(docFreq=799, maxDocs=44421)
               0.078125 = fieldNorm(doc=3756)
         0.08400961 = weight(abstract_txt:algorithm in 3756) [ClassicSimilarity], result of:
           0.08400961 = score(doc=3756,freq=2.0), product of:
             0.13328065 = queryWeight, product of:
               2.3178623 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.010079121 = queryNorm
             0.63032115 = fieldWeight in 3756, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.078125 = fieldNorm(doc=3756)
         0.3365287 = weight(abstract_txt:genetic in 3756) [ClassicSimilarity], result of:
           0.3365287 = score(doc=3756,freq=3.0), product of:
             0.32323003 = queryWeight, product of:
               4.168015 = boost
               7.694134 = idf(docFreq=54, maxDocs=44421)
               0.010079121 = queryNorm
             1.0411431 = fieldWeight in 3756, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.694134 = idf(docFreq=54, maxDocs=44421)
               0.078125 = fieldNorm(doc=3756)
       0.2 = coord(5/25)
   

4. Chen, H.; Houston, A.L.; Sewell, R.R.; Schatz, B.R.: Internet browsing and searching : user evaluations of category map and concept space techniques (1998) 0.10

   0.10281487 = sum of:
     0.10281487 = product of:
       0.36719596 = sum of:
         0.009453709 = weight(abstract_txt:developed in 1869) [ClassicSimilarity], result of:
           0.009453709 = score(doc=1869,freq=1.0), product of:
             0.0480649 = queryWeight, product of:
               1.1365079 = boost
               4.1959753 = idf(docFreq=1817, maxDocs=44421)
               0.010079121 = queryNorm
             0.19668634 = fieldWeight in 1869, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1959753 = idf(docFreq=1817, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
         0.011441967 = weight(abstract_txt:found in 1869) [ClassicSimilarity], result of:
           0.011441967 = score(doc=1869,freq=1.0), product of:
             0.054587558 = queryWeight, product of:
               1.2111704 = boost
               4.4716287 = idf(docFreq=1379, maxDocs=44421)
               0.010079121 = queryNorm
             0.2096076 = fieldWeight in 1869, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4716287 = idf(docFreq=1379, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
         0.011673178 = weight(abstract_txt:based in 1869) [ClassicSimilarity], result of:
           0.011673178 = score(doc=1869,freq=2.0), product of:
             0.055320483 = queryWeight, product of:
               1.7243142 = boost
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.010079121 = queryNorm
             0.21101005 = fieldWeight in 1869, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1830752 = idf(docFreq=5005, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
         0.0242376 = weight(abstract_txt:best in 1869) [ClassicSimilarity], result of:
           0.0242376 = score(doc=1869,freq=1.0), product of:
             0.1030663 = queryWeight, product of:
               2.0382733 = boost
               5.0168557 = idf(docFreq=799, maxDocs=44421)
               0.010079121 = queryNorm
             0.23516512 = fieldWeight in 1869, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0168557 = idf(docFreq=799, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
         0.07128452 = weight(abstract_txt:algorithm in 1869) [ClassicSimilarity], result of:
           0.07128452 = score(doc=1869,freq=4.0), product of:
             0.13328065 = queryWeight, product of:
               2.3178623 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.010079121 = queryNorm
             0.53484523 = fieldWeight in 1869, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
         0.027047241 = weight(abstract_txt:search in 1869) [ClassicSimilarity], result of:
           0.027047241 = score(doc=1869,freq=3.0), product of:
             0.091155194 = queryWeight, product of:
               2.4746804 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010079121 = queryNorm
             0.2967164 = fieldWeight in 1869, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
         0.21205774 = weight(abstract_txt:homepages in 1869) [ClassicSimilarity], result of:
           0.21205774 = score(doc=1869,freq=4.0), product of:
             0.27568072 = queryWeight, product of:
               3.3335521 = boost
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.010079121 = queryNorm
             0.769215 = fieldWeight in 1869, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.046875 = fieldNorm(doc=1869)
       0.28 = coord(7/25)
   

5. Chen, H.; Fan, H.; Chau, M.; Zeng, D.: MetaSpider : meta-searching and categorization on the Web (2001) 0.09

   0.09000698 = sum of:
     0.09000698 = product of:
       0.56254363 = sum of:
         0.015338017 = weight(abstract_txt:evaluation in 849) [ClassicSimilarity], result of:
           0.015338017 = score(doc=849,freq=1.0), product of:
             0.054783132 = queryWeight, product of:
               1.2133381 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.010079121 = queryNorm
             0.279977 = fieldWeight in 849, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=849)
         0.032316796 = weight(abstract_txt:best in 849) [ClassicSimilarity], result of:
           0.032316796 = score(doc=849,freq=1.0), product of:
             0.1030663 = queryWeight, product of:
               2.0382733 = boost
               5.0168557 = idf(docFreq=799, maxDocs=44421)
               0.010079121 = queryNorm
             0.31355348 = fieldWeight in 849, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.0168557 = idf(docFreq=799, maxDocs=44421)
               0.0625 = fieldNorm(doc=849)
         0.05889061 = weight(abstract_txt:search in 849) [ClassicSimilarity], result of:
           0.05889061 = score(doc=849,freq=8.0), product of:
             0.091155194 = queryWeight, product of:
               2.4746804 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.010079121 = queryNorm
             0.6460478 = fieldWeight in 849, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=849)
         0.4559982 = weight(abstract_txt:spider in 849) [ClassicSimilarity], result of:
           0.4559982 = score(doc=849,freq=1.0), product of:
             0.75825727 = queryWeight, product of:
               7.81857 = boost
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.010079121 = queryNorm
             0.60137665 = fieldWeight in 849, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.0625 = fieldNorm(doc=849)
       0.16 = coord(4/25)