Document (#32021)

Author

Sherman, C.
Price, G.

Title

¬The invisible Web : uncovering sources search engines can't see

Source

Library trends. 52(2004) no.2, S.282-298

Year

2004

Abstract

The paradox of the Invisible Web is that it's easy to understand why it exists, but it's very hard to actually define in concrete, specific terms. In a nutshell, the Invisible Web consists of content that's been excluded from general-purpose search engines and Web directories such as Lycos and LookSmart-and yes, even Google. There's nothing inherently "invisible" about this content. But since this content is not easily located with the information-seeking tools used by most Web users, it's effectively invisible because it's so difficult to find unless you know exactly where to look. In this paper, we define the Invisible Web and delve into the reasons search engines can't "see" its content. We also discuss the four different "types" of invisibility, ranging from the "opaque" Web which is relatively accessible to the searcher, to the truly invisible Web, which requires specialized finding aids to access effectively.

Footnote

Beitrag in einem Themenheft: Organizing the Internet

Theme

Internet

Similar documents (author)

1. Sherman, C.; Price, G.: ¬The invisible Web : uncovering information sources search engines can't see (2001) 6.26

   6.2613115 = sum of:
     6.2613115 = sum of:
       2.6576781 = weight(author_txt:price in 187) [ClassicSimilarity], result of:
         2.6576781 = score(doc=187,freq=1.0), product of:
           0.6323566 = queryWeight, product of:
             8.405631 = idf(docFreq=26, maxDocs=44421)
             0.075230114 = queryNorm
           4.2028155 = fieldWeight in 187, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.405631 = idf(docFreq=26, maxDocs=44421)
             0.5 = fieldNorm(doc=187)
       3.6036336 = weight(author_txt:sherman in 187) [ClassicSimilarity], result of:
         3.6036336 = score(doc=187,freq=1.0), product of:
           0.7746774 = queryWeight, product of:
             1.1068262 = boost
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.075230114 = queryNorm
           4.6517863 = fieldWeight in 187, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.303573 = idf(docFreq=10, maxDocs=44421)
             0.5 = fieldNorm(doc=187)
   

2. Sherman, C.R.: ICONCLASS: a historical perspective (1987) 2.25

   2.2522712 = sum of:
     2.2522712 = product of:
       4.5045424 = sum of:
         4.5045424 = weight(author_txt:sherman in 3268) [ClassicSimilarity], result of:
           4.5045424 = score(doc=3268,freq=1.0), product of:
             0.7746774 = queryWeight, product of:
               1.1068262 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.075230114 = queryNorm
             5.814733 = fieldWeight in 3268, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.625 = fieldNorm(doc=3268)
       0.5 = coord(1/2)
   

3. Sherman, R.J.: ¬The electronic book (1993) 2.25

   2.2522712 = sum of:
     2.2522712 = product of:
       4.5045424 = sum of:
         4.5045424 = weight(author_txt:sherman in 4567) [ClassicSimilarity], result of:
           4.5045424 = score(doc=4567,freq=1.0), product of:
             0.7746774 = queryWeight, product of:
               1.1068262 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.075230114 = queryNorm
             5.814733 = fieldWeight in 4567, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.625 = fieldNorm(doc=4567)
       0.5 = coord(1/2)
   

4. Sherman, C.: What's new with Web search (2000) 2.25

   2.2522712 = sum of:
     2.2522712 = product of:
       4.5045424 = sum of:
         4.5045424 = weight(author_txt:sherman in 5795) [ClassicSimilarity], result of:
           4.5045424 = score(doc=5795,freq=1.0), product of:
             0.7746774 = queryWeight, product of:
               1.1068262 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.075230114 = queryNorm
             5.814733 = fieldWeight in 5795, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.625 = fieldNorm(doc=5795)
       0.5 = coord(1/2)
   

5. Sherman, C.: ¬The future of Web search (1999) 2.25

   2.2522712 = sum of:
     2.2522712 = product of:
       4.5045424 = sum of:
         4.5045424 = weight(author_txt:sherman in 5796) [ClassicSimilarity], result of:
           4.5045424 = score(doc=5796,freq=1.0), product of:
             0.7746774 = queryWeight, product of:
               1.1068262 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.075230114 = queryNorm
             5.814733 = fieldWeight in 5796, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.625 = fieldNorm(doc=5796)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Lewandowski, D.; Mayr, P.: Exploring the academic invisible Web (2006) 0.26

   0.26198408 = sum of:
     0.26198408 = product of:
       1.0916004 = sum of:
         0.0061839954 = weight(abstract_txt:this in 3580) [ClassicSimilarity], result of:
           0.0061839954 = score(doc=3580,freq=2.0), product of:
             0.0290761 = queryWeight, product of:
               1.040265 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.011615936 = queryNorm
             0.21268311 = fieldWeight in 3580, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=3580)
         0.066787116 = weight(abstract_txt:uncovering in 3580) [ClassicSimilarity], result of:
           0.066787116 = score(doc=3580,freq=1.0), product of:
             0.12410466 = queryWeight, product of:
               1.2408215 = boost
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.011615936 = queryNorm
             0.53815156 = fieldWeight in 3580, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.0625 = fieldNorm(doc=3580)
         0.026535034 = weight(abstract_txt:search in 3580) [ClassicSimilarity], result of:
           0.026535034 = score(doc=3580,freq=3.0), product of:
             0.06707171 = queryWeight, product of:
               1.5799581 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.011615936 = queryNorm
             0.39562184 = fieldWeight in 3580, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=3580)
         0.04722212 = weight(abstract_txt:define in 3580) [ClassicSimilarity], result of:
           0.04722212 = score(doc=3580,freq=1.0), product of:
             0.12409852 = queryWeight, product of:
               1.7547432 = boost
               6.0883393 = idf(docFreq=273, maxDocs=44421)
               0.011615936 = queryNorm
             0.3805212 = fieldWeight in 3580, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0883393 = idf(docFreq=273, maxDocs=44421)
               0.0625 = fieldNorm(doc=3580)
         0.08046027 = weight(abstract_txt:engines in 3580) [ClassicSimilarity], result of:
           0.08046027 = score(doc=3580,freq=3.0), product of:
             0.14051218 = queryWeight, product of:
               2.2868247 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.011615936 = queryNorm
             0.57262135 = fieldWeight in 3580, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=3580)
         0.86441195 = weight(abstract_txt:invisible in 3580) [ClassicSimilarity], result of:
           0.86441195 = score(doc=3580,freq=7.0), product of:
             0.6841324 = queryWeight, product of:
               7.7078695 = boost
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.011615936 = queryNorm
             1.2635156 = fieldWeight in 3580, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.0625 = fieldNorm(doc=3580)
       0.24 = coord(6/25)
   

2. Lewandowski, D.; Mayr, P.: Exploring the academic invisible Web (2006) 0.25

   0.24659485 = sum of:
     0.24659485 = product of:
       1.0274786 = sum of:
         0.0061839954 = weight(abstract_txt:this in 4752) [ClassicSimilarity], result of:
           0.0061839954 = score(doc=4752,freq=2.0), product of:
             0.0290761 = queryWeight, product of:
               1.040265 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.011615936 = queryNorm
             0.21268311 = fieldWeight in 4752, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=4752)
         0.066787116 = weight(abstract_txt:uncovering in 4752) [ClassicSimilarity], result of:
           0.066787116 = score(doc=4752,freq=1.0), product of:
             0.12410466 = queryWeight, product of:
               1.2408215 = boost
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.011615936 = queryNorm
             0.53815156 = fieldWeight in 4752, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.0625 = fieldNorm(doc=4752)
         0.026535034 = weight(abstract_txt:search in 4752) [ClassicSimilarity], result of:
           0.026535034 = score(doc=4752,freq=3.0), product of:
             0.06707171 = queryWeight, product of:
               1.5799581 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.011615936 = queryNorm
             0.39562184 = fieldWeight in 4752, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=4752)
         0.04722212 = weight(abstract_txt:define in 4752) [ClassicSimilarity], result of:
           0.04722212 = score(doc=4752,freq=1.0), product of:
             0.12409852 = queryWeight, product of:
               1.7547432 = boost
               6.0883393 = idf(docFreq=273, maxDocs=44421)
               0.011615936 = queryNorm
             0.3805212 = fieldWeight in 4752, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0883393 = idf(docFreq=273, maxDocs=44421)
               0.0625 = fieldNorm(doc=4752)
         0.08046027 = weight(abstract_txt:engines in 4752) [ClassicSimilarity], result of:
           0.08046027 = score(doc=4752,freq=3.0), product of:
             0.14051218 = queryWeight, product of:
               2.2868247 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.011615936 = queryNorm
             0.57262135 = fieldWeight in 4752, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=4752)
         0.80029005 = weight(abstract_txt:invisible in 4752) [ClassicSimilarity], result of:
           0.80029005 = score(doc=4752,freq=6.0), product of:
             0.6841324 = queryWeight, product of:
               7.7078695 = boost
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.011615936 = queryNorm
             1.1697882 = fieldWeight in 4752, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.0625 = fieldNorm(doc=4752)
       0.24 = coord(6/25)
   

3. Sherman, C.; Price, G.: ¬The invisible Web : uncovering information sources search engines can't see (2001) 0.14

   0.13600455 = sum of:
     0.13600455 = product of:
       0.6800227 = sum of:
         0.034959465 = weight(abstract_txt:located in 187) [ClassicSimilarity], result of:
           0.034959465 = score(doc=187,freq=1.0), product of:
             0.08060639 = queryWeight, product of:
               6.939294 = idf(docFreq=116, maxDocs=44421)
               0.011615936 = queryNorm
             0.43370587 = fieldWeight in 187, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.939294 = idf(docFreq=116, maxDocs=44421)
               0.0625 = fieldNorm(doc=187)
         0.0061839954 = weight(abstract_txt:this in 187) [ClassicSimilarity], result of:
           0.0061839954 = score(doc=187,freq=2.0), product of:
             0.0290761 = queryWeight, product of:
               1.040265 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.011615936 = queryNorm
             0.21268311 = fieldWeight in 187, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=187)
         0.026535034 = weight(abstract_txt:search in 187) [ClassicSimilarity], result of:
           0.026535034 = score(doc=187,freq=3.0), product of:
             0.06707171 = queryWeight, product of:
               1.5799581 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.011615936 = queryNorm
             0.39562184 = fieldWeight in 187, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=187)
         0.046453763 = weight(abstract_txt:engines in 187) [ClassicSimilarity], result of:
           0.046453763 = score(doc=187,freq=1.0), product of:
             0.14051218 = queryWeight, product of:
               2.2868247 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.011615936 = queryNorm
             0.3306031 = fieldWeight in 187, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=187)
         0.5658905 = weight(abstract_txt:invisible in 187) [ClassicSimilarity], result of:
           0.5658905 = score(doc=187,freq=3.0), product of:
             0.6841324 = queryWeight, product of:
               7.7078695 = boost
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.011615936 = queryNorm
             0.8271652 = fieldWeight in 187, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.0625 = fieldNorm(doc=187)
       0.2 = coord(5/25)
   

4. Morrison, R.: Making the invisible visible : critical discourse analysis as a tool for search engine research (2024) 0.10

   0.095719315 = sum of:
     0.095719315 = product of:
       0.59824574 = sum of:
         0.0097777555 = weight(abstract_txt:this in 2276) [ClassicSimilarity], result of:
           0.0097777555 = score(doc=2276,freq=5.0), product of:
             0.0290761 = queryWeight, product of:
               1.040265 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.011615936 = queryNorm
             0.33628154 = fieldWeight in 2276, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=2276)
         0.045960028 = weight(abstract_txt:search in 2276) [ClassicSimilarity], result of:
           0.045960028 = score(doc=2276,freq=9.0), product of:
             0.06707171 = queryWeight, product of:
               1.5799581 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.011615936 = queryNorm
             0.6852371 = fieldWeight in 2276, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=2276)
         0.08046027 = weight(abstract_txt:engines in 2276) [ClassicSimilarity], result of:
           0.08046027 = score(doc=2276,freq=3.0), product of:
             0.14051218 = queryWeight, product of:
               2.2868247 = boost
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.011615936 = queryNorm
             0.57262135 = fieldWeight in 2276, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.2896495 = idf(docFreq=608, maxDocs=44421)
               0.0625 = fieldNorm(doc=2276)
         0.46204767 = weight(abstract_txt:invisible in 2276) [ClassicSimilarity], result of:
           0.46204767 = score(doc=2276,freq=2.0), product of:
             0.6841324 = queryWeight, product of:
               7.7078695 = boost
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.011615936 = queryNorm
             0.67537755 = fieldWeight in 2276, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.0625 = fieldNorm(doc=2276)
       0.16 = coord(4/25)
   

5. Borgman, C.L.: ¬The invisible library : paradox of the global information infrastructure (2003) 0.09

   0.08945562 = sum of:
     0.08945562 = product of:
       0.7454635 = sum of:
         0.0065591177 = weight(abstract_txt:this in 1001) [ClassicSimilarity], result of:
           0.0065591177 = score(doc=1001,freq=1.0), product of:
             0.0290761 = queryWeight, product of:
               1.040265 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.011615936 = queryNorm
             0.2255845 = fieldWeight in 1001, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.09375 = fieldNorm(doc=1001)
         0.045832857 = weight(abstract_txt:content in 1001) [ClassicSimilarity], result of:
           0.045832857 = score(doc=1001,freq=1.0), product of:
             0.11696881 = queryWeight, product of:
               2.4092412 = boost
               4.1796083 = idf(docFreq=1847, maxDocs=44421)
               0.011615936 = queryNorm
             0.39183828 = fieldWeight in 1001, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1796083 = idf(docFreq=1847, maxDocs=44421)
               0.09375 = fieldNorm(doc=1001)
         0.6930715 = weight(abstract_txt:invisible in 1001) [ClassicSimilarity], result of:
           0.6930715 = score(doc=1001,freq=2.0), product of:
             0.6841324 = queryWeight, product of:
               7.7078695 = boost
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.011615936 = queryNorm
             1.0130663 = fieldWeight in 1001, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.6410246 = idf(docFreq=57, maxDocs=44421)
               0.09375 = fieldNorm(doc=1001)
       0.12 = coord(3/25)