Document (#37664)

Author
Ostermann, F.O.
Tomko, M.
Purves, R.
Title
User evaluation of automatically generated keywords and toponyms for geo-referenced images
Source
Journal of the American Society for Information Science and Technology. 64(2013) no.3, S.480-499
Year
2013
Abstract
This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.
Field
Geowissenschaften
Form
Bilder

Similar documents (author)

  1. Purves, R.S.; Sanderson, M.: ¬A methodology to allow avalanche forecasting on an information retrieval system (1998) 4.95
    4.954854 = sum of:
      4.954854 = weight(author_txt:purves in 2073) [ClassicSimilarity], result of:
        4.954854 = fieldWeight in 2073, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.909708 = idf(docFreq=5, maxDocs=44421)
          0.5 = fieldNorm(doc=2073)
    
  2. Purves, D.; Lotto, R.B.; Nundy, S.: Richtiges Sehen - eine optische Täuschung? (2003) 3.72
    3.7161405 = sum of:
      3.7161405 = weight(author_txt:purves in 2891) [ClassicSimilarity], result of:
        3.7161405 = fieldWeight in 2891, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.909708 = idf(docFreq=5, maxDocs=44421)
          0.375 = fieldNorm(doc=2891)
    
  3. Purves, R.S.; Winter, S.; Kuhn, W.: Places in information science (2019) 3.72
    3.7161405 = sum of:
      3.7161405 = weight(author_txt:purves in 433) [ClassicSimilarity], result of:
        3.7161405 = fieldWeight in 433, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.909708 = idf(docFreq=5, maxDocs=44421)
          0.375 = fieldNorm(doc=433)
    
  4. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 3.10
    3.0967836 = sum of:
      3.0967836 = weight(author_txt:purves in 4145) [ClassicSimilarity], result of:
        3.0967836 = fieldWeight in 4145, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.909708 = idf(docFreq=5, maxDocs=44421)
          0.3125 = fieldNorm(doc=4145)
    

Similar documents (content)

  1. Tsai, C.-F.; McGarry, K.; Tait, J.: Qualitative evaluation of automatic assignment of keywords to images (2006) 0.28
    0.2817865 = sum of:
      0.2817865 = product of:
        0.78274024 = sum of:
          0.028451351 = weight(abstract_txt:methods in 1963) [ClassicSimilarity], result of:
            0.028451351 = score(doc=1963,freq=4.0), product of:
              0.05493238 = queryWeight, product of:
                1.0466913 = boost
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.01266617 = queryNorm
              0.5179341 = fieldWeight in 1963, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.03343664 = weight(abstract_txt:qualitative in 1963) [ClassicSimilarity], result of:
            0.03343664 = score(doc=1963,freq=1.0), product of:
              0.09710946 = queryWeight, product of:
                1.3916665 = boost
                5.509105 = idf(docFreq=488, maxDocs=44421)
                0.01266617 = queryNorm
              0.34431908 = fieldWeight in 1963, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.509105 = idf(docFreq=488, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.021156147 = weight(abstract_txt:user in 1963) [ClassicSimilarity], result of:
            0.021156147 = score(doc=1963,freq=2.0), product of:
              0.06502663 = queryWeight, product of:
                1.3947477 = boost
                3.6808684 = idf(docFreq=3042, maxDocs=44421)
                0.01266617 = queryNorm
              0.32534587 = fieldWeight in 1963, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6808684 = idf(docFreq=3042, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.080694474 = weight(abstract_txt:image in 1963) [ClassicSimilarity], result of:
            0.080694474 = score(doc=1963,freq=3.0), product of:
              0.13867526 = queryWeight, product of:
                2.0368054 = boost
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.01266617 = queryNorm
              0.58189523 = fieldWeight in 1963, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.101690724 = weight(abstract_txt:evaluation in 1963) [ClassicSimilarity], result of:
            0.101690724 = score(doc=1963,freq=8.0), product of:
              0.12841448 = queryWeight, product of:
                2.2632182 = boost
                4.479632 = idf(docFreq=1368, maxDocs=44421)
                0.01266617 = queryNorm
              0.7918945 = fieldWeight in 1963, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                4.479632 = idf(docFreq=1368, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.06732387 = weight(abstract_txt:automatically in 1963) [ClassicSimilarity], result of:
            0.06732387 = score(doc=1963,freq=1.0), product of:
              0.19509038 = queryWeight, product of:
                2.7895703 = boost
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.01266617 = queryNorm
              0.3450907 = fieldWeight in 1963, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.12011657 = weight(abstract_txt:annotations in 1963) [ClassicSimilarity], result of:
            0.12011657 = score(doc=1963,freq=1.0), product of:
              0.2607427 = queryWeight, product of:
                2.7929046 = boost
                7.370734 = idf(docFreq=75, maxDocs=44421)
                0.01266617 = queryNorm
              0.4606709 = fieldWeight in 1963, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.370734 = idf(docFreq=75, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.10822377 = weight(abstract_txt:keywords in 1963) [ClassicSimilarity], result of:
            0.10822377 = score(doc=1963,freq=1.0), product of:
              0.2883865 = queryWeight, product of:
                3.7919436 = boost
                6.004374 = idf(docFreq=297, maxDocs=44421)
                0.01266617 = queryNorm
              0.37527338 = fieldWeight in 1963, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.004374 = idf(docFreq=297, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
          0.22164673 = weight(abstract_txt:images in 1963) [ClassicSimilarity], result of:
            0.22164673 = score(doc=1963,freq=3.0), product of:
              0.37716693 = queryWeight, product of:
                5.4853106 = boost
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.01266617 = queryNorm
              0.58766216 = fieldWeight in 1963, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.0625 = fieldNorm(doc=1963)
        0.36 = coord(9/25)
    
  2. Jörgensen, C.; Jaimes, A.; Benitez, A.B.; Chang, S.-F.: ¬A conceptual framework and empirical research for classifying visual descriptors (2001) 0.12
    0.123255186 = sum of:
      0.123255186 = product of:
        0.5135633 = sum of:
          0.05229203 = weight(abstract_txt:experiments in 532) [ClassicSimilarity], result of:
            0.05229203 = score(doc=532,freq=3.0), product of:
              0.090718605 = queryWeight, product of:
                1.3450938 = boost
                5.324741 = idf(docFreq=587, maxDocs=44421)
                0.01266617 = queryNorm
              0.5764201 = fieldWeight in 532, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.324741 = idf(docFreq=587, maxDocs=44421)
                0.0625 = fieldNorm(doc=532)
          0.016451238 = weight(abstract_txt:article in 532) [ClassicSimilarity], result of:
            0.016451238 = score(doc=532,freq=1.0), product of:
              0.06928022 = queryWeight, product of:
                1.4396425 = boost
                3.79935 = idf(docFreq=2702, maxDocs=44421)
                0.01266617 = queryNorm
              0.23745938 = fieldWeight in 532, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.79935 = idf(docFreq=2702, maxDocs=44421)
                0.0625 = fieldNorm(doc=532)
          0.080694474 = weight(abstract_txt:image in 532) [ClassicSimilarity], result of:
            0.080694474 = score(doc=532,freq=3.0), product of:
              0.13867526 = queryWeight, product of:
                2.0368054 = boost
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.01266617 = queryNorm
              0.58189523 = fieldWeight in 532, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.0625 = fieldNorm(doc=532)
          0.11582792 = weight(abstract_txt:generated in 532) [ClassicSimilarity], result of:
            0.11582792 = score(doc=532,freq=3.0), product of:
              0.19421892 = queryWeight, product of:
                2.783333 = boost
                5.509105 = idf(docFreq=488, maxDocs=44421)
                0.01266617 = queryNorm
              0.59637815 = fieldWeight in 532, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.509105 = idf(docFreq=488, maxDocs=44421)
                0.0625 = fieldNorm(doc=532)
          0.06732387 = weight(abstract_txt:automatically in 532) [ClassicSimilarity], result of:
            0.06732387 = score(doc=532,freq=1.0), product of:
              0.19509038 = queryWeight, product of:
                2.7895703 = boost
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.01266617 = queryNorm
              0.3450907 = fieldWeight in 532, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.0625 = fieldNorm(doc=532)
          0.1809738 = weight(abstract_txt:images in 532) [ClassicSimilarity], result of:
            0.1809738 = score(doc=532,freq=2.0), product of:
              0.37716693 = queryWeight, product of:
                5.4853106 = boost
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.01266617 = queryNorm
              0.47982416 = fieldWeight in 532, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.0625 = fieldNorm(doc=532)
        0.24 = coord(6/25)
    
  3. Rasmussen, E.M.: Indexing images (1997) 0.12
    0.1194036 = sum of:
      0.1194036 = product of:
        0.597018 = sum of:
          0.017782094 = weight(abstract_txt:methods in 3215) [ClassicSimilarity], result of:
            0.017782094 = score(doc=3215,freq=1.0), product of:
              0.05493238 = queryWeight, product of:
                1.0466913 = boost
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.01266617 = queryNorm
              0.3237088 = fieldWeight in 3215, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.078125 = fieldNorm(doc=3215)
          0.13022014 = weight(abstract_txt:image in 3215) [ClassicSimilarity], result of:
            0.13022014 = score(doc=3215,freq=5.0), product of:
              0.13867526 = queryWeight, product of:
                2.0368054 = boost
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.01266617 = queryNorm
              0.93902946 = fieldWeight in 3215, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.078125 = fieldNorm(doc=3215)
          0.044941373 = weight(abstract_txt:evaluation in 3215) [ClassicSimilarity], result of:
            0.044941373 = score(doc=3215,freq=1.0), product of:
              0.12841448 = queryWeight, product of:
                2.2632182 = boost
                4.479632 = idf(docFreq=1368, maxDocs=44421)
                0.01266617 = queryNorm
              0.34997123 = fieldWeight in 3215, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.479632 = idf(docFreq=1368, maxDocs=44421)
                0.078125 = fieldNorm(doc=3215)
          0.08415484 = weight(abstract_txt:automatically in 3215) [ClassicSimilarity], result of:
            0.08415484 = score(doc=3215,freq=1.0), product of:
              0.19509038 = queryWeight, product of:
                2.7895703 = boost
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.01266617 = queryNorm
              0.43136334 = fieldWeight in 3215, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.078125 = fieldNorm(doc=3215)
          0.31991953 = weight(abstract_txt:images in 3215) [ClassicSimilarity], result of:
            0.31991953 = score(doc=3215,freq=4.0), product of:
              0.37716693 = queryWeight, product of:
                5.4853106 = boost
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.01266617 = queryNorm
              0.8482173 = fieldWeight in 3215, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.078125 = fieldNorm(doc=3215)
        0.2 = coord(5/25)
    
  4. Rowe, N.C.: Precise and efficient retrieval of captioned images : the MARIE project (1999) 0.12
    0.1173589 = sum of:
      0.1173589 = product of:
        0.5867945 = sum of:
          0.025147678 = weight(abstract_txt:methods in 972) [ClassicSimilarity], result of:
            0.025147678 = score(doc=972,freq=2.0), product of:
              0.05493238 = queryWeight, product of:
                1.0466913 = boost
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.01266617 = queryNorm
              0.45779335 = fieldWeight in 972, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.078125 = fieldNorm(doc=972)
          0.121630356 = weight(abstract_txt:captioned in 972) [ClassicSimilarity], result of:
            0.121630356 = score(doc=972,freq=1.0), product of:
              0.1571054 = queryWeight, product of:
                1.2516559 = boost
                9.909708 = idf(docFreq=5, maxDocs=44421)
                0.01266617 = queryNorm
              0.7741959 = fieldWeight in 972, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.909708 = idf(docFreq=5, maxDocs=44421)
                0.078125 = fieldNorm(doc=972)
          0.03773852 = weight(abstract_txt:experiments in 972) [ClassicSimilarity], result of:
            0.03773852 = score(doc=972,freq=1.0), product of:
              0.090718605 = queryWeight, product of:
                1.3450938 = boost
                5.324741 = idf(docFreq=587, maxDocs=44421)
                0.01266617 = queryNorm
              0.4159954 = fieldWeight in 972, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.324741 = idf(docFreq=587, maxDocs=44421)
                0.078125 = fieldNorm(doc=972)
          0.08235846 = weight(abstract_txt:image in 972) [ClassicSimilarity], result of:
            0.08235846 = score(doc=972,freq=2.0), product of:
              0.13867526 = queryWeight, product of:
                2.0368054 = boost
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.01266617 = queryNorm
              0.59389436 = fieldWeight in 972, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.078125 = fieldNorm(doc=972)
          0.31991953 = weight(abstract_txt:images in 972) [ClassicSimilarity], result of:
            0.31991953 = score(doc=972,freq=4.0), product of:
              0.37716693 = queryWeight, product of:
                5.4853106 = boost
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.01266617 = queryNorm
              0.8482173 = fieldWeight in 972, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.078125 = fieldNorm(doc=972)
        0.2 = coord(5/25)
    
  5. Dolin, R.; Agrawal, D.; El Abbadi, A.; Pearlman, J.: Using automated classification for summarizing and selecting heterogeneous information sources (1998) 0.12
    0.1166571 = sum of:
      0.1166571 = product of:
        0.36455345 = sum of:
          0.0070727533 = weight(abstract_txt:number in 2253) [ClassicSimilarity], result of:
            0.0070727533 = score(doc=2253,freq=1.0), product of:
              0.054725803 = queryWeight, product of:
                1.0447214 = boost
                4.1356745 = idf(docFreq=1930, maxDocs=44421)
                0.01266617 = queryNorm
              0.12923983 = fieldWeight in 2253, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1356745 = idf(docFreq=1930, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.007112838 = weight(abstract_txt:methods in 2253) [ClassicSimilarity], result of:
            0.007112838 = score(doc=2253,freq=1.0), product of:
              0.05493238 = queryWeight, product of:
                1.0466913 = boost
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.01266617 = queryNorm
              0.12948352 = fieldWeight in 2253, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1434727 = idf(docFreq=1915, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.010578074 = weight(abstract_txt:user in 2253) [ClassicSimilarity], result of:
            0.010578074 = score(doc=2253,freq=2.0), product of:
              0.06502663 = queryWeight, product of:
                1.3947477 = boost
                3.6808684 = idf(docFreq=3042, maxDocs=44421)
                0.01266617 = queryNorm
              0.16267294 = fieldWeight in 2253, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6808684 = idf(docFreq=3042, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.023294488 = weight(abstract_txt:image in 2253) [ClassicSimilarity], result of:
            0.023294488 = score(doc=2253,freq=1.0), product of:
              0.13867526 = queryWeight, product of:
                2.0368054 = boost
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.01266617 = queryNorm
              0.16797869 = fieldWeight in 2253, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.375318 = idf(docFreq=558, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.06732387 = weight(abstract_txt:automatically in 2253) [ClassicSimilarity], result of:
            0.06732387 = score(doc=2253,freq=4.0), product of:
              0.19509038 = queryWeight, product of:
                2.7895703 = boost
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.01266617 = queryNorm
              0.3450907 = fieldWeight in 2253, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.521451 = idf(docFreq=482, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.054111887 = weight(abstract_txt:keywords in 2253) [ClassicSimilarity], result of:
            0.054111887 = score(doc=2253,freq=1.0), product of:
              0.2883865 = queryWeight, product of:
                3.7919436 = boost
                6.004374 = idf(docFreq=297, maxDocs=44421)
                0.01266617 = queryNorm
              0.18763669 = fieldWeight in 2253, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.004374 = idf(docFreq=297, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.1310756 = weight(abstract_txt:referenced in 2253) [ClassicSimilarity], result of:
            0.1310756 = score(doc=2253,freq=1.0), product of:
              0.52014893 = queryWeight, product of:
                5.0925846 = boost
                8.063882 = idf(docFreq=37, maxDocs=44421)
                0.01266617 = queryNorm
              0.2519963 = fieldWeight in 2253, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.063882 = idf(docFreq=37, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
          0.0639839 = weight(abstract_txt:images in 2253) [ClassicSimilarity], result of:
            0.0639839 = score(doc=2253,freq=1.0), product of:
              0.37716693 = queryWeight, product of:
                5.4853106 = boost
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.01266617 = queryNorm
              0.16964346 = fieldWeight in 2253, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.428591 = idf(docFreq=529, maxDocs=44421)
                0.03125 = fieldNorm(doc=2253)
        0.32 = coord(8/25)