Document (#24980)

Author

Peters, C.
Braschler, M.

Title

Cross-language system evaluation : the CLEF campaigns

Source

Journal of the American Society for Information Science and technology. 52(2001) no.12, S.1067-1072

Year

2001

Abstract

The goals of the CLEF (Cross-Language Evaluation Forum) series of evaluation campaigns for information retrieval systems operating on European languages are described. The difficulties of organizing an activity which aims at an objective evaluation of systems running on and over a number' of different languages are examined. The discussion includes an analysis of the first results and proposals for possible developments in the future.

Theme

Multilinguale Probleme

Similar documents (author)

1. Peters, C.; Braschler, M.; Clough, P.: Multilingual information retrieval : from research to practice (2012) 4.65

   4.654178 = sum of:
     4.654178 = sum of:
       1.4562198 = weight(author_txt:peters in 1361) [ClassicSimilarity], result of:
         1.4562198 = score(doc=1361,freq=1.0), product of:
           0.50935054 = queryWeight, product of:
             7.62393 = idf(docFreq=58, maxDocs=44421)
             0.066809446 = queryNorm
           2.8589737 = fieldWeight in 1361, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.62393 = idf(docFreq=58, maxDocs=44421)
             0.375 = fieldNorm(doc=1361)
       3.1979582 = weight(author_txt:braschler in 1361) [ClassicSimilarity], result of:
         3.1979582 = score(doc=1361,freq=1.0), product of:
           0.86055905 = queryWeight, product of:
             1.2998163 = boost
             9.909708 = idf(docFreq=5, maxDocs=44421)
             0.066809446 = queryNorm
           3.7161405 = fieldWeight in 1361, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.909708 = idf(docFreq=5, maxDocs=44421)
             0.375 = fieldNorm(doc=1361)
   

2. Agosti, M.; Braschler, M.; Ferro, N.; Peters, C.; Siebinga, S.: Roadmap for multiLingual information access in the European Library (2007) 3.88

   3.8784819 = sum of:
     3.8784819 = sum of:
       1.2135166 = weight(author_txt:peters in 3431) [ClassicSimilarity], result of:
         1.2135166 = score(doc=3431,freq=1.0), product of:
           0.50935054 = queryWeight, product of:
             7.62393 = idf(docFreq=58, maxDocs=44421)
             0.066809446 = queryNorm
           2.3824782 = fieldWeight in 3431, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.62393 = idf(docFreq=58, maxDocs=44421)
             0.3125 = fieldNorm(doc=3431)
       2.6649652 = weight(author_txt:braschler in 3431) [ClassicSimilarity], result of:
         2.6649652 = score(doc=3431,freq=1.0), product of:
           0.86055905 = queryWeight, product of:
             1.2998163 = boost
             9.909708 = idf(docFreq=5, maxDocs=44421)
             0.066809446 = queryNorm
           3.0967836 = fieldWeight in 3431, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.909708 = idf(docFreq=5, maxDocs=44421)
             0.3125 = fieldNorm(doc=3431)
   

3. Peters, C.M.: CD-ROM: its potential in libraries (1986) 1.21

   1.2135166 = sum of:
     1.2135166 = product of:
       2.4270332 = sum of:
         2.4270332 = weight(author_txt:peters in 534) [ClassicSimilarity], result of:
           2.4270332 = score(doc=534,freq=1.0), product of:
             0.50935054 = queryWeight, product of:
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.066809446 = queryNorm
             4.7649565 = fieldWeight in 534, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.625 = fieldNorm(doc=534)
       0.5 = coord(1/2)
   

4. Peters, T.A.: When smart people fail : an analysis of the transaction log of an online public access catalog (1989) 1.21

   1.2135166 = sum of:
     1.2135166 = product of:
       2.4270332 = sum of:
         2.4270332 = weight(author_txt:peters in 2282) [ClassicSimilarity], result of:
           2.4270332 = score(doc=2282,freq=1.0), product of:
             0.50935054 = queryWeight, product of:
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.066809446 = queryNorm
             4.7649565 = fieldWeight in 2282, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.625 = fieldNorm(doc=2282)
       0.5 = coord(1/2)
   

5. Peters, C.M.: CD-ROM and optical technology : the user interface (1988) 1.21

   1.2135166 = sum of:
     1.2135166 = product of:
       2.4270332 = sum of:
         2.4270332 = weight(author_txt:peters in 4012) [ClassicSimilarity], result of:
           2.4270332 = score(doc=4012,freq=1.0), product of:
             0.50935054 = queryWeight, product of:
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.066809446 = queryNorm
             4.7649565 = fieldWeight in 4012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.62393 = idf(docFreq=58, maxDocs=44421)
               0.625 = fieldNorm(doc=4012)
       0.5 = coord(1/2)
   

Similar documents (content)

1. García Cumbreras, M.A.; Perea-Ortega, J.M.; García Vega, M.; Ureña López, L.A.: Information retrieval with geographical references : relevant documents filtering vs. query expansion (2009) 0.20

   0.19649455 = sum of:
     0.19649455 = product of:
       0.8187273 = sum of:
         0.025560625 = weight(abstract_txt:systems in 222) [ClassicSimilarity], result of:
           0.025560625 = score(doc=222,freq=1.0), product of:
             0.07992749 = queryWeight, product of:
               1.4757006 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.01587793 = queryNorm
             0.31979766 = fieldWeight in 222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.09375 = fieldNorm(doc=222)
         0.1064175 = weight(abstract_txt:forum in 222) [ClassicSimilarity], result of:
           0.1064175 = score(doc=222,freq=1.0), product of:
             0.1641776 = queryWeight, product of:
               1.4955196 = boost
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.01587793 = queryNorm
             0.64818525 = fieldWeight in 222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.09375 = fieldNorm(doc=222)
         0.04672788 = weight(abstract_txt:language in 222) [ClassicSimilarity], result of:
           0.04672788 = score(doc=222,freq=1.0), product of:
             0.11949946 = queryWeight, product of:
               1.8044014 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01587793 = queryNorm
             0.39103007 = fieldWeight in 222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.09375 = fieldNorm(doc=222)
         0.113159426 = weight(abstract_txt:cross in 222) [ClassicSimilarity], result of:
           0.113159426 = score(doc=222,freq=1.0), product of:
             0.21549757 = queryWeight, product of:
               2.4231002 = boost
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.01587793 = queryNorm
             0.5251077 = fieldWeight in 222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.09375 = fieldNorm(doc=222)
         0.4110865 = weight(abstract_txt:clef in 222) [ClassicSimilarity], result of:
           0.4110865 = score(doc=222,freq=1.0), product of:
             0.5092574 = queryWeight, product of:
               3.7249362 = boost
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.01587793 = queryNorm
             0.8072274 = fieldWeight in 222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.09375 = fieldNorm(doc=222)
         0.11577543 = weight(abstract_txt:evaluation in 222) [ClassicSimilarity], result of:
           0.11577543 = score(doc=222,freq=1.0), product of:
             0.27567843 = queryWeight, product of:
               3.8758466 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.01587793 = queryNorm
             0.4199655 = fieldWeight in 222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=222)
       0.24 = coord(6/25)
   

2. Kluck, M.; Mandl, T.; Womser-Hacker, C.: Cross-Language Evaluation Forum (CLEF) : Europäische Initiative zur Bewertung sprachübergreifender Retrievalverfahren (2002) 0.19

   0.19268887 = sum of:
     0.19268887 = product of:
       0.9634443 = sum of:
         0.1064175 = weight(abstract_txt:forum in 1266) [ClassicSimilarity], result of:
           0.1064175 = score(doc=1266,freq=1.0), product of:
             0.1641776 = queryWeight, product of:
               1.4955196 = boost
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.01587793 = queryNorm
             0.64818525 = fieldWeight in 1266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.09375 = fieldNorm(doc=1266)
         0.04672788 = weight(abstract_txt:language in 1266) [ClassicSimilarity], result of:
           0.04672788 = score(doc=1266,freq=1.0), product of:
             0.11949946 = queryWeight, product of:
               1.8044014 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01587793 = queryNorm
             0.39103007 = fieldWeight in 1266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.09375 = fieldNorm(doc=1266)
         0.113159426 = weight(abstract_txt:cross in 1266) [ClassicSimilarity], result of:
           0.113159426 = score(doc=1266,freq=1.0), product of:
             0.21549757 = queryWeight, product of:
               2.4231002 = boost
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.01587793 = queryNorm
             0.5251077 = fieldWeight in 1266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.09375 = fieldNorm(doc=1266)
         0.5813641 = weight(abstract_txt:clef in 1266) [ClassicSimilarity], result of:
           0.5813641 = score(doc=1266,freq=2.0), product of:
             0.5092574 = queryWeight, product of:
               3.7249362 = boost
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.01587793 = queryNorm
             1.1415919 = fieldWeight in 1266, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.09375 = fieldNorm(doc=1266)
         0.11577543 = weight(abstract_txt:evaluation in 1266) [ClassicSimilarity], result of:
           0.11577543 = score(doc=1266,freq=1.0), product of:
             0.27567843 = queryWeight, product of:
               3.8758466 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.01587793 = queryNorm
             0.4199655 = fieldWeight in 1266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=1266)
       0.2 = coord(5/25)
   

3. Scheufen, J.-H.: RECOIN: Modell offener Schnittstellen für Information-Retrieval-Systeme und -Komponenten (2006) 0.19

   0.18507223 = sum of:
     0.18507223 = product of:
       0.92536116 = sum of:
         0.12415374 = weight(abstract_txt:forum in 5901) [ClassicSimilarity], result of:
           0.12415374 = score(doc=5901,freq=1.0), product of:
             0.1641776 = queryWeight, product of:
               1.4955196 = boost
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.01587793 = queryNorm
             0.7562161 = fieldWeight in 5901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.109375 = fieldNorm(doc=5901)
         0.054515865 = weight(abstract_txt:language in 5901) [ClassicSimilarity], result of:
           0.054515865 = score(doc=5901,freq=1.0), product of:
             0.11949946 = queryWeight, product of:
               1.8044014 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01587793 = queryNorm
             0.45620176 = fieldWeight in 5901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.109375 = fieldNorm(doc=5901)
         0.13201933 = weight(abstract_txt:cross in 5901) [ClassicSimilarity], result of:
           0.13201933 = score(doc=5901,freq=1.0), product of:
             0.21549757 = queryWeight, product of:
               2.4231002 = boost
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.01587793 = queryNorm
             0.6126256 = fieldWeight in 5901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.109375 = fieldNorm(doc=5901)
         0.4796009 = weight(abstract_txt:clef in 5901) [ClassicSimilarity], result of:
           0.4796009 = score(doc=5901,freq=1.0), product of:
             0.5092574 = queryWeight, product of:
               3.7249362 = boost
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.01587793 = queryNorm
             0.94176525 = fieldWeight in 5901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.109375 = fieldNorm(doc=5901)
         0.13507134 = weight(abstract_txt:evaluation in 5901) [ClassicSimilarity], result of:
           0.13507134 = score(doc=5901,freq=1.0), product of:
             0.27567843 = queryWeight, product of:
               3.8758466 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.01587793 = queryNorm
             0.48995975 = fieldWeight in 5901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.109375 = fieldNorm(doc=5901)
       0.2 = coord(5/25)
   

4. Effektive Information Retrieval Verfahren in Theorie und Praxis : ausgewählte und erweiterte Beiträge des Vierten Hildesheimer Evaluierungs- und Retrievalworkshop (HIER 2005), Hildesheim, 20.7.2005 (2006) 0.16

   0.15863334 = sum of:
     0.15863334 = product of:
       0.7931667 = sum of:
         0.1064175 = weight(abstract_txt:forum in 973) [ClassicSimilarity], result of:
           0.1064175 = score(doc=973,freq=1.0), product of:
             0.1641776 = queryWeight, product of:
               1.4955196 = boost
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.01587793 = queryNorm
             0.64818525 = fieldWeight in 973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.09375 = fieldNorm(doc=973)
         0.04672788 = weight(abstract_txt:language in 973) [ClassicSimilarity], result of:
           0.04672788 = score(doc=973,freq=1.0), product of:
             0.11949946 = queryWeight, product of:
               1.8044014 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01587793 = queryNorm
             0.39103007 = fieldWeight in 973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.09375 = fieldNorm(doc=973)
         0.113159426 = weight(abstract_txt:cross in 973) [ClassicSimilarity], result of:
           0.113159426 = score(doc=973,freq=1.0), product of:
             0.21549757 = queryWeight, product of:
               2.4231002 = boost
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.01587793 = queryNorm
             0.5251077 = fieldWeight in 973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.09375 = fieldNorm(doc=973)
         0.4110865 = weight(abstract_txt:clef in 973) [ClassicSimilarity], result of:
           0.4110865 = score(doc=973,freq=1.0), product of:
             0.5092574 = queryWeight, product of:
               3.7249362 = boost
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.01587793 = queryNorm
             0.8072274 = fieldWeight in 973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.610425 = idf(docFreq=21, maxDocs=44421)
               0.09375 = fieldNorm(doc=973)
         0.11577543 = weight(abstract_txt:evaluation in 973) [ClassicSimilarity], result of:
           0.11577543 = score(doc=973,freq=1.0), product of:
             0.27567843 = queryWeight, product of:
               3.8758466 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.01587793 = queryNorm
             0.4199655 = fieldWeight in 973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=973)
       0.2 = coord(5/25)
   

5. Fujita, S.: NTCIR-2 as a Rosetta stone in laboratory experiments of IR systems (2005) 0.12

   0.122574374 = sum of:
     0.122574374 = product of:
       0.5107266 = sum of:
         0.05847638 = weight(abstract_txt:difficulties in 2017) [ClassicSimilarity], result of:
           0.05847638 = score(doc=2017,freq=1.0), product of:
             0.1243788 = queryWeight, product of:
               1.3016924 = boost
               6.017888 = idf(docFreq=293, maxDocs=44421)
               0.01587793 = queryNorm
             0.4701475 = fieldWeight in 2017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.017888 = idf(docFreq=293, maxDocs=44421)
               0.078125 = fieldNorm(doc=2017)
         0.02130052 = weight(abstract_txt:systems in 2017) [ClassicSimilarity], result of:
           0.02130052 = score(doc=2017,freq=1.0), product of:
             0.07992749 = queryWeight, product of:
               1.4757006 = boost
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.01587793 = queryNorm
             0.26649806 = fieldWeight in 2017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.411175 = idf(docFreq=3984, maxDocs=44421)
               0.078125 = fieldNorm(doc=2017)
         0.05506934 = weight(abstract_txt:language in 2017) [ClassicSimilarity], result of:
           0.05506934 = score(doc=2017,freq=2.0), product of:
             0.11949946 = queryWeight, product of:
               1.8044014 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.01587793 = queryNorm
             0.46083337 = fieldWeight in 2017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.078125 = fieldNorm(doc=2017)
         0.106078036 = weight(abstract_txt:languages in 2017) [ClassicSimilarity], result of:
           0.106078036 = score(doc=2017,freq=2.0), product of:
             0.18500197 = queryWeight, product of:
               2.2451138 = boost
               5.189722 = idf(docFreq=672, maxDocs=44421)
               0.01587793 = queryNorm
             0.5733887 = fieldWeight in 2017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.189722 = idf(docFreq=672, maxDocs=44421)
               0.078125 = fieldNorm(doc=2017)
         0.13335967 = weight(abstract_txt:cross in 2017) [ClassicSimilarity], result of:
           0.13335967 = score(doc=2017,freq=2.0), product of:
             0.21549757 = queryWeight, product of:
               2.4231002 = boost
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.01587793 = queryNorm
             0.61884534 = fieldWeight in 2017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.6011486 = idf(docFreq=445, maxDocs=44421)
               0.078125 = fieldNorm(doc=2017)
         0.13644265 = weight(abstract_txt:evaluation in 2017) [ClassicSimilarity], result of:
           0.13644265 = score(doc=2017,freq=2.0), product of:
             0.27567843 = queryWeight, product of:
               3.8758466 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.01587793 = queryNorm
             0.49493405 = fieldWeight in 2017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.078125 = fieldNorm(doc=2017)
       0.24 = coord(6/25)