Document (#31762)

Editor

Voorhees, E.M. u. D.K. Harman

Title

TREC: experiment and evaluation in information retrieval

Imprint

Cambridge, MA : MIT Press

Year

2005

Pages

X, 462 S

Isbn

0-262-22073-3

Series

Digital libraries and electronic publishing

Abstract

The Text REtrieval Conference (TREC), a yearly workshop hosted by the US government's National Institute of Standards and Technology, provides the infrastructure necessary for large-scale evaluation of text retrieval methodologies. With the goal of accelerating research in this area, TREC created the first large test collections of full-text documents and standardized retrieval evaluation. The impact has been significant; since TREC's beginning in 1992, retrieval effectiveness has approximately doubled. TREC has built a variety of large test collections, including collections for such specialized retrieval tasks as cross-language retrieval and retrieval of speech. Moreover, TREC has accelerated the transfer of research ideas into commercial systems, as demonstrated in the number of retrieval techniques developed in TREC that are now used in Web search engines. This book provides a comprehensive review of TREC research, summarizing the variety of TREC results, documenting the best practices in experimental information retrieval, and suggesting areas for further research. The first part of the book describes TREC's history, test collections, and retrieval methodology. Next, the book provides "track" reports -- describing the evaluations of specific tasks, including routing and filtering, interactive retrieval, and retrieving noisy text. The final part of the book offers perspectives on TREC from such participants as Microsoft Research, University of Massachusetts, Cornell University, University of Waterloo, City University of New York, and IBM. The book will be of interest to researchers in information retrieval and related technologies, including natural language processing.

Content

Enthält die Beiträge: 1. The Text REtrieval Conference - Ellen M. Voorhees and Donna K. Harman 2. The TREC Test Collections - Donna K. Harman 3. Retrieval System Evaluation - Chris Buckley and Ellen M. Voorhees 4. The TREC Ad Hoc Experiments - Donna K. Harman 5. Routing and Filtering - Stephen Robertson and Jamie Callan 6. The TREC Interactive Tracks: Putting the User into Search - Susan T. Dumais and Nicholas J. Belkin 7. Beyond English - Donna K. Harman 8. Retrieving Noisy Text - Ellen M. Voorhees and John S. Garofolo 9.The Very Large Collection and Web Tracks - David Hawking and Nick Craswell 10. Question Answering in TREC - Ellen M. Voorhees 11. The University of Massachusetts and a Dozen TRECs - James Allan, W. Bruce Croft and Jamie Callan 12. How Okapi Came to TREC - Stephen Robertson 13. The SMART Project at TREC - Chris Buckley 14. Ten Years of Ad Hoc Retrieval at TREC Using PIRCS - Kui-Lam Kwok 15. MultiText Experiments for TREC - Gordon V. Cormack, Charles L. A. Clarke, Christopher R. Palmer and Thomas R. Lynam 16. A Language-Modeling Approach to TREC - Djoerd Hiemstra and Wessel Kraaij 17. BM Research Activities at TREC - Eric W. Brown, David Carmel, Martin Franz, Abraham Ittycheriah, Tapas Kanungo, Yoelle Maarek, J. Scott McCarley, Robert L. Mack, John M. Prager, John R. Smith, Aya Soffer, Jason Y. Zien and Alan D. Marwick Epilogue: Metareflections on TREC - Karen Sparck Jones

Footnote

Rez. in: JASIST 58(2007) no.6, S.910-911 (J.L. Vicedo u. J. Gomez): "The Text REtrieval Conference (TREC) is a yearly workshop hosted by the U.S. government's National Institute of Standards and Technology (NIST) that fosters and supports research in information retrieval as well as speeding the transfer of technology between research labs and industry. Since 1992, TREC has provided the infrastructure necessary for large-scale evaluations of different text retrieval methodologies. TREC impact has been very important and its success has been mainly supported by its continuous adaptation to the emerging information retrieval needs. Not in vain, TREC has built evaluation benchmarks for more than 20 different retrieval problems such as Web retrieval, speech retrieval, or question-answering. The large and intense trajectory of annual TREC conferences has resulted in an immense bulk of documents reflecting the different eval uation and research efforts developed. This situation makes it difficult sometimes to observe clearly how research in information retrieval (IR) has evolved over the course of TREC. TREC: Experiment and Evaluation in Information Retrieval succeeds in organizing and condensing all this research into a manageable volume that describes TREC history and summarizes the main lessons learned. The book is organized into three parts. The first part is devoted to the description of TREC's origin and history, the test collections, and the evaluation methodology developed. The second part describes a selection of the major evaluation exercises (tracks), and the third part contains contributions from research groups that had a large and remarkable participation in TREC. Finally, Karen Spark Jones, one of the main promoters of research in IR, closes the book with an epilogue that analyzes the impact of TREC on this research field.
... TREC: Experiment and Evaluation in Information Retrieval is a reliable and comprehensive review of the TREC program and has been adopted by NIST as the official history of TREC (see http://trec.nist.gov). We were favorably surprised by the book. Well structured and written, chapters are self-contained and the existence of references to specialized and more detailed publications is continuous, which makes it easier to expand into the different aspects analyzed in the text. This book succeeds in compiling TREC evolution from its inception in 1992 to 2003 in an adequate and manageable volume. Thanks to the impressive effort performed by the authors and their experience in the field, it can satiate the interests of a great variety of readers. While expert researchers in the IR field and IR-related industrial companies can use it as a reference manual, it seems especially useful for students and non-expert readers willing to approach this research area. Like NIST, we would recommend this reading to anyone who may be interested in textual information retrieval."

Theme

Retrievalstudien

Object

TREC

LCSH

Text processing (Computer science) / Congresses
Information storage and retrieval systems / Congresses
Text REtrieval Conference

RSWK

Information Retrieval / Textverarbeitung / Aufsatzsammlung (BVB)
Kongress / Information Retrieval / Kongress (GBV)

BK

06.74 / Informationssysteme
54.75 / Sprachverarbeitung <Informatik>

DDC

025.04 / dc22
005.74 (SWB)

LCC

QA76.9.T48T74 2005

RVK

ST 270

Similar documents (content)

1. Voorhees, E.M.; Harman, D.K.: ¬The Text REtrieval Conference (2005) 0.83

   0.833055 = sum of:
     0.833055 = product of:
       1.388425 = sum of:
         0.022181878 = weight(abstract_txt:part in 82) [ClassicSimilarity], result of:
           0.022181878 = score(doc=82,freq=3.0), product of:
             0.071567394 = queryWeight, product of:
               1.0109944 = boost
               4.581023 = idf(docFreq=1236, maxDocs=44421)
               0.015452685 = queryNorm
             0.3099439 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.581023 = idf(docFreq=1236, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.055303033 = weight(abstract_txt:doubled in 82) [ClassicSimilarity], result of:
           0.055303033 = score(doc=82,freq=1.0), product of:
             0.15063041 = queryWeight, product of:
               1.0371283 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.015452685 = queryNorm
             0.36714387 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.018169712 = weight(abstract_txt:variety in 82) [ClassicSimilarity], result of:
           0.018169712 = score(doc=82,freq=1.0), product of:
             0.09036261 = queryWeight, product of:
               1.1360189 = boost
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.015452685 = queryNorm
             0.20107555 = fieldWeight in 82, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.03157578 = weight(abstract_txt:tasks in 82) [ClassicSimilarity], result of:
           0.03157578 = score(doc=82,freq=3.0), product of:
             0.09056333 = queryWeight, product of:
               1.1372799 = boost
               5.1532483 = idf(docFreq=697, maxDocs=44421)
               0.015452685 = queryNorm
             0.3486597 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.1532483 = idf(docFreq=697, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.039738502 = weight(abstract_txt:conference in 82) [ClassicSimilarity], result of:
           0.039738502 = score(doc=82,freq=3.0), product of:
             0.10556597 = queryWeight, product of:
               1.2278721 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015452685 = queryNorm
             0.37643287 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.030341765 = weight(abstract_txt:large in 82) [ClassicSimilarity], result of:
           0.030341765 = score(doc=82,freq=3.0), product of:
             0.100950226 = queryWeight, product of:
               1.4705861 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.015452685 = queryNorm
             0.30056164 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.031112086 = weight(abstract_txt:evaluation in 82) [ClassicSimilarity], result of:
           0.031112086 = score(doc=82,freq=3.0), product of:
             0.1026517 = queryWeight, product of:
               1.4829273 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015452685 = queryNorm
             0.303084 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.012647656 = weight(abstract_txt:information in 82) [ClassicSimilarity], result of:
           0.012647656 = score(doc=82,freq=5.0), product of:
             0.059861436 = queryWeight, product of:
               1.6014942 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.015452685 = queryNorm
             0.2112822 = fieldWeight in 82, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.036308248 = weight(abstract_txt:test in 82) [ClassicSimilarity], result of:
           0.036308248 = score(doc=82,freq=2.0), product of:
             0.13025087 = queryWeight, product of:
               1.6704255 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.015452685 = queryNorm
             0.27875626 = fieldWeight in 82, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.018194485 = weight(abstract_txt:research in 82) [ClassicSimilarity], result of:
           0.018194485 = score(doc=82,freq=3.0), product of:
             0.085111745 = queryWeight, product of:
               1.7432349 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.015452685 = queryNorm
             0.21377173 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.039056543 = weight(abstract_txt:collections in 82) [ClassicSimilarity], result of:
           0.039056543 = score(doc=82,freq=2.0), product of:
             0.15050559 = queryWeight, product of:
               2.073397 = boost
               4.6974936 = idf(docFreq=1100, maxDocs=44421)
               0.015452685 = queryNorm
             0.2595023 = fieldWeight in 82, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6974936 = idf(docFreq=1100, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.045673117 = weight(abstract_txt:text in 82) [ClassicSimilarity], result of:
           0.045673117 = score(doc=82,freq=3.0), product of:
             0.16705668 = queryWeight, product of:
               2.675369 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.015452685 = queryNorm
             0.27339894 = fieldWeight in 82, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.08530732 = weight(abstract_txt:book in 82) [ClassicSimilarity], result of:
           0.08530732 = score(doc=82,freq=5.0), product of:
             0.20109637 = queryWeight, product of:
               2.6795588 = boost
               4.8566523 = idf(docFreq=938, maxDocs=44421)
               0.015452685 = queryNorm
             0.42421114 = fieldWeight in 82, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.8566523 = idf(docFreq=938, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.17801635 = weight(abstract_txt:retrieval in 82) [ClassicSimilarity], result of:
           0.17801635 = score(doc=82,freq=14.0), product of:
             0.35034314 = queryWeight, product of:
               6.521499 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.015452685 = queryNorm
             0.50812 = fieldWeight in 82, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
         0.7447985 = weight(abstract_txt:trec in 82) [ClassicSimilarity], result of:
           0.7447985 = score(doc=82,freq=17.0), product of:
             0.68976915 = queryWeight, product of:
               6.6580844 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.015452685 = queryNorm
             1.0797794 = fieldWeight in 82, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0390625 = fieldNorm(doc=82)
       0.6 = coord(15/25)
   

2. Harman, D.: ¬The Text REtrieval Conferences (TRECs) : providing a test-bed for information retrieval systems (1998) 0.56

   0.556924 = sum of:
     0.556924 = product of:
       1.3923099 = sum of:
         0.04360731 = weight(abstract_txt:variety in 2314) [ClassicSimilarity], result of:
           0.04360731 = score(doc=2314,freq=1.0), product of:
             0.09036261 = queryWeight, product of:
               1.1360189 = boost
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.015452685 = queryNorm
             0.48258135 = fieldWeight in 2314, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.09537241 = weight(abstract_txt:conference in 2314) [ClassicSimilarity], result of:
           0.09537241 = score(doc=2314,freq=3.0), product of:
             0.10556597 = queryWeight, product of:
               1.2278721 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015452685 = queryNorm
             0.9034389 = fieldWeight in 2314, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.039222963 = weight(abstract_txt:including in 2314) [ClassicSimilarity], result of:
           0.039222963 = score(doc=2314,freq=1.0), product of:
             0.09638436 = queryWeight, product of:
               1.4369448 = boost
               4.340728 = idf(docFreq=1572, maxDocs=44421)
               0.015452685 = queryNorm
             0.40694323 = fieldWeight in 2314, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.340728 = idf(docFreq=1572, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.05945748 = weight(abstract_txt:large in 2314) [ClassicSimilarity], result of:
           0.05945748 = score(doc=2314,freq=2.0), product of:
             0.100950226 = queryWeight, product of:
               1.4705861 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.015452685 = queryNorm
             0.5889782 = fieldWeight in 2314, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.01919779 = weight(abstract_txt:information in 2314) [ClassicSimilarity], result of:
           0.01919779 = score(doc=2314,freq=2.0), product of:
             0.059861436 = queryWeight, product of:
               1.6014942 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.015452685 = queryNorm
             0.3207038 = fieldWeight in 2314, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.061617136 = weight(abstract_txt:test in 2314) [ClassicSimilarity], result of:
           0.061617136 = score(doc=2314,freq=1.0), product of:
             0.13025087 = queryWeight, product of:
               1.6704255 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.015452685 = queryNorm
             0.47306505 = fieldWeight in 2314, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.035653766 = weight(abstract_txt:research in 2314) [ClassicSimilarity], result of:
           0.035653766 = score(doc=2314,freq=2.0), product of:
             0.085111745 = queryWeight, product of:
               1.7432349 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.015452685 = queryNorm
             0.41890535 = fieldWeight in 2314, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.08950067 = weight(abstract_txt:text in 2314) [ClassicSimilarity], result of:
           0.08950067 = score(doc=2314,freq=2.0), product of:
             0.16705668 = queryWeight, product of:
               2.675369 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.015452685 = queryNorm
             0.5357503 = fieldWeight in 2314, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.19777332 = weight(abstract_txt:retrieval in 2314) [ClassicSimilarity], result of:
           0.19777332 = score(doc=2314,freq=3.0), product of:
             0.35034314 = queryWeight, product of:
               6.521499 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.015452685 = queryNorm
             0.5645132 = fieldWeight in 2314, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
         0.75090706 = weight(abstract_txt:trec in 2314) [ClassicSimilarity], result of:
           0.75090706 = score(doc=2314,freq=3.0), product of:
             0.68976915 = queryWeight, product of:
               6.6580844 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.015452685 = queryNorm
             1.0886353 = fieldWeight in 2314, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.09375 = fieldNorm(doc=2314)
       0.4 = coord(10/25)
   

3. Emerging frameworks and methods : Proceedings of the Fourth International Conference on the Conceptions of Library and Information Science (CoLIS4), Seattle, WA, July 21 - 25, 2002 (2002) 0.44

   0.4429843 = sum of:
     0.4429843 = product of:
       1.2305119 = sum of:
         0.025461178 = weight(abstract_txt:processing in 1055) [ClassicSimilarity], result of:
           0.025461178 = score(doc=1055,freq=1.0), product of:
             0.08271718 = queryWeight, product of:
               1.0868986 = boost
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.015452685 = queryNorm
             0.30781004 = fieldWeight in 1055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9249606 = idf(docFreq=876, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
         0.041113365 = weight(abstract_txt:variety in 1055) [ClassicSimilarity], result of:
           0.041113365 = score(doc=1055,freq=2.0), product of:
             0.09036261 = queryWeight, product of:
               1.1360189 = boost
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.015452685 = queryNorm
             0.45498204 = fieldWeight in 1055, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
         0.036708858 = weight(abstract_txt:conference in 1055) [ClassicSimilarity], result of:
           0.036708858 = score(doc=1055,freq=1.0), product of:
             0.10556597 = queryWeight, product of:
               1.2278721 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015452685 = queryNorm
             0.3477338 = fieldWeight in 1055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
         0.023882015 = weight(abstract_txt:provides in 1055) [ClassicSimilarity], result of:
           0.023882015 = score(doc=1055,freq=1.0), product of:
             0.09073074 = queryWeight, product of:
               1.3941646 = boost
               4.211497 = idf(docFreq=1789, maxDocs=44421)
               0.015452685 = queryNorm
             0.26321855 = fieldWeight in 1055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.211497 = idf(docFreq=1789, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
         0.025597053 = weight(abstract_txt:information in 1055) [ClassicSimilarity], result of:
           0.025597053 = score(doc=1055,freq=8.0), product of:
             0.059861436 = queryWeight, product of:
               1.6014942 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.015452685 = queryNorm
             0.42760506 = fieldWeight in 1055, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
         0.32608262 = weight(subject_txt:kongress in 1055) [ClassicSimilarity], result of:
           0.32608262 = score(doc=1055,freq=1.0), product of:
             0.17968914 = queryWeight, product of:
               1.6019615 = boost
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.015452685 = queryNorm
             1.8147041 = fieldWeight in 1055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.2588162 = idf(docFreq=84, maxDocs=44421)
               0.25 = fieldNorm(doc=1055)
         0.023769177 = weight(abstract_txt:research in 1055) [ClassicSimilarity], result of:
           0.023769177 = score(doc=1055,freq=2.0), product of:
             0.085111745 = queryWeight, product of:
               1.7432349 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.015452685 = queryNorm
             0.27927023 = fieldWeight in 1055, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
         0.6517747 = weight(subject_txt:congresses in 1055) [ClassicSimilarity], result of:
           0.6517747 = score(doc=1055,freq=2.0), product of:
             0.22630374 = queryWeight, product of:
               1.797782 = boost
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.015452685 = queryNorm
             2.880088 = fieldWeight in 1055, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.25 = fieldNorm(doc=1055)
         0.076123 = weight(abstract_txt:retrieval in 1055) [ClassicSimilarity], result of:
           0.076123 = score(doc=1055,freq=1.0), product of:
             0.35034314 = queryWeight, product of:
               6.521499 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.015452685 = queryNorm
             0.21728125 = fieldWeight in 1055, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.0625 = fieldNorm(doc=1055)
       0.36 = coord(9/25)
   

4. Harman, D.: Overview of the first Text Retrieval Conference (1993) 0.40

   0.4033606 = sum of:
     0.4033606 = product of:
       1.0084015 = sum of:
         0.04360731 = weight(abstract_txt:variety in 616) [ClassicSimilarity], result of:
           0.04360731 = score(doc=616,freq=1.0), product of:
             0.09036261 = queryWeight, product of:
               1.1360189 = boost
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.015452685 = queryNorm
             0.48258135 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1475344 = idf(docFreq=701, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.07787125 = weight(abstract_txt:conference in 616) [ClassicSimilarity], result of:
           0.07787125 = score(doc=616,freq=2.0), product of:
             0.10556597 = queryWeight, product of:
               1.2278721 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015452685 = queryNorm
             0.7376548 = fieldWeight in 616, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.039222963 = weight(abstract_txt:including in 616) [ClassicSimilarity], result of:
           0.039222963 = score(doc=616,freq=1.0), product of:
             0.09638436 = queryWeight, product of:
               1.4369448 = boost
               4.340728 = idf(docFreq=1572, maxDocs=44421)
               0.015452685 = queryNorm
             0.40694323 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.340728 = idf(docFreq=1572, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.05945748 = weight(abstract_txt:large in 616) [ClassicSimilarity], result of:
           0.05945748 = score(doc=616,freq=2.0), product of:
             0.100950226 = queryWeight, product of:
               1.4705861 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.015452685 = queryNorm
             0.5889782 = fieldWeight in 616, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.043110173 = weight(abstract_txt:evaluation in 616) [ClassicSimilarity], result of:
           0.043110173 = score(doc=616,freq=1.0), product of:
             0.1026517 = queryWeight, product of:
               1.4829273 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015452685 = queryNorm
             0.4199655 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.061617136 = weight(abstract_txt:test in 616) [ClassicSimilarity], result of:
           0.061617136 = score(doc=616,freq=1.0), product of:
             0.13025087 = queryWeight, product of:
               1.6704255 = boost
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.015452685 = queryNorm
             0.47306505 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046027 = idf(docFreq=776, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.025211018 = weight(abstract_txt:research in 616) [ClassicSimilarity], result of:
           0.025211018 = score(doc=616,freq=1.0), product of:
             0.085111745 = queryWeight, product of:
               1.7432349 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.015452685 = queryNorm
             0.2962108 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.063286535 = weight(abstract_txt:text in 616) [ClassicSimilarity], result of:
           0.063286535 = score(doc=616,freq=1.0), product of:
             0.16705668 = queryWeight, product of:
               2.675369 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.015452685 = queryNorm
             0.3788327 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.16148126 = weight(abstract_txt:retrieval in 616) [ClassicSimilarity], result of:
           0.16148126 = score(doc=616,freq=2.0), product of:
             0.35034314 = queryWeight, product of:
               6.521499 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.015452685 = queryNorm
             0.46092314 = fieldWeight in 616, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
         0.43353644 = weight(abstract_txt:trec in 616) [ClassicSimilarity], result of:
           0.43353644 = score(doc=616,freq=1.0), product of:
             0.68976915 = queryWeight, product of:
               6.6580844 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.015452685 = queryNorm
             0.62852395 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.09375 = fieldNorm(doc=616)
       0.4 = coord(10/25)
   

5. Voorhees, E.M.: Text REtrieval Conference (TREC) (2009) 0.40

   0.3959141 = sum of:
     0.3959141 = product of:
       1.4139789 = sum of:
         0.09177214 = weight(abstract_txt:conference in 877) [ClassicSimilarity], result of:
           0.09177214 = score(doc=877,freq=1.0), product of:
             0.10556597 = queryWeight, product of:
               1.2278721 = boost
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.015452685 = queryNorm
             0.86933446 = fieldWeight in 877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5637407 = idf(docFreq=462, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
         0.07007131 = weight(abstract_txt:large in 877) [ClassicSimilarity], result of:
           0.07007131 = score(doc=877,freq=1.0), product of:
             0.100950226 = queryWeight, product of:
               1.4705861 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.015452685 = queryNorm
             0.6941174 = fieldWeight in 877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
         0.071850285 = weight(abstract_txt:evaluation in 877) [ClassicSimilarity], result of:
           0.071850285 = score(doc=877,freq=1.0), product of:
             0.1026517 = queryWeight, product of:
               1.4829273 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.015452685 = queryNorm
             0.69994247 = fieldWeight in 877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
         0.022624813 = weight(abstract_txt:information in 877) [ClassicSimilarity], result of:
           0.022624813 = score(doc=877,freq=1.0), product of:
             0.059861436 = queryWeight, product of:
               1.6014942 = boost
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.015452685 = queryNorm
             0.37795305 = fieldWeight in 877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4188995 = idf(docFreq=10748, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
         0.10547756 = weight(abstract_txt:text in 877) [ClassicSimilarity], result of:
           0.10547756 = score(doc=877,freq=1.0), product of:
             0.16705668 = queryWeight, product of:
               2.675369 = boost
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.015452685 = queryNorm
             0.6313878 = fieldWeight in 877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.040882 = idf(docFreq=2122, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
         0.32962224 = weight(abstract_txt:retrieval in 877) [ClassicSimilarity], result of:
           0.32962224 = score(doc=877,freq=3.0), product of:
             0.35034314 = queryWeight, product of:
               6.521499 = boost
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.015452685 = queryNorm
             0.9408554 = fieldWeight in 877, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4765 = idf(docFreq=3732, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
         0.72256064 = weight(abstract_txt:trec in 877) [ClassicSimilarity], result of:
           0.72256064 = score(doc=877,freq=1.0), product of:
             0.68976915 = queryWeight, product of:
               6.6580844 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.015452685 = queryNorm
             1.0475398 = fieldWeight in 877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.15625 = fieldNorm(doc=877)
       0.28 = coord(7/25)