Document (#32836)

Author

Ruthven, I.
Baillie, M.
Elsweiler, D.

Title

¬The relative effects of knowledge, interest and confidence in assessing relevance

Source

Journal of documentation. 63(2007) no.4, S.482-504

Year

2007

Abstract

Purpose - The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant. Design/methodology/approach - The study was conducted as part of the Text REtrieval Conference (TREC) HARD track. Using a specially constructed questionnaire information was sought on TREC assessors' personal context and, using the TREC assessments gathered, the responses were correlated to the questionnaire questions and the final relevance decisions. Findings - This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant. Research limitations/implications - The major limitation is that the research is conducted within the TREC initiative. This means that we can report on results but cannot report on discussions with the assessors. The research implications are numerous but mainly on the effect of personal context on the outcomes of a user study. Practical implications - One major consequence is that we should take more account of how we construct search tasks for IIR evaluation to create tasks that are interesting and relevant to experimental subjects. Originality/value - Examining different search variables within one study to compare the relative effects on these variables on the search outcomes.

Theme

Retrievalstudien

Similar documents (author)

1. Elsweiler, D.; Ruthven, I.; Jones, C.: Towards memory supporting personal information management tools (2007) 2.45

   2.4473505 = sum of:
     2.4473505 = product of:
       3.6710258 = sum of:
         1.3508693 = weight(author_txt:ruthven in 57) [ClassicSimilarity], result of:
           1.3508693 = score(doc=57,freq=1.0), product of:
             0.4422128 = queryWeight, product of:
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.054285087 = queryNorm
             3.0547948 = fieldWeight in 57, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.375 = fieldNorm(doc=57)
         2.3201563 = weight(author_txt:elsweiler in 57) [ClassicSimilarity], result of:
           2.3201563 = score(doc=57,freq=1.0), product of:
             0.63421124 = queryWeight, product of:
               1.1975712 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.054285087 = queryNorm
             3.6583338 = fieldWeight in 57, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.375 = fieldNorm(doc=57)
       0.6666667 = coord(2/3)
   

2. Baillie, M.; Azzopardi, L.; Ruthven, I.: Evaluating epistemic uncertainty under incomplete assessments (2008) 2.45

   2.4473505 = sum of:
     2.4473505 = product of:
       3.6710258 = sum of:
         1.3508693 = weight(author_txt:ruthven in 3065) [ClassicSimilarity], result of:
           1.3508693 = score(doc=3065,freq=1.0), product of:
             0.4422128 = queryWeight, product of:
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.054285087 = queryNorm
             3.0547948 = fieldWeight in 3065, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.375 = fieldNorm(doc=3065)
         2.3201563 = weight(author_txt:baillie in 3065) [ClassicSimilarity], result of:
           2.3201563 = score(doc=3065,freq=1.0), product of:
             0.63421124 = queryWeight, product of:
               1.1975712 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.054285087 = queryNorm
             3.6583338 = fieldWeight in 3065, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.375 = fieldNorm(doc=3065)
       0.6666667 = coord(2/3)
   

3. Ruthven, I.; Baillie, M.; Azzopardi, L.; Bierig, R.; Nicol, E.; Sweeney, S.; Yaciki, M.: Contextual factors affecting the utility of surrogates within exploratory search (2008) 1.63

   1.631567 = sum of:
     1.631567 = product of:
       2.4473505 = sum of:
         0.9005795 = weight(author_txt:ruthven in 3042) [ClassicSimilarity], result of:
           0.9005795 = score(doc=3042,freq=1.0), product of:
             0.4422128 = queryWeight, product of:
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.054285087 = queryNorm
             2.0365298 = fieldWeight in 3042, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.146119 = idf(docFreq=34, maxDocs=44421)
               0.25 = fieldNorm(doc=3042)
         1.546771 = weight(author_txt:baillie in 3042) [ClassicSimilarity], result of:
           1.546771 = score(doc=3042,freq=1.0), product of:
             0.63421124 = queryWeight, product of:
               1.1975712 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.054285087 = queryNorm
             2.4388893 = fieldWeight in 3042, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.25 = fieldNorm(doc=3042)
       0.6666667 = coord(2/3)
   

4. Elsweiler, D.; Harvey, M.: Engaging and maintaining a sense of being informed : understanding the tasks motivating twitter search (2015) 1.03

   1.0311807 = sum of:
     1.0311807 = product of:
       3.093542 = sum of:
         3.093542 = weight(author_txt:elsweiler in 2635) [ClassicSimilarity], result of:
           3.093542 = score(doc=2635,freq=1.0), product of:
             0.63421124 = queryWeight, product of:
               1.1975712 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.054285087 = queryNorm
             4.8777785 = fieldWeight in 2635, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.5 = fieldNorm(doc=2635)
       0.33333334 = coord(1/3)
   

5. Elsweiler, D.; Hammwöhner, R.: Information Behavior : ein zentrales Forschungsthema der Informationswissenschaft (2015) 1.03

   1.0311807 = sum of:
     1.0311807 = product of:
       3.093542 = sum of:
         3.093542 = weight(author_txt:elsweiler in 2713) [ClassicSimilarity], result of:
           3.093542 = score(doc=2713,freq=1.0), product of:
             0.63421124 = queryWeight, product of:
               1.1975712 = boost
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.054285087 = queryNorm
             4.8777785 = fieldWeight in 2713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.755557 = idf(docFreq=6, maxDocs=44421)
               0.5 = fieldNorm(doc=2713)
       0.33333334 = coord(1/3)
   

Similar documents (content)

1. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.50

   0.495701 = sum of:
     0.495701 = product of:
       1.2392524 = sum of:
         0.019475674 = weight(abstract_txt:were in 540) [ClassicSimilarity], result of:
           0.019475674 = score(doc=540,freq=1.0), product of:
             0.0849658 = queryWeight, product of:
               1.0341451 = boost
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.02240239 = queryNorm
             0.2292178 = fieldWeight in 540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6674848 = idf(docFreq=3083, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.015074905 = weight(abstract_txt:that in 540) [ClassicSimilarity], result of:
           0.015074905 = score(doc=540,freq=3.0), product of:
             0.058883652 = queryWeight, product of:
               1.1114275 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.02240239 = queryNorm
             0.25601172 = fieldWeight in 540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.064009294 = weight(abstract_txt:questionnaire in 540) [ClassicSimilarity], result of:
           0.064009294 = score(doc=540,freq=1.0), product of:
             0.16407691 = queryWeight, product of:
               1.1733768 = boost
               6.2418823 = idf(docFreq=234, maxDocs=44421)
               0.02240239 = queryNorm
             0.39011765 = fieldWeight in 540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2418823 = idf(docFreq=234, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.05024058 = weight(abstract_txt:implications in 540) [ClassicSimilarity], result of:
           0.05024058 = score(doc=540,freq=2.0), product of:
             0.12684566 = queryWeight, product of:
               1.2635639 = boost
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.02240239 = queryNorm
             0.39607647 = fieldWeight in 540, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.041937646 = weight(abstract_txt:study in 540) [ClassicSimilarity], result of:
           0.041937646 = score(doc=540,freq=4.0), product of:
             0.098237686 = queryWeight, product of:
               1.2840083 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.02240239 = queryNorm
             0.42689976 = fieldWeight in 540, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.08130883 = weight(abstract_txt:affect in 540) [ClassicSimilarity], result of:
           0.08130883 = score(doc=540,freq=1.0), product of:
             0.22029652 = queryWeight, product of:
               1.6651877 = boost
               5.90541 = idf(docFreq=328, maxDocs=44421)
               0.02240239 = queryNorm
             0.3690881 = fieldWeight in 540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90541 = idf(docFreq=328, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.032118488 = weight(abstract_txt:search in 540) [ClassicSimilarity], result of:
           0.032118488 = score(doc=540,freq=1.0), product of:
             0.14061624 = queryWeight, product of:
               1.7175188 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.02240239 = queryNorm
             0.22841237 = fieldWeight in 540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.20914127 = weight(abstract_txt:relevance in 540) [ClassicSimilarity], result of:
           0.20914127 = score(doc=540,freq=11.0), product of:
             0.20467164 = queryWeight, product of:
               1.8533506 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.02240239 = queryNorm
             1.0218381 = fieldWeight in 540, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.45119527 = weight(abstract_txt:assessors in 540) [ClassicSimilarity], result of:
           0.45119527 = score(doc=540,freq=3.0), product of:
             0.47876066 = queryWeight, product of:
               2.4548142 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.02240239 = queryNorm
             0.94242346 = fieldWeight in 540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
         0.2747505 = weight(abstract_txt:trec in 540) [ClassicSimilarity], result of:
           0.2747505 = score(doc=540,freq=3.0), product of:
             0.378571 = queryWeight, product of:
               2.5205913 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.02240239 = queryNorm
             0.7257569 = fieldWeight in 540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=540)
       0.4 = coord(10/25)
   

2. Ruthven, I.: Relevance behaviour in TREC (2014) 0.39

   0.38516584 = sum of:
     0.38516584 = product of:
       1.2036433 = sum of:
         0.023027299 = weight(abstract_txt:that in 2785) [ClassicSimilarity], result of:
           0.023027299 = score(doc=2785,freq=7.0), product of:
             0.058883652 = queryWeight, product of:
               1.1114275 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.02240239 = queryNorm
             0.39106438 = fieldWeight in 2785, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.035525456 = weight(abstract_txt:implications in 2785) [ClassicSimilarity], result of:
           0.035525456 = score(doc=2785,freq=1.0), product of:
             0.12684566 = queryWeight, product of:
               1.2635639 = boost
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.02240239 = queryNorm
             0.28006837 = fieldWeight in 2785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.052463073 = weight(abstract_txt:interest in 2785) [ClassicSimilarity], result of:
           0.052463073 = score(doc=2785,freq=1.0), product of:
             0.1644946 = queryWeight, product of:
               1.4389153 = boost
               5.1029587 = idf(docFreq=733, maxDocs=44421)
               0.02240239 = queryNorm
             0.31893492 = fieldWeight in 2785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1029587 = idf(docFreq=733, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.11498804 = weight(abstract_txt:affect in 2785) [ClassicSimilarity], result of:
           0.11498804 = score(doc=2785,freq=2.0), product of:
             0.22029652 = queryWeight, product of:
               1.6651877 = boost
               5.90541 = idf(docFreq=328, maxDocs=44421)
               0.02240239 = queryNorm
             0.5219694 = fieldWeight in 2785, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.90541 = idf(docFreq=328, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.032118488 = weight(abstract_txt:search in 2785) [ClassicSimilarity], result of:
           0.032118488 = score(doc=2785,freq=1.0), product of:
             0.14061624 = queryWeight, product of:
               1.7175188 = boost
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.02240239 = queryNorm
             0.22841237 = fieldWeight in 2785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.654598 = idf(docFreq=3123, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.20914127 = weight(abstract_txt:relevance in 2785) [ClassicSimilarity], result of:
           0.20914127 = score(doc=2785,freq=11.0), product of:
             0.20467164 = queryWeight, product of:
               1.8533506 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.02240239 = queryNorm
             1.0218381 = fieldWeight in 2785, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.26049772 = weight(abstract_txt:assessors in 2785) [ClassicSimilarity], result of:
           0.26049772 = score(doc=2785,freq=1.0), product of:
             0.47876066 = queryWeight, product of:
               2.4548142 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.02240239 = queryNorm
             0.54410845 = fieldWeight in 2785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
         0.47588187 = weight(abstract_txt:trec in 2785) [ClassicSimilarity], result of:
           0.47588187 = score(doc=2785,freq=9.0), product of:
             0.378571 = queryWeight, product of:
               2.5205913 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.02240239 = queryNorm
             1.2570479 = fieldWeight in 2785, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=2785)
       0.32 = coord(8/25)
   

3. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.28

   0.28385493 = sum of:
     0.28385493 = product of:
       1.0137676 = sum of:
         0.015074905 = weight(abstract_txt:that in 3591) [ClassicSimilarity], result of:
           0.015074905 = score(doc=3591,freq=3.0), product of:
             0.058883652 = queryWeight, product of:
               1.1114275 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.02240239 = queryNorm
             0.25601172 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.047939107 = weight(abstract_txt:documents in 3591) [ClassicSimilarity], result of:
           0.047939107 = score(doc=3591,freq=3.0), product of:
             0.10739945 = queryWeight, product of:
               1.1626806 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.02240239 = queryNorm
             0.4463627 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.036319066 = weight(abstract_txt:study in 3591) [ClassicSimilarity], result of:
           0.036319066 = score(doc=3591,freq=3.0), product of:
             0.098237686 = queryWeight, product of:
               1.2840083 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.02240239 = queryNorm
             0.36970603 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.072069466 = weight(abstract_txt:topic in 3591) [ClassicSimilarity], result of:
           0.072069466 = score(doc=3591,freq=2.0), product of:
             0.16133925 = queryWeight, product of:
               1.4250478 = boost
               5.053779 = idf(docFreq=770, maxDocs=44421)
               0.02240239 = queryNorm
             0.44669518 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.053779 = idf(docFreq=770, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.166837 = weight(abstract_txt:relevance in 3591) [ClassicSimilarity], result of:
           0.166837 = score(doc=3591,freq=7.0), product of:
             0.20467164 = queryWeight, product of:
               1.8533506 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.02240239 = queryNorm
             0.8151447 = fieldWeight in 3591, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.45119527 = weight(abstract_txt:assessors in 3591) [ClassicSimilarity], result of:
           0.45119527 = score(doc=3591,freq=3.0), product of:
             0.47876066 = queryWeight, product of:
               2.4548142 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.02240239 = queryNorm
             0.94242346 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.22433284 = weight(abstract_txt:trec in 3591) [ClassicSimilarity], result of:
           0.22433284 = score(doc=3591,freq=2.0), product of:
             0.378571 = queryWeight, product of:
               2.5205913 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.02240239 = queryNorm
             0.592578 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
       0.28 = coord(7/25)
   

4. Osman, D.J.; Yearwood, J.; Vamplew, P.: Automated opinion detection : implications of the level of agreement between human raters (2010) 0.26

   0.25976276 = sum of:
     0.25976276 = product of:
       1.0823449 = sum of:
         0.045278452 = weight(abstract_txt:ability in 232) [ClassicSimilarity], result of:
           0.045278452 = score(doc=232,freq=1.0), product of:
             0.13026062 = queryWeight, product of:
               1.0454911 = boost
               5.561583 = idf(docFreq=463, maxDocs=44421)
               0.02240239 = queryNorm
             0.34759894 = fieldWeight in 232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.561583 = idf(docFreq=463, maxDocs=44421)
               0.0625 = fieldNorm(doc=232)
         0.01230861 = weight(abstract_txt:that in 232) [ClassicSimilarity], result of:
           0.01230861 = score(doc=232,freq=2.0), product of:
             0.058883652 = queryWeight, product of:
               1.1114275 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.02240239 = queryNorm
             0.20903271 = fieldWeight in 232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.0625 = fieldNorm(doc=232)
         0.07837093 = weight(abstract_txt:assessing in 232) [ClassicSimilarity], result of:
           0.07837093 = score(doc=232,freq=1.0), product of:
             0.18778263 = queryWeight, product of:
               1.2552825 = boost
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.02240239 = queryNorm
             0.4173492 = fieldWeight in 232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.0625 = fieldNorm(doc=232)
         0.05096081 = weight(abstract_txt:topic in 232) [ClassicSimilarity], result of:
           0.05096081 = score(doc=232,freq=1.0), product of:
             0.16133925 = queryWeight, product of:
               1.4250478 = boost
               5.053779 = idf(docFreq=770, maxDocs=44421)
               0.02240239 = queryNorm
             0.3158612 = fieldWeight in 232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.053779 = idf(docFreq=770, maxDocs=44421)
               0.0625 = fieldNorm(doc=232)
         0.7367988 = weight(abstract_txt:assessors in 232) [ClassicSimilarity], result of:
           0.7367988 = score(doc=232,freq=8.0), product of:
             0.47876066 = queryWeight, product of:
               2.4548142 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.02240239 = queryNorm
             1.5389711 = fieldWeight in 232, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=232)
         0.15862729 = weight(abstract_txt:trec in 232) [ClassicSimilarity], result of:
           0.15862729 = score(doc=232,freq=1.0), product of:
             0.378571 = queryWeight, product of:
               2.5205913 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.02240239 = queryNorm
             0.41901594 = fieldWeight in 232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=232)
       0.24 = coord(6/25)
   

5. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 0.25

   0.25045344 = sum of:
     0.25045344 = product of:
       0.78266704 = sum of:
         0.018843632 = weight(abstract_txt:that in 4145) [ClassicSimilarity], result of:
           0.018843632 = score(doc=4145,freq=3.0), product of:
             0.058883652 = queryWeight, product of:
               1.1114275 = boost
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.02240239 = queryNorm
             0.32001466 = fieldWeight in 4145, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3649352 = idf(docFreq=11344, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.03459707 = weight(abstract_txt:documents in 4145) [ClassicSimilarity], result of:
           0.03459707 = score(doc=4145,freq=1.0), product of:
             0.10739945 = queryWeight, product of:
               1.1626806 = boost
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.02240239 = queryNorm
             0.32213452 = fieldWeight in 4145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.123322 = idf(docFreq=1954, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.040151533 = weight(abstract_txt:context in 4145) [ClassicSimilarity], result of:
           0.040151533 = score(doc=4145,freq=1.0), product of:
             0.118607074 = queryWeight, product of:
               1.221841 = boost
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.02240239 = queryNorm
             0.33852562 = fieldWeight in 4145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.333128 = idf(docFreq=1584, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.09796366 = weight(abstract_txt:assessing in 4145) [ClassicSimilarity], result of:
           0.09796366 = score(doc=4145,freq=1.0), product of:
             0.18778263 = queryWeight, product of:
               1.2552825 = boost
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.02240239 = queryNorm
             0.5216865 = fieldWeight in 4145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.677587 = idf(docFreq=151, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.026211029 = weight(abstract_txt:study in 4145) [ClassicSimilarity], result of:
           0.026211029 = score(doc=4145,freq=1.0), product of:
             0.098237686 = queryWeight, product of:
               1.2840083 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.02240239 = queryNorm
             0.26681235 = fieldWeight in 4145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.15764615 = weight(abstract_txt:relevance in 4145) [ClassicSimilarity], result of:
           0.15764615 = score(doc=4145,freq=4.0), product of:
             0.20467164 = queryWeight, product of:
               1.8533506 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.02240239 = queryNorm
             0.77023935 = fieldWeight in 4145, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.08163183 = weight(abstract_txt:relevant in 4145) [ClassicSimilarity], result of:
           0.08163183 = score(doc=4145,freq=1.0), product of:
             0.22568278 = queryWeight, product of:
               2.1758702 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.02240239 = queryNorm
             0.3617105 = fieldWeight in 4145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
         0.32562214 = weight(abstract_txt:assessors in 4145) [ClassicSimilarity], result of:
           0.32562214 = score(doc=4145,freq=1.0), product of:
             0.47876066 = queryWeight, product of:
               2.4548142 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.02240239 = queryNorm
             0.68013555 = fieldWeight in 4145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.078125 = fieldNorm(doc=4145)
       0.32 = coord(8/25)