Document (#36541)

Author

Chu, H.

Title

Factors affecting relevance judgment : a report from TREC Legal track

Source

Journal of documentation. 67(2011) no.2, S.264-278

Year

2011

Abstract

Purpose - This study intends to identify factors that affect relevance judgment of retrieved information as part of the 2007 TREC Legal track interactive task. Design/methodology/approach - Data were gathered and analyzed from the participants of the 2007 TREC Legal track interactive task using a questionnaire which includes not only a list of 80 relevance factors identified in prior research, but also a space for expressing their thoughts on relevance judgment in the process. Findings - This study finds that topicality remains a primary criterion, out of various options, for determining relevance, while specificity of the search request, task, or retrieved results also helps greatly in relevance judgment. Research limitations/implications - Relevance research should focus on the topicality and specificity of what is being evaluated as well as conducted in real environments. Practical implications - If multiple relevance factors are presented to assessors, the total number in a list should be below ten to take account of the limited processing capacity of human beings' short-term memory. Otherwise, the assessors might either completely ignore or inadequately consider some of the relevance factors when making judgment decisions. Originality/value - This study presents a method for reducing the artificiality of relevance research design, an apparent limitation in many related studies. Specifically, relevance judgment was made in this research as part of the 2007 TREC Legal track interactive task rather than a study devised for the sake of it. The assessors also served as searchers so that their searching experience would facilitate their subsequent relevance judgments.

Theme

Retrievalstudien

Object

TREC

Similar documents (content)

1. Ruthven, I.; Baillie, M.; Elsweiler, D.: ¬The relative effects of knowledge, interest and confidence in assessing relevance (2007) 0.52

   0.5150083 = sum of:
     0.5150083 = product of:
       0.99040055 = sum of:
         0.014129783 = weight(abstract_txt:should in 1835) [ClassicSimilarity], result of:
           0.014129783 = score(doc=1835,freq=1.0), product of:
             0.058308315 = queryWeight, product of:
               1.0719479 = boost
               4.4311547 = idf(docFreq=1436, maxDocs=44421)
               0.01227552 = queryNorm
             0.24232878 = fieldWeight in 1835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4311547 = idf(docFreq=1436, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.025310313 = weight(abstract_txt:implications in 1835) [ClassicSimilarity], result of:
           0.025310313 = score(doc=1835,freq=3.0), product of:
             0.059629995 = queryWeight, product of:
               1.0840288 = boost
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.01227552 = queryNorm
             0.42445606 = fieldWeight in 1835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.015612492 = weight(abstract_txt:part in 1835) [ClassicSimilarity], result of:
           0.015612492 = score(doc=1835,freq=1.0), product of:
             0.062319174 = queryWeight, product of:
               1.1082029 = boost
               4.581023 = idf(docFreq=1236, maxDocs=44421)
               0.01227552 = queryNorm
             0.2505247 = fieldWeight in 1835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.581023 = idf(docFreq=1236, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.013217696 = weight(abstract_txt:their in 1835) [ClassicSimilarity], result of:
           0.013217696 = score(doc=1835,freq=3.0), product of:
             0.044265702 = queryWeight, product of:
               1.1438992 = boost
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.01227552 = queryNorm
             0.29859903 = fieldWeight in 1835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.007837713 = weight(abstract_txt:this in 1835) [ClassicSimilarity], result of:
           0.007837713 = score(doc=1835,freq=3.0), product of:
             0.034387663 = queryWeight, product of:
               1.1641924 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.01227552 = queryNorm
             0.22792223 = fieldWeight in 1835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.0095322095 = weight(abstract_txt:also in 1835) [ClassicSimilarity], result of:
           0.0095322095 = score(doc=1835,freq=1.0), product of:
             0.051341236 = queryWeight, product of:
               1.2319335 = boost
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.01227552 = queryNorm
             0.18566382 = fieldWeight in 1835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.02587572 = weight(abstract_txt:study in 1835) [ClassicSimilarity], result of:
           0.02587572 = score(doc=1835,freq=4.0), product of:
             0.069272116 = queryWeight, product of:
               1.6523525 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01227552 = queryNorm
             0.3735373 = fieldWeight in 1835, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.022180649 = weight(abstract_txt:research in 1835) [ClassicSimilarity], result of:
           0.022180649 = score(doc=1835,freq=3.0), product of:
             0.074113265 = queryWeight, product of:
               1.9108492 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.01227552 = queryNorm
             0.2992804 = fieldWeight in 1835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.06980339 = weight(abstract_txt:factors in 1835) [ClassicSimilarity], result of:
           0.06980339 = score(doc=1835,freq=2.0), product of:
             0.18219139 = queryWeight, product of:
               2.9960039 = boost
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.01227552 = queryNorm
             0.38313222 = fieldWeight in 1835, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.2783896 = weight(abstract_txt:assessors in 1835) [ClassicSimilarity], result of:
           0.2783896 = score(doc=1835,freq=3.0), product of:
             0.33759722 = queryWeight, product of:
               3.1590283 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.01227552 = queryNorm
             0.82462054 = fieldWeight in 1835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.19574754 = weight(abstract_txt:trec in 1835) [ClassicSimilarity], result of:
           0.19574754 = score(doc=1835,freq=4.0), product of:
             0.26694864 = queryWeight, product of:
               3.243675 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.01227552 = queryNorm
             0.7332779 = fieldWeight in 1835, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.11059489 = weight(abstract_txt:track in 1835) [ClassicSimilarity], result of:
           0.11059489 = score(doc=1835,freq=1.0), product of:
             0.28960556 = queryWeight, product of:
               3.378523 = boost
               6.982969 = idf(docFreq=111, maxDocs=44421)
               0.01227552 = queryNorm
             0.38188112 = fieldWeight in 1835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.982969 = idf(docFreq=111, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
         0.20216858 = weight(abstract_txt:relevance in 1835) [ClassicSimilarity], result of:
           0.20216858 = score(doc=1835,freq=3.0), product of:
             0.4329715 = queryWeight, product of:
               7.1550674 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01227552 = queryNorm
             0.46693277 = fieldWeight in 1835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1835)
       0.52 = coord(13/25)
   

2. Xu, Y.C..; Chen, Z.: Relevance judgment : what do information users consider beyond topicality? (2006) 0.42

   0.41734034 = sum of:
     0.41734034 = product of:
       1.3041886 = sum of:
         0.025050713 = weight(abstract_txt:implications in 73) [ClassicSimilarity], result of:
           0.025050713 = score(doc=73,freq=1.0), product of:
             0.059629995 = queryWeight, product of:
               1.0840288 = boost
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.01227552 = queryNorm
             0.42010254 = fieldWeight in 73, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.010970513 = weight(abstract_txt:this in 73) [ClassicSimilarity], result of:
           0.010970513 = score(doc=73,freq=2.0), product of:
             0.034387663 = queryWeight, product of:
               1.1641924 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.01227552 = queryNorm
             0.31902468 = fieldWeight in 73, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.01634093 = weight(abstract_txt:also in 73) [ClassicSimilarity], result of:
           0.01634093 = score(doc=73,freq=1.0), product of:
             0.051341236 = queryWeight, product of:
               1.2319335 = boost
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.01227552 = queryNorm
             0.31828082 = fieldWeight in 73, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.031366106 = weight(abstract_txt:study in 73) [ClassicSimilarity], result of:
           0.031366106 = score(doc=73,freq=2.0), product of:
             0.069272116 = queryWeight, product of:
               1.6523525 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01227552 = queryNorm
             0.45279557 = fieldWeight in 73, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.2724885 = weight(abstract_txt:topicality in 73) [ClassicSimilarity], result of:
           0.2724885 = score(doc=73,freq=3.0), product of:
             0.20297524 = queryWeight, product of:
               2.0 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.01227552 = queryNorm
             1.3424716 = fieldWeight in 73, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.084614486 = weight(abstract_txt:factors in 73) [ClassicSimilarity], result of:
           0.084614486 = score(doc=73,freq=1.0), product of:
             0.18219139 = queryWeight, product of:
               2.9960039 = boost
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.01227552 = queryNorm
             0.46442637 = fieldWeight in 73, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.4631673 = weight(abstract_txt:judgment in 73) [ClassicSimilarity], result of:
           0.4631673 = score(doc=73,freq=2.0), product of:
             0.47728056 = queryWeight, product of:
               5.311974 = boost
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.01227552 = queryNorm
             0.9704299 = fieldWeight in 73, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
         0.40019006 = weight(abstract_txt:relevance in 73) [ClassicSimilarity], result of:
           0.40019006 = score(doc=73,freq=4.0), product of:
             0.4329715 = queryWeight, product of:
               7.1550674 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01227552 = queryNorm
             0.92428726 = fieldWeight in 73, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.09375 = fieldNorm(doc=73)
       0.32 = coord(8/25)
   

3. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.39

   0.39437455 = sum of:
     0.39437455 = product of:
       1.4084805 = sum of:
         0.008957386 = weight(abstract_txt:this in 3591) [ClassicSimilarity], result of:
           0.008957386 = score(doc=3591,freq=3.0), product of:
             0.034387663 = queryWeight, product of:
               1.1641924 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.01227552 = queryNorm
             0.26048255 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.025610318 = weight(abstract_txt:study in 3591) [ClassicSimilarity], result of:
           0.025610318 = score(doc=3591,freq=3.0), product of:
             0.069272116 = queryWeight, product of:
               1.6523525 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01227552 = queryNorm
             0.36970603 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.056409657 = weight(abstract_txt:factors in 3591) [ClassicSimilarity], result of:
           0.056409657 = score(doc=3591,freq=1.0), product of:
             0.18219139 = queryWeight, product of:
               2.9960039 = boost
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.01227552 = queryNorm
             0.30961758 = fieldWeight in 3591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9538813 = idf(docFreq=851, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.31815955 = weight(abstract_txt:assessors in 3591) [ClassicSimilarity], result of:
           0.31815955 = score(doc=3591,freq=3.0), product of:
             0.33759722 = queryWeight, product of:
               3.1590283 = boost
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.01227552 = queryNorm
             0.94242346 = fieldWeight in 3591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.705735 = idf(docFreq=19, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.1581879 = weight(abstract_txt:trec in 3591) [ClassicSimilarity], result of:
           0.1581879 = score(doc=3591,freq=2.0), product of:
             0.26694864 = queryWeight, product of:
               3.243675 = boost
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.01227552 = queryNorm
             0.592578 = fieldWeight in 3591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.704255 = idf(docFreq=147, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.48822126 = weight(abstract_txt:judgment in 3591) [ClassicSimilarity], result of:
           0.48822126 = score(doc=3591,freq=5.0), product of:
             0.47728056 = queryWeight, product of:
               5.311974 = boost
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.01227552 = queryNorm
             1.022923 = fieldWeight in 3591, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
         0.35293445 = weight(abstract_txt:relevance in 3591) [ClassicSimilarity], result of:
           0.35293445 = score(doc=3591,freq=7.0), product of:
             0.4329715 = queryWeight, product of:
               7.1550674 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01227552 = queryNorm
             0.8151447 = fieldWeight in 3591, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=3591)
       0.28 = coord(7/25)
   

4. Xu, Y.; Yin, H.: Novelty and topicality in interactive information retrieval (2008) 0.37

   0.37446654 = sum of:
     0.37446654 = product of:
       1.170208 = sum of:
         0.008957386 = weight(abstract_txt:this in 2355) [ClassicSimilarity], result of:
           0.008957386 = score(doc=2355,freq=3.0), product of:
             0.034387663 = queryWeight, product of:
               1.1641924 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.01227552 = queryNorm
             0.26048255 = fieldWeight in 2355, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.010893954 = weight(abstract_txt:also in 2355) [ClassicSimilarity], result of:
           0.010893954 = score(doc=2355,freq=1.0), product of:
             0.051341236 = queryWeight, product of:
               1.2319335 = boost
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.01227552 = queryNorm
             0.21218722 = fieldWeight in 2355, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.02957225 = weight(abstract_txt:study in 2355) [ClassicSimilarity], result of:
           0.02957225 = score(doc=2355,freq=4.0), product of:
             0.069272116 = queryWeight, product of:
               1.6523525 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01227552 = queryNorm
             0.42689976 = fieldWeight in 2355, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.014635433 = weight(abstract_txt:research in 2355) [ClassicSimilarity], result of:
           0.014635433 = score(doc=2355,freq=1.0), product of:
             0.074113265 = queryWeight, product of:
               1.9108492 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.01227552 = queryNorm
             0.19747387 = fieldWeight in 2355, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.20976174 = weight(abstract_txt:topicality in 2355) [ClassicSimilarity], result of:
           0.20976174 = score(doc=2355,freq=4.0), product of:
             0.20297524 = queryWeight, product of:
               2.0 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.01227552 = queryNorm
             1.0334351 = fieldWeight in 2355, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.047781 = weight(abstract_txt:interactive in 2355) [ClassicSimilarity], result of:
           0.047781 = score(doc=2355,freq=1.0), product of:
             0.13756654 = queryWeight, product of:
               2.016557 = boost
               5.557282 = idf(docFreq=465, maxDocs=44421)
               0.01227552 = queryNorm
             0.34733012 = fieldWeight in 2355, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.557282 = idf(docFreq=465, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.61755645 = weight(abstract_txt:judgment in 2355) [ClassicSimilarity], result of:
           0.61755645 = score(doc=2355,freq=8.0), product of:
             0.47728056 = queryWeight, product of:
               5.311974 = boost
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.01227552 = queryNorm
             1.2939066 = fieldWeight in 2355, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
         0.23104982 = weight(abstract_txt:relevance in 2355) [ClassicSimilarity], result of:
           0.23104982 = score(doc=2355,freq=3.0), product of:
             0.4329715 = queryWeight, product of:
               7.1550674 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01227552 = queryNorm
             0.53363746 = fieldWeight in 2355, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=2355)
       0.32 = coord(8/25)
   

5. Spink, A.; Greisdorf, H.: Regions and levels : Measuring and mapping users' relevance judgements (2001) 0.37

   0.37353635 = sum of:
     0.37353635 = product of:
       0.9338409 = sum of:
         0.016700475 = weight(abstract_txt:implications in 6586) [ClassicSimilarity], result of:
           0.016700475 = score(doc=6586,freq=1.0), product of:
             0.059629995 = queryWeight, product of:
               1.0840288 = boost
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.01227552 = queryNorm
             0.28006837 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481094 = idf(docFreq=1366, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.008721418 = weight(abstract_txt:their in 6586) [ClassicSimilarity], result of:
           0.008721418 = score(doc=6586,freq=1.0), product of:
             0.044265702 = queryWeight, product of:
               1.1438992 = boost
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.01227552 = queryNorm
             0.19702427 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1523883 = idf(docFreq=5161, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.0051715495 = weight(abstract_txt:this in 6586) [ClassicSimilarity], result of:
           0.0051715495 = score(doc=6586,freq=1.0), product of:
             0.034387663 = queryWeight, product of:
               1.1641924 = boost
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.01227552 = queryNorm
             0.15038967 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4062347 = idf(docFreq=10885, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.010893954 = weight(abstract_txt:also in 6586) [ClassicSimilarity], result of:
           0.010893954 = score(doc=6586,freq=1.0), product of:
             0.051341236 = queryWeight, product of:
               1.2319335 = boost
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.01227552 = queryNorm
             0.21218722 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3949955 = idf(docFreq=4049, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.034462538 = weight(abstract_txt:retrieved in 6586) [ClassicSimilarity], result of:
           0.034462538 = score(doc=6586,freq=1.0), product of:
             0.0966519 = queryWeight, product of:
               1.3801098 = boost
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.01227552 = queryNorm
             0.35656348 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7050157 = idf(docFreq=401, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.020910738 = weight(abstract_txt:study in 6586) [ClassicSimilarity], result of:
           0.020910738 = score(doc=6586,freq=2.0), product of:
             0.069272116 = queryWeight, product of:
               1.6523525 = boost
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.01227552 = queryNorm
             0.3018637 = fieldWeight in 6586, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.415198 = idf(docFreq=3968, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.014635433 = weight(abstract_txt:research in 6586) [ClassicSimilarity], result of:
           0.014635433 = score(doc=6586,freq=1.0), product of:
             0.074113265 = queryWeight, product of:
               1.9108492 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.01227552 = queryNorm
             0.19747387 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.10488087 = weight(abstract_txt:topicality in 6586) [ClassicSimilarity], result of:
           0.10488087 = score(doc=6586,freq=1.0), product of:
             0.20297524 = queryWeight, product of:
               2.0 = boost
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.01227552 = queryNorm
             0.51671755 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.267481 = idf(docFreq=30, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.21833917 = weight(abstract_txt:judgment in 6586) [ClassicSimilarity], result of:
           0.21833917 = score(doc=6586,freq=1.0), product of:
             0.47728056 = queryWeight, product of:
               5.311974 = boost
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.01227552 = queryNorm
             0.45746505 = fieldWeight in 6586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.319441 = idf(docFreq=79, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
         0.49912468 = weight(abstract_txt:relevance in 6586) [ClassicSimilarity], result of:
           0.49912468 = score(doc=6586,freq=14.0), product of:
             0.4329715 = queryWeight, product of:
               7.1550674 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.01227552 = queryNorm
             1.1527888 = fieldWeight in 6586, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=6586)
       0.4 = coord(10/25)