Document (#39592)

Author
Ravana, S.D.
Rajagopal, P.
Balakrishnan, V.
Title
Ranking retrieval systems using pseudo relevance judgments
Source
Aslib journal of information management. 67(2015) no.6, S.700-714
Year
2015
Abstract
Purpose In a system-based approach, replicating the web would require large test collections, and judging the relevancy of all documents per topic in creating relevance judgment through human assessors is infeasible. Due to the large amount of documents that requires judgment, there are possible errors introduced by human assessors because of disagreements. The paper aims to discuss these issues. Design/methodology/approach This study explores exponential variation and document ranking methods that generate a reliable set of relevance judgments (pseudo relevance judgments) to reduce human efforts. These methods overcome problems with large amounts of documents for judgment while avoiding human disagreement errors during the judgment process. This study utilizes two key factors: number of occurrences of each document per topic from all the system runs; and document rankings to generate the alternate methods. Findings The effectiveness of the proposed method is evaluated using the correlation coefficient of ranked systems using mean average precision scores between the original Text REtrieval Conference (TREC) relevance judgments and pseudo relevance judgments. The results suggest that the proposed document ranking method with a pool depth of 100 could be a reliable alternative to reduce human effort and disagreement errors involved in generating TREC-like relevance judgments. Originality/value Simple methods proposed in this study show improvement in the correlation coefficient in generating alternate relevance judgment without human assessors while contributing to information retrieval evaluation.
Content
Vgl.: http://dx.doi.org/10.1108/AJIM-03-2015-0046.
Theme
Retrievalalgorithmen

Similar documents (author)

  1. Balakrishnan, U.: DFG-Projekt: Coli-conc : das Mapping Tool "Cocoda" (2016) 5.81 
    5.814733 = sum of:
  5.814733 = weight(author_txt:balakrishnan in 4036) [ClassicSimilarity], result of:
    5.814733 = fieldWeight in 4036, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.303573 = idf(docFreq=10, maxDocs=44421)
      0.625 = fieldNorm(doc=4036)
  2. Balakrishnan, U.; Voß, J.: ¬The Cocoda mapping tool (2015) 4.65 
    4.6517863 = sum of:
  4.6517863 = weight(author_txt:balakrishnan in 205) [ClassicSimilarity], result of:
    4.6517863 = fieldWeight in 205, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.303573 = idf(docFreq=10, maxDocs=44421)
      0.5 = fieldNorm(doc=205)
  3. Balakrishnan, U.; Krausz, A,; Voss, J.: Cocoda - ein Konkordanztool für bibliothekarische Klassifikationssysteme (2015) 3.49 
    3.4888396 = sum of:
  3.4888396 = weight(author_txt:balakrishnan in 3030) [ClassicSimilarity], result of:
    3.4888396 = fieldWeight in 3030, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.303573 = idf(docFreq=10, maxDocs=44421)
      0.375 = fieldNorm(doc=3030)
  4. Balakrishnan, V.; Ahmadi, K.; Ravana, S.D.: Improving retrieval relevance using users' explicit feedback (2016) 3.49 
    3.4888396 = sum of:
  3.4888396 = weight(author_txt:balakrishnan in 3921) [ClassicSimilarity], result of:
    3.4888396 = fieldWeight in 3921, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.303573 = idf(docFreq=10, maxDocs=44421)
      0.375 = fieldNorm(doc=3921)
  5. Balakrishnan, U.; Voß, J.; Soergel, D.: Towards integrated systems for KOS management, mapping, and access : Coli-conc and its collaborative computer-assisted KOS mapping tool Cocoda (2018) 3.49 
    3.4888396 = sum of:
  3.4888396 = weight(author_txt:balakrishnan in 825) [ClassicSimilarity], result of:
    3.4888396 = fieldWeight in 825, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.303573 = idf(docFreq=10, maxDocs=44421)
      0.375 = fieldNorm(doc=825)

Similar documents (content)

  1. Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019) 0.46 
    0.4574003 = sum of:
  0.4574003 = product of:
    1.0395461 = sum of:
      0.011123553 = weight(abstract_txt:study in 287) [ClassicSimilarity], result of:
        0.011123553 = score(doc=287,freq=1.0), product of:
          0.052113183 = queryWeight, product of:
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.015259198 = queryNorm
          0.21344988 = fieldWeight in 287, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.019980479 = weight(abstract_txt:using in 287) [ClassicSimilarity], result of:
        0.019980479 = score(doc=287,freq=3.0), product of:
          0.053392727 = queryWeight, product of:
            1.0122021 = boost
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.015259198 = queryNorm
          0.37421724 = fieldWeight in 287, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.020322783 = weight(abstract_txt:retrieval in 287) [ClassicSimilarity], result of:
        0.020322783 = score(doc=287,freq=3.0), product of:
          0.054000814 = queryWeight, product of:
            1.0179498 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.015259198 = queryNorm
          0.37634215 = fieldWeight in 287, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.039153233 = weight(abstract_txt:documents in 287) [ClassicSimilarity], result of:
        0.039153233 = score(doc=287,freq=4.0), product of:
          0.07596444 = queryWeight, product of:
            1.207345 = boost
            4.123322 = idf(docFreq=1954, maxDocs=44421)
            0.015259198 = queryNorm
          0.51541525 = fieldWeight in 287, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.123322 = idf(docFreq=1954, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.05768885 = weight(abstract_txt:generating in 287) [ClassicSimilarity], result of:
        0.05768885 = score(doc=287,freq=1.0), product of:
          0.13640055 = queryWeight, product of:
            1.3209568 = boost
            6.7669935 = idf(docFreq=138, maxDocs=44421)
            0.015259198 = queryNorm
          0.4229371 = fieldWeight in 287, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.7669935 = idf(docFreq=138, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.02732443 = weight(abstract_txt:proposed in 287) [ClassicSimilarity], result of:
        0.02732443 = score(doc=287,freq=1.0), product of:
          0.094875194 = queryWeight, product of:
            1.3492814 = boost
            4.608063 = idf(docFreq=1203, maxDocs=44421)
            0.015259198 = queryNorm
          0.28800395 = fieldWeight in 287, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.608063 = idf(docFreq=1203, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.04169473 = weight(abstract_txt:document in 287) [ClassicSimilarity], result of:
        0.04169473 = score(doc=287,freq=2.0), product of:
          0.109852254 = queryWeight, product of:
            1.6764865 = boost
            4.29415 = idf(docFreq=1647, maxDocs=44421)
            0.015259198 = queryNorm
          0.3795528 = fieldWeight in 287, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.29415 = idf(docFreq=1647, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.057295278 = weight(abstract_txt:human in 287) [ClassicSimilarity], result of:
        0.057295278 = score(doc=287,freq=1.0), product of:
          0.19582787 = queryWeight, product of:
            2.7414382 = boost
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.015259198 = queryNorm
          0.2925798 = fieldWeight in 287, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.18250649 = weight(abstract_txt:judgment in 287) [ClassicSimilarity], result of:
        0.18250649 = score(doc=287,freq=1.0), product of:
          0.39895177 = queryWeight, product of:
            3.5719945 = boost
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.015259198 = queryNorm
          0.45746505 = fieldWeight in 287, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.21850295 = weight(abstract_txt:relevance in 287) [ClassicSimilarity], result of:
        0.21850295 = score(doc=287,freq=6.0), product of:
          0.2895316 = queryWeight, product of:
            3.8490942 = boost
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.015259198 = queryNorm
          0.7546774 = fieldWeight in 287, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
      0.36395335 = weight(abstract_txt:judgments in 287) [ClassicSimilarity], result of:
        0.36395335 = score(doc=287,freq=4.0), product of:
          0.42312863 = queryWeight, product of:
            4.0297437 = boost
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.015259198 = queryNorm
          0.86014825 = fieldWeight in 287, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.0625 = fieldNorm(doc=287)
    0.44 = coord(11/25)
  2. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.45 
    0.44585803 = sum of:
  0.44585803 = product of:
    1.3933064 = sum of:
      0.022247106 = weight(abstract_txt:study in 540) [ClassicSimilarity], result of:
        0.022247106 = score(doc=540,freq=4.0), product of:
          0.052113183 = queryWeight, product of:
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.015259198 = queryNorm
          0.42689976 = fieldWeight in 540, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.011535735 = weight(abstract_txt:using in 540) [ClassicSimilarity], result of:
        0.011535735 = score(doc=540,freq=1.0), product of:
          0.053392727 = queryWeight, product of:
            1.0122021 = boost
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.015259198 = queryNorm
          0.21605442 = fieldWeight in 540, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.09716655 = weight(abstract_txt:trec in 540) [ClassicSimilarity], result of:
        0.09716655 = score(doc=540,freq=3.0), product of:
          0.13388306 = queryWeight, product of:
            1.3087099 = boost
            6.704255 = idf(docFreq=147, maxDocs=44421)
            0.015259198 = queryNorm
          0.7257569 = fieldWeight in 540, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.704255 = idf(docFreq=147, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.31913382 = weight(abstract_txt:assessors in 540) [ClassicSimilarity], result of:
        0.31913382 = score(doc=540,freq=3.0), product of:
          0.33863103 = queryWeight, product of:
            2.549116 = boost
            8.705735 = idf(docFreq=19, maxDocs=44421)
            0.015259198 = queryNorm
          0.94242346 = fieldWeight in 540, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            8.705735 = idf(docFreq=19, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.057295278 = weight(abstract_txt:human in 540) [ClassicSimilarity], result of:
        0.057295278 = score(doc=540,freq=1.0), product of:
          0.19582787 = queryWeight, product of:
            2.7414382 = boost
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.015259198 = queryNorm
          0.2925798 = fieldWeight in 540, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.40809694 = weight(abstract_txt:judgment in 540) [ClassicSimilarity], result of:
        0.40809694 = score(doc=540,freq=5.0), product of:
          0.39895177 = queryWeight, product of:
            3.5719945 = boost
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.015259198 = queryNorm
          1.022923 = fieldWeight in 540, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.2958544 = weight(abstract_txt:relevance in 540) [ClassicSimilarity], result of:
        0.2958544 = score(doc=540,freq=11.0), product of:
          0.2895316 = queryWeight, product of:
            3.8490942 = boost
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.015259198 = queryNorm
          1.0218381 = fieldWeight in 540, product of:
            3.3166249 = tf(freq=11.0), with freq of:
              11.0 = termFreq=11.0
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
      0.18197668 = weight(abstract_txt:judgments in 540) [ClassicSimilarity], result of:
        0.18197668 = score(doc=540,freq=1.0), product of:
          0.42312863 = queryWeight, product of:
            4.0297437 = boost
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.015259198 = queryNorm
          0.43007413 = fieldWeight in 540, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.0625 = fieldNorm(doc=540)
    0.32 = coord(8/25)
  3. Losada, D.E.; Parapar, J.; Barreiro, A.: When to stop making relevance judgments? : a study of stopping methods for building information retrieval test collections (2019) 0.37 
    0.36942968 = sum of:
  0.36942968 = product of:
    0.9235742 = sum of:
      0.011535735 = weight(abstract_txt:using in 674) [ClassicSimilarity], result of:
        0.011535735 = score(doc=674,freq=1.0), product of:
          0.053392727 = queryWeight, product of:
            1.0122021 = boost
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.015259198 = queryNorm
          0.21605442 = fieldWeight in 674, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.011733364 = weight(abstract_txt:retrieval in 674) [ClassicSimilarity], result of:
        0.011733364 = score(doc=674,freq=1.0), product of:
          0.054000814 = queryWeight, product of:
            1.0179498 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.015259198 = queryNorm
          0.21728125 = fieldWeight in 674, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.033907693 = weight(abstract_txt:documents in 674) [ClassicSimilarity], result of:
        0.033907693 = score(doc=674,freq=3.0), product of:
          0.07596444 = queryWeight, product of:
            1.207345 = boost
            4.123322 = idf(docFreq=1954, maxDocs=44421)
            0.015259198 = queryNorm
          0.4463627 = fieldWeight in 674, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.123322 = idf(docFreq=1954, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.06537497 = weight(abstract_txt:reduce in 674) [ClassicSimilarity], result of:
        0.06537497 = score(doc=674,freq=2.0), product of:
          0.117675446 = queryWeight, product of:
            1.2269405 = boost
            6.285367 = idf(docFreq=224, maxDocs=44421)
            0.015259198 = queryNorm
          0.5555532 = fieldWeight in 674, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            6.285367 = idf(docFreq=224, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.056614786 = weight(abstract_txt:reliable in 674) [ClassicSimilarity], result of:
        0.056614786 = score(doc=674,freq=1.0), product of:
          0.13470222 = queryWeight, product of:
            1.3127074 = boost
            6.724734 = idf(docFreq=144, maxDocs=44421)
            0.015259198 = queryNorm
          0.42029586 = fieldWeight in 674, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.724734 = idf(docFreq=144, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.02732443 = weight(abstract_txt:proposed in 674) [ClassicSimilarity], result of:
        0.02732443 = score(doc=674,freq=1.0), product of:
          0.094875194 = queryWeight, product of:
            1.3492814 = boost
            4.608063 = idf(docFreq=1203, maxDocs=44421)
            0.015259198 = queryNorm
          0.28800395 = fieldWeight in 674, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.608063 = idf(docFreq=1203, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.064878926 = weight(abstract_txt:methods in 674) [ClassicSimilarity], result of:
        0.064878926 = score(doc=674,freq=6.0), product of:
          0.1022783 = queryWeight, product of:
            1.6176603 = boost
            4.1434727 = idf(docFreq=1915, maxDocs=44421)
            0.015259198 = queryNorm
          0.6343371 = fieldWeight in 674, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            4.1434727 = idf(docFreq=1915, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.18250649 = weight(abstract_txt:judgment in 674) [ClassicSimilarity], result of:
        0.18250649 = score(doc=674,freq=1.0), product of:
          0.39895177 = queryWeight, product of:
            3.5719945 = boost
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.015259198 = queryNorm
          0.45746505 = fieldWeight in 674, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.15450491 = weight(abstract_txt:relevance in 674) [ClassicSimilarity], result of:
        0.15450491 = score(doc=674,freq=3.0), product of:
          0.2895316 = queryWeight, product of:
            3.8490942 = boost
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.015259198 = queryNorm
          0.53363746 = fieldWeight in 674, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
      0.31519285 = weight(abstract_txt:judgments in 674) [ClassicSimilarity], result of:
        0.31519285 = score(doc=674,freq=3.0), product of:
          0.42312863 = queryWeight, product of:
            4.0297437 = boost
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.015259198 = queryNorm
          0.74491024 = fieldWeight in 674, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.0625 = fieldNorm(doc=674)
    0.4 = coord(10/25)
  4. Balatsoukas, P.; Ruthven, I.: ¬An eye-tracking approach to the analysis of relevance judgments on the Web : the case of Google search engine (2012) 0.30 
    0.29539573 = sum of:
  0.29539573 = product of:
    0.9231117 = sum of:
      0.011123553 = weight(abstract_txt:study in 1379) [ClassicSimilarity], result of:
        0.011123553 = score(doc=1379,freq=1.0), product of:
          0.052113183 = queryWeight, product of:
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.015259198 = queryNorm
          0.21344988 = fieldWeight in 1379, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.011535735 = weight(abstract_txt:using in 1379) [ClassicSimilarity], result of:
        0.011535735 = score(doc=1379,freq=1.0), product of:
          0.053392727 = queryWeight, product of:
            1.0122021 = boost
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.015259198 = queryNorm
          0.21605442 = fieldWeight in 1379, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.4568708 = idf(docFreq=3806, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.011733364 = weight(abstract_txt:retrieval in 1379) [ClassicSimilarity], result of:
        0.011733364 = score(doc=1379,freq=1.0), product of:
          0.054000814 = queryWeight, product of:
            1.0179498 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.015259198 = queryNorm
          0.21728125 = fieldWeight in 1379, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.048953753 = weight(abstract_txt:ranking in 1379) [ClassicSimilarity], result of:
        0.048953753 = score(doc=1379,freq=1.0), product of:
          0.13995098 = queryWeight, product of:
            1.6387556 = boost
            5.5966744 = idf(docFreq=447, maxDocs=44421)
            0.015259198 = queryNorm
          0.34979215 = fieldWeight in 1379, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.5966744 = idf(docFreq=447, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.057295278 = weight(abstract_txt:human in 1379) [ClassicSimilarity], result of:
        0.057295278 = score(doc=1379,freq=1.0), product of:
          0.19582787 = queryWeight, product of:
            2.7414382 = boost
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.015259198 = queryNorm
          0.2925798 = fieldWeight in 1379, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.18250649 = weight(abstract_txt:judgment in 1379) [ClassicSimilarity], result of:
        0.18250649 = score(doc=1379,freq=1.0), product of:
          0.39895177 = queryWeight, product of:
            3.5719945 = boost
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.015259198 = queryNorm
          0.45746505 = fieldWeight in 1379, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.319441 = idf(docFreq=79, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.23601015 = weight(abstract_txt:relevance in 1379) [ClassicSimilarity], result of:
        0.23601015 = score(doc=1379,freq=7.0), product of:
          0.2895316 = queryWeight, product of:
            3.8490942 = boost
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.015259198 = queryNorm
          0.8151447 = fieldWeight in 1379, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
      0.36395335 = weight(abstract_txt:judgments in 1379) [ClassicSimilarity], result of:
        0.36395335 = score(doc=1379,freq=4.0), product of:
          0.42312863 = queryWeight, product of:
            4.0297437 = boost
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.015259198 = queryNorm
          0.86014825 = fieldWeight in 1379, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.0625 = fieldNorm(doc=1379)
    0.32 = coord(8/25)
  5. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 0.29 
    0.29287323 = sum of:
  0.29287323 = product of:
    0.91522884 = sum of:
      0.013904441 = weight(abstract_txt:study in 4145) [ClassicSimilarity], result of:
        0.013904441 = score(doc=4145,freq=1.0), product of:
          0.052113183 = queryWeight, product of:
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.015259198 = queryNorm
          0.26681235 = fieldWeight in 4145, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.415198 = idf(docFreq=3968, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.020741854 = weight(abstract_txt:retrieval in 4145) [ClassicSimilarity], result of:
        0.020741854 = score(doc=4145,freq=2.0), product of:
          0.054000814 = queryWeight, product of:
            1.0179498 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.015259198 = queryNorm
          0.3841026 = fieldWeight in 4145, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.02447077 = weight(abstract_txt:documents in 4145) [ClassicSimilarity], result of:
        0.02447077 = score(doc=4145,freq=1.0), product of:
          0.07596444 = queryWeight, product of:
            1.207345 = boost
            4.123322 = idf(docFreq=1954, maxDocs=44421)
            0.015259198 = queryNorm
          0.32213452 = fieldWeight in 4145, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.123322 = idf(docFreq=1954, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.074032605 = weight(abstract_txt:methods in 4145) [ClassicSimilarity], result of:
        0.074032605 = score(doc=4145,freq=5.0), product of:
          0.1022783 = queryWeight, product of:
            1.6176603 = boost
            4.1434727 = idf(docFreq=1915, maxDocs=44421)
            0.015259198 = queryNorm
          0.72383493 = fieldWeight in 4145, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            4.1434727 = idf(docFreq=1915, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.230315 = weight(abstract_txt:assessors in 4145) [ClassicSimilarity], result of:
        0.230315 = score(doc=4145,freq=1.0), product of:
          0.33863103 = queryWeight, product of:
            2.549116 = boost
            8.705735 = idf(docFreq=19, maxDocs=44421)
            0.015259198 = queryNorm
          0.68013555 = fieldWeight in 4145, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.705735 = idf(docFreq=19, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.1012847 = weight(abstract_txt:human in 4145) [ClassicSimilarity], result of:
        0.1012847 = score(doc=4145,freq=2.0), product of:
          0.19582787 = queryWeight, product of:
            2.7414382 = boost
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.015259198 = queryNorm
          0.51721287 = fieldWeight in 4145, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.681277 = idf(docFreq=1118, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.22300862 = weight(abstract_txt:relevance in 4145) [ClassicSimilarity], result of:
        0.22300862 = score(doc=4145,freq=4.0), product of:
          0.2895316 = queryWeight, product of:
            3.8490942 = boost
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.015259198 = queryNorm
          0.77023935 = fieldWeight in 4145, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.929532 = idf(docFreq=872, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
      0.22747084 = weight(abstract_txt:judgments in 4145) [ClassicSimilarity], result of:
        0.22747084 = score(doc=4145,freq=1.0), product of:
          0.42312863 = queryWeight, product of:
            4.0297437 = boost
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.015259198 = queryNorm
          0.53759265 = fieldWeight in 4145, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.881186 = idf(docFreq=123, maxDocs=44421)
            0.078125 = fieldNorm(doc=4145)
    0.32 = coord(8/25)