Document (#44345)

Author
Keller, K.
Breuer, T.
Schaer, P.
Title
Evaluation of temporal change in IR test collections
Source
ICTIR '24: The 2024 ACM SIGIR International Conference on the Theory of Information Retrieval
Year
2024
Abstract
Information retrieval systems have been evaluated using the Cranfield paradigm for many years. This paradigm allows a systematic, fair, and reproducible evaluation of different retrieval methods in fixed experimental environments. However, real-world retrieval systems must cope with dynamic environments and temporal changes that affect the document collection, topical trends, and the individual user's perception of what is considered relevant. Yet, the temporal dimension in IR evaluations is still understudied. To this end, this work investigates how the temporal generalizability of effectiveness evaluations can be assessed. As a conceptual model, we generalize Cranfield-type experiments to the temporal context by classifying the change in the essential components according to the create, update, and delete operations of persistent storage known from CRUD. From the different types of change different evaluation scenarios are derived and it is outlined what they imply. Based on these scenarios, renowned state-of-the-art retrieval systems are tested and it is investigated how the retrieval effectiveness changes on different levels of granularity. We show that the proposed measures can be well adapted to describe the changes in the retrieval results. The experiments conducted confirm that the retrieval effectiveness strongly depends on the evaluation scenario investigated. We find that not only the average retrieval performance of single systems but also the relative system performance are strongly affected by the components that change and to what extent these components changed.
Content
DOI: 10.1145/3664190.3672530. Vgl.: https://www.researchgate.net/publication/382893247_Evaluation_of_Temporal_Change_in_IR_Test_Collections. Vgl. auch: https://www.researchgate.net/publication/381885150_Evaluation_of_Temporal_Change_in_IR_Test_Collections.
Theme
Retrievalstudien

Similar documents (author)

  1. Breuer, T.; Tavakolpoursaleh, N.; Schaer, P.; Hienert, D.; Schaible, J.; Castro, L.J.: Online Information Retrieval Evaluation using the STELLA Framework (2022) 1.90 
    1.9049907 = sum of:
  1.9049907 = product of:
    2.857486 = sum of:
      1.2257601 = weight(author_txt:schaer in 1641) [ClassicSimilarity], result of:
        1.2257601 = score(doc=1641,freq=1.0), product of:
          0.55287546 = queryWeight, product of:
            1.0550374 = boost
            8.868255 = idf(docFreq=16, maxDocs=44421)
            0.059090994 = queryNorm
          2.2170637 = fieldWeight in 1641, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.868255 = idf(docFreq=16, maxDocs=44421)
            0.25 = fieldNorm(doc=1641)
      1.6317259 = weight(author_txt:breuer in 1641) [ClassicSimilarity], result of:
        1.6317259 = score(doc=1641,freq=1.0), product of:
          0.6690447 = queryWeight, product of:
            1.1605978 = boost
            9.755557 = idf(docFreq=6, maxDocs=44421)
            0.059090994 = queryNorm
          2.4388893 = fieldWeight in 1641, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.755557 = idf(docFreq=6, maxDocs=44421)
            0.25 = fieldNorm(doc=1641)
    0.6666667 = coord(2/3)
  2. Breuer, H.: Gedankenschranken : Freier Wille (2003) 1.36 
    1.3597716 = sum of:
  1.3597716 = product of:
    4.0793147 = sum of:
      4.0793147 = weight(author_txt:breuer in 3361) [ClassicSimilarity], result of:
        4.0793147 = score(doc=3361,freq=1.0), product of:
          0.6690447 = queryWeight, product of:
            1.1605978 = boost
            9.755557 = idf(docFreq=6, maxDocs=44421)
            0.059090994 = queryNorm
          6.0972233 = fieldWeight in 3361, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.755557 = idf(docFreq=6, maxDocs=44421)
            0.625 = fieldNorm(doc=3361)
    0.33333334 = coord(1/3)
  3. Schaer, P.: Integration von Open-Access-Repositorien in Fachportale (2010) 1.02 
    1.0214667 = sum of:
  1.0214667 = product of:
    3.0644002 = sum of:
      3.0644002 = weight(author_txt:schaer in 3320) [ClassicSimilarity], result of:
        3.0644002 = score(doc=3320,freq=1.0), product of:
          0.55287546 = queryWeight, product of:
            1.0550374 = boost
            8.868255 = idf(docFreq=16, maxDocs=44421)
            0.059090994 = queryNorm
          5.5426593 = fieldWeight in 3320, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.868255 = idf(docFreq=16, maxDocs=44421)
            0.625 = fieldNorm(doc=3320)
    0.33333334 = coord(1/3)
  4. Schaer, P.: Sprachmodelle und neuronale Netze im Information Retrieval (2023) 1.02 
    1.0214667 = sum of:
  1.0214667 = product of:
    3.0644002 = sum of:
      3.0644002 = weight(author_txt:schaer in 1800) [ClassicSimilarity], result of:
        3.0644002 = score(doc=1800,freq=1.0), product of:
          0.55287546 = queryWeight, product of:
            1.0550374 = boost
            8.868255 = idf(docFreq=16, maxDocs=44421)
            0.059090994 = queryNorm
          5.5426593 = fieldWeight in 1800, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.868255 = idf(docFreq=16, maxDocs=44421)
            0.625 = fieldNorm(doc=1800)
    0.33333334 = coord(1/3)
  5. Keller, F.: How do humans deal with ungrammatical input? : Experimental evidence and computational modelling (1996) 0.87 
    0.86980265 = sum of:
  0.86980265 = product of:
    2.609408 = sum of:
      2.609408 = weight(author_txt:keller in 362) [ClassicSimilarity], result of:
        2.609408 = score(doc=362,freq=1.0), product of:
          0.4966971 = queryWeight, product of:
            8.405631 = idf(docFreq=26, maxDocs=44421)
            0.059090994 = queryNorm
          5.2535195 = fieldWeight in 362, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.405631 = idf(docFreq=26, maxDocs=44421)
            0.625 = fieldNorm(doc=362)
    0.33333334 = coord(1/3)

Similar documents (content)

  1. Harter, S.P.; Hert, C.A.: Evaluation of information retrieval systems : approaches, issues, and methods (1997) 0.25 
    0.25019574 = sum of:
  0.25019574 = product of:
    0.89355624 = sum of:
      0.122188255 = weight(abstract_txt:generalizability in 3264) [ClassicSimilarity], result of:
        0.122188255 = score(doc=3264,freq=1.0), product of:
          0.17390604 = queryWeight, product of:
            1.0206922 = boost
            8.993418 = idf(docFreq=14, maxDocs=44421)
            0.01894502 = queryNorm
          0.70261073 = fieldWeight in 3264, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.993418 = idf(docFreq=14, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
      0.011109202 = weight(abstract_txt:that in 3264) [ClassicSimilarity], result of:
        0.011109202 = score(doc=3264,freq=1.0), product of:
          0.060127564 = queryWeight, product of:
            1.342021 = boost
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.01894502 = queryNorm
          0.18476056 = fieldWeight in 3264, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
      0.059636503 = weight(abstract_txt:systems in 3264) [ClassicSimilarity], result of:
        0.059636503 = score(doc=3264,freq=5.0), product of:
          0.100076735 = queryWeight, product of:
            1.5485815 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.01894502 = queryNorm
          0.59590775 = fieldWeight in 3264, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
      0.094324164 = weight(abstract_txt:effectiveness in 3264) [ClassicSimilarity], result of:
        0.094324164 = score(doc=3264,freq=2.0), product of:
          0.16752286 = queryWeight, product of:
            1.7351426 = boost
            5.0961695 = idf(docFreq=738, maxDocs=44421)
            0.01894502 = queryNorm
          0.5630525 = fieldWeight in 3264, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.0961695 = idf(docFreq=738, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
      0.41963995 = weight(abstract_txt:cranfield in 3264) [ClassicSimilarity], result of:
        0.41963995 = score(doc=3264,freq=5.0), product of:
          0.29167524 = queryWeight, product of:
            1.8694001 = boost
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.01894502 = queryNorm
          1.4387232 = fieldWeight in 3264, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
      0.060400724 = weight(abstract_txt:evaluation in 3264) [ClassicSimilarity], result of:
        0.060400724 = score(doc=3264,freq=1.0), product of:
          0.17258768 = queryWeight, product of:
            2.033632 = boost
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.01894502 = queryNorm
          0.34997123 = fieldWeight in 3264, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
      0.1262574 = weight(abstract_txt:retrieval in 3264) [ClassicSimilarity], result of:
        0.1262574 = score(doc=3264,freq=5.0), product of:
          0.20789286 = queryWeight, product of:
            3.1564746 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.01894502 = queryNorm
          0.6073196 = fieldWeight in 3264, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.078125 = fieldNorm(doc=3264)
    0.28 = coord(7/25)
  2. Borlund, P.; Ruthven, I.: Introduction to the special issue on evaluating interactive information retrieval systems (2008) 0.24 
    0.2364173 = sum of:
  0.2364173 = product of:
    0.59104323 = sum of:
      0.011545021 = weight(abstract_txt:that in 3019) [ClassicSimilarity], result of:
        0.011545021 = score(doc=3019,freq=3.0), product of:
          0.060127564 = queryWeight, product of:
            1.342021 = boost
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.01894502 = queryNorm
          0.1920088 = fieldWeight in 3019, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.024167437 = weight(abstract_txt:what in 3019) [ClassicSimilarity], result of:
        0.024167437 = score(doc=3019,freq=1.0), product of:
          0.11968921 = queryWeight, product of:
            1.4666471 = boost
            4.3075895 = idf(docFreq=1625, maxDocs=44421)
            0.01894502 = queryNorm
          0.20191826 = fieldWeight in 3019, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.3075895 = idf(docFreq=1625, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.12039607 = weight(abstract_txt:evaluations in 3019) [ClassicSimilarity], result of:
        0.12039607 = score(doc=3019,freq=4.0), product of:
          0.19212878 = queryWeight, product of:
            1.5172205 = boost
            6.684188 = idf(docFreq=150, maxDocs=44421)
            0.01894502 = queryNorm
          0.6266426 = fieldWeight in 3019, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            6.684188 = idf(docFreq=150, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.04233772 = weight(abstract_txt:systems in 3019) [ClassicSimilarity], result of:
        0.04233772 = score(doc=3019,freq=7.0), product of:
          0.100076735 = queryWeight, product of:
            1.5485815 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.01894502 = queryNorm
          0.42305255 = fieldWeight in 3019, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.019761255 = weight(abstract_txt:different in 3019) [ClassicSimilarity], result of:
        0.019761255 = score(doc=3019,freq=1.0), product of:
          0.11519241 = queryWeight, product of:
            1.66142 = boost
            3.6597328 = idf(docFreq=3107, maxDocs=44421)
            0.01894502 = queryNorm
          0.17154998 = fieldWeight in 3019, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.6597328 = idf(docFreq=3107, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.040018354 = weight(abstract_txt:effectiveness in 3019) [ClassicSimilarity], result of:
        0.040018354 = score(doc=3019,freq=1.0), product of:
          0.16752286 = queryWeight, product of:
            1.7351426 = boost
            5.0961695 = idf(docFreq=738, maxDocs=44421)
            0.01894502 = queryNorm
          0.23888294 = fieldWeight in 3019, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.0961695 = idf(docFreq=738, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.112601206 = weight(abstract_txt:cranfield in 3019) [ClassicSimilarity], result of:
        0.112601206 = score(doc=3019,freq=1.0), product of:
          0.29167524 = queryWeight, product of:
            1.8694001 = boost
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.01894502 = queryNorm
          0.38604993 = fieldWeight in 3019, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.05281571 = weight(abstract_txt:components in 3019) [ClassicSimilarity], result of:
        0.05281571 = score(doc=3019,freq=1.0), product of:
          0.20156263 = queryWeight, product of:
            1.9032819 = boost
            5.59 = idf(docFreq=450, maxDocs=44421)
            0.01894502 = queryNorm
          0.26203126 = fieldWeight in 3019, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.59 = idf(docFreq=450, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.10872131 = weight(abstract_txt:evaluation in 3019) [ClassicSimilarity], result of:
        0.10872131 = score(doc=3019,freq=9.0), product of:
          0.17258768 = queryWeight, product of:
            2.033632 = boost
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.01894502 = queryNorm
          0.62994826 = fieldWeight in 3019, product of:
            3.0 = tf(freq=9.0), with freq of:
              9.0 = termFreq=9.0
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
      0.058679134 = weight(abstract_txt:retrieval in 3019) [ClassicSimilarity], result of:
        0.058679134 = score(doc=3019,freq=3.0), product of:
          0.20789286 = queryWeight, product of:
            3.1564746 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.01894502 = queryNorm
          0.2822566 = fieldWeight in 3019, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.046875 = fieldNorm(doc=3019)
    0.4 = coord(10/25)
  3. Cleverdon, C.W.; Mills, J.: ¬The testing of index language devices (1985) 0.20 
    0.19801325 = sum of:
  0.19801325 = product of:
    0.6187914 = sum of:
      0.019874416 = weight(abstract_txt:performance in 4643) [ClassicSimilarity], result of:
        0.019874416 = score(doc=4643,freq=1.0), product of:
          0.09177698 = queryWeight, product of:
            1.048623 = boost
            4.619759 = idf(docFreq=1189, maxDocs=44421)
            0.01894502 = queryNorm
          0.21655121 = fieldWeight in 4643, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.619759 = idf(docFreq=1189, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.030432092 = weight(abstract_txt:experiments in 4643) [ClassicSimilarity], result of:
        0.030432092 = score(doc=4643,freq=1.0), product of:
          0.1219248 = queryWeight, product of:
            1.2086444 = boost
            5.324741 = idf(docFreq=587, maxDocs=44421)
            0.01894502 = queryNorm
          0.24959722 = fieldWeight in 4643, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.324741 = idf(docFreq=587, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.019996565 = weight(abstract_txt:that in 4643) [ClassicSimilarity], result of:
        0.019996565 = score(doc=4643,freq=9.0), product of:
          0.060127564 = queryWeight, product of:
            1.342021 = boost
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.01894502 = queryNorm
          0.332569 = fieldWeight in 4643, product of:
            3.0 = tf(freq=9.0), with freq of:
              9.0 = termFreq=9.0
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.04739388 = weight(abstract_txt:paradigm in 4643) [ClassicSimilarity], result of:
        0.04739388 = score(doc=4643,freq=1.0), product of:
          0.16381451 = queryWeight, product of:
            1.4009695 = boost
            6.1720386 = idf(docFreq=251, maxDocs=44421)
            0.01894502 = queryNorm
          0.2893143 = fieldWeight in 4643, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.1720386 = idf(docFreq=251, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.024167437 = weight(abstract_txt:what in 4643) [ClassicSimilarity], result of:
        0.024167437 = score(doc=4643,freq=1.0), product of:
          0.11968921 = queryWeight, product of:
            1.4666471 = boost
            4.3075895 = idf(docFreq=1625, maxDocs=44421)
            0.01894502 = queryNorm
          0.20191826 = fieldWeight in 4643, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.3075895 = idf(docFreq=1625, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.027716542 = weight(abstract_txt:systems in 4643) [ClassicSimilarity], result of:
        0.027716542 = score(doc=4643,freq=3.0), product of:
          0.100076735 = queryWeight, product of:
            1.5485815 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.01894502 = queryNorm
          0.2769529 = fieldWeight in 4643, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.37345597 = weight(abstract_txt:cranfield in 4643) [ClassicSimilarity], result of:
        0.37345597 = score(doc=4643,freq=11.0), product of:
          0.29167524 = queryWeight, product of:
            1.8694001 = boost
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.01894502 = queryNorm
          1.2803829 = fieldWeight in 4643, product of:
            3.3166249 = tf(freq=11.0), with freq of:
              11.0 = termFreq=11.0
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
      0.07575444 = weight(abstract_txt:retrieval in 4643) [ClassicSimilarity], result of:
        0.07575444 = score(doc=4643,freq=5.0), product of:
          0.20789286 = queryWeight, product of:
            3.1564746 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.01894502 = queryNorm
          0.36439174 = fieldWeight in 4643, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.046875 = fieldNorm(doc=4643)
    0.32 = coord(8/25)
  4. Voorhees, E.M.: On test collections for adaptive information retrieval (2008) 0.20 
    0.19787796 = sum of:
  0.19787796 = product of:
    0.706707 = sum of:
      0.087849885 = weight(abstract_txt:experiments in 3444) [ClassicSimilarity], result of:
        0.087849885 = score(doc=3444,freq=3.0), product of:
          0.1219248 = queryWeight, product of:
            1.2086444 = boost
            5.324741 = idf(docFreq=587, maxDocs=44421)
            0.01894502 = queryNorm
          0.72052515 = fieldWeight in 3444, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.324741 = idf(docFreq=587, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
      0.027211878 = weight(abstract_txt:that in 3444) [ClassicSimilarity], result of:
        0.027211878 = score(doc=3444,freq=6.0), product of:
          0.060127564 = queryWeight, product of:
            1.342021 = boost
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.01894502 = queryNorm
          0.45256913 = fieldWeight in 3444, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
      0.078989804 = weight(abstract_txt:paradigm in 3444) [ClassicSimilarity], result of:
        0.078989804 = score(doc=3444,freq=1.0), product of:
          0.16381451 = queryWeight, product of:
            1.4009695 = boost
            6.1720386 = idf(docFreq=251, maxDocs=44421)
            0.01894502 = queryNorm
          0.48219052 = fieldWeight in 3444, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.1720386 = idf(docFreq=251, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
      0.026670255 = weight(abstract_txt:systems in 3444) [ClassicSimilarity], result of:
        0.026670255 = score(doc=3444,freq=1.0), product of:
          0.100076735 = queryWeight, product of:
            1.5485815 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.01894502 = queryNorm
          0.26649806 = fieldWeight in 3444, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
      0.094324164 = weight(abstract_txt:effectiveness in 3444) [ClassicSimilarity], result of:
        0.094324164 = score(doc=3444,freq=2.0), product of:
          0.16752286 = queryWeight, product of:
            1.7351426 = boost
            5.0961695 = idf(docFreq=738, maxDocs=44421)
            0.01894502 = queryNorm
          0.5630525 = fieldWeight in 3444, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.0961695 = idf(docFreq=738, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
      0.2654036 = weight(abstract_txt:cranfield in 3444) [ClassicSimilarity], result of:
        0.2654036 = score(doc=3444,freq=2.0), product of:
          0.29167524 = queryWeight, product of:
            1.8694001 = boost
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.01894502 = queryNorm
          0.90992844 = fieldWeight in 3444, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            8.235732 = idf(docFreq=31, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
      0.1262574 = weight(abstract_txt:retrieval in 3444) [ClassicSimilarity], result of:
        0.1262574 = score(doc=3444,freq=5.0), product of:
          0.20789286 = queryWeight, product of:
            3.1564746 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.01894502 = queryNorm
          0.6073196 = fieldWeight in 3444, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.078125 = fieldNorm(doc=3444)
    0.28 = coord(7/25)
  5. Daoud, M.; Huang, J.X.: Modeling geographic, temporal, and proximity contexts for improving geotemporal search (2013) 0.19 
    0.19194171 = sum of:
  0.19194171 = product of:
    0.7997571 = sum of:
      0.03279111 = weight(abstract_txt:performance in 1533) [ClassicSimilarity], result of:
        0.03279111 = score(doc=1533,freq=2.0), product of:
          0.09177698 = queryWeight, product of:
            1.048623 = boost
            4.619759 = idf(docFreq=1189, maxDocs=44421)
            0.01894502 = queryNorm
          0.35729125 = fieldWeight in 1533, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.619759 = idf(docFreq=1189, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1533)
      0.019048315 = weight(abstract_txt:that in 1533) [ClassicSimilarity], result of:
        0.019048315 = score(doc=1533,freq=6.0), product of:
          0.060127564 = queryWeight, product of:
            1.342021 = boost
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.01894502 = queryNorm
          0.3167984 = fieldWeight in 1533, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            2.3649352 = idf(docFreq=11344, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1533)
      0.018669179 = weight(abstract_txt:systems in 1533) [ClassicSimilarity], result of:
        0.018669179 = score(doc=1533,freq=1.0), product of:
          0.100076735 = queryWeight, product of:
            1.5485815 = boost
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.01894502 = queryNorm
          0.18654864 = fieldWeight in 1533, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.411175 = idf(docFreq=3984, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1533)
      0.042280506 = weight(abstract_txt:evaluation in 1533) [ClassicSimilarity], result of:
        0.042280506 = score(doc=1533,freq=1.0), product of:
          0.17258768 = queryWeight, product of:
            2.033632 = boost
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.01894502 = queryNorm
          0.24497987 = fieldWeight in 1533, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.479632 = idf(docFreq=1368, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1533)
      0.07904964 = weight(abstract_txt:retrieval in 1533) [ClassicSimilarity], result of:
        0.07904964 = score(doc=1533,freq=4.0), product of:
          0.20789286 = queryWeight, product of:
            3.1564746 = boost
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.01894502 = queryNorm
          0.3802422 = fieldWeight in 1533, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.4765 = idf(docFreq=3732, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1533)
      0.6079184 = weight(abstract_txt:temporal in 1533) [ClassicSimilarity], result of:
        0.6079184 = score(doc=1533,freq=10.0), product of:
          0.5102502 = queryWeight, product of:
            3.9094357 = boost
            6.889283 = idf(docFreq=122, maxDocs=44421)
            0.01894502 = queryNorm
          1.1914123 = fieldWeight in 1533, product of:
            3.1622777 = tf(freq=10.0), with freq of:
              10.0 = termFreq=10.0
            6.889283 = idf(docFreq=122, maxDocs=44421)
            0.0546875 = fieldNorm(doc=1533)
    0.24 = coord(6/25)