Document (#13227)

Author

Huffman, G.D.

Title

Semi-automatic determination of citation relevancy : user evaluation

Source

Information processing and management. 26(1990) no.2, S.295-302

Year

1990

Abstract

Online bibiographic, database searches typically produce hundreds of retrieved citations with only about 20-40% relevant to the search topic and/or problem statement. Significant amounts of time are required to categorize and select the relevant citations. A software system-SORT-AIDS/SABRE-has been developes which ranks the citations in terms of relevance. This paper presents the results of a comprehensive user evaluation of the relevance ranking procedures. Test results show that the software generated distributions approach the ideal distribution-all relevant citations at the beginning of the collection-in 22% of the cases, are 23% better than the random distribution-relevant citations distributed uniformly throughout the dcollection-on average and are poorer than the random distribution in 4% of the cae.

Object

SORT-AID/SABRE

Similar documents (content)

1. Ruthven, I.; Lalmas, M.; Rijsbergen, K. van: Combining and selecting characteristics of information use (2002) 0.16

   0.15826051 = sum of:
     0.15826051 = product of:
       0.5652161 = sum of:
         0.063596494 = weight(abstract_txt:ranks in 208) [ClassicSimilarity], result of:
           0.063596494 = score(doc=208,freq=1.0), product of:
             0.1795006 = queryWeight, product of:
               1.1593097 = boost
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.020485207 = queryNorm
             0.35429686 = fieldWeight in 208, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
         0.024573427 = weight(abstract_txt:than in 208) [ClassicSimilarity], result of:
           0.024573427 = score(doc=208,freq=2.0), product of:
             0.095225714 = queryWeight, product of:
               1.1941494 = boost
               3.8927383 = idf(docFreq=2461, maxDocs=44421)
               0.020485207 = queryNorm
             0.25805452 = fieldWeight in 208, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.8927383 = idf(docFreq=2461, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
         0.118605375 = weight(abstract_txt:poorer in 208) [ClassicSimilarity], result of:
           0.118605375 = score(doc=208,freq=1.0), product of:
             0.2719652 = queryWeight, product of:
               1.4269975 = boost
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.020485207 = queryNorm
             0.43610495 = fieldWeight in 208, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.303573 = idf(docFreq=10, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
         0.049901914 = weight(abstract_txt:relevance in 208) [ClassicSimilarity], result of:
           0.049901914 = score(doc=208,freq=2.0), product of:
             0.1527057 = queryWeight, product of:
               1.5121998 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.020485207 = queryNorm
             0.32678488 = fieldWeight in 208, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
         0.08205799 = weight(abstract_txt:random in 208) [ClassicSimilarity], result of:
           0.08205799 = score(doc=208,freq=1.0), product of:
             0.26804087 = queryWeight, product of:
               2.0034664 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.020485207 = queryNorm
             0.30613986 = fieldWeight in 208, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
         0.10954222 = weight(abstract_txt:distribution in 208) [ClassicSimilarity], result of:
           0.10954222 = score(doc=208,freq=2.0), product of:
             0.2952536 = queryWeight, product of:
               2.5752819 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.020485207 = queryNorm
             0.3710106 = fieldWeight in 208, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
         0.116938725 = weight(abstract_txt:relevant in 208) [ClassicSimilarity], result of:
           0.116938725 = score(doc=208,freq=4.0), product of:
             0.2694114 = queryWeight, product of:
               2.8405638 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.020485207 = queryNorm
             0.4340526 = fieldWeight in 208, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.046875 = fieldNorm(doc=208)
       0.28 = coord(7/25)
   

2. Bensman, S.J.; Smolinsky, L.J.; Pudovkin, A.I.: Mean citation rate per article in mathematics journals : differences from the scientific model (2010) 0.13

   0.13301502 = sum of:
     0.13301502 = product of:
       0.55422926 = sum of:
         0.07729784 = weight(abstract_txt:distributions in 582) [ClassicSimilarity], result of:
           0.07729784 = score(doc=582,freq=2.0), product of:
             0.14641339 = queryWeight, product of:
               1.0470247 = boost
               6.82627 = idf(docFreq=130, maxDocs=44421)
               0.020485207 = queryNorm
             0.5279424 = fieldWeight in 582, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.82627 = idf(docFreq=130, maxDocs=44421)
               0.0546875 = fieldNorm(doc=582)
         0.074195914 = weight(abstract_txt:ranks in 582) [ClassicSimilarity], result of:
           0.074195914 = score(doc=582,freq=1.0), product of:
             0.1795006 = queryWeight, product of:
               1.1593097 = boost
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.020485207 = queryNorm
             0.41334632 = fieldWeight in 582, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.0546875 = fieldNorm(doc=582)
         0.020272043 = weight(abstract_txt:than in 582) [ClassicSimilarity], result of:
           0.020272043 = score(doc=582,freq=1.0), product of:
             0.095225714 = queryWeight, product of:
               1.1941494 = boost
               3.8927383 = idf(docFreq=2461, maxDocs=44421)
               0.020485207 = queryNorm
             0.21288413 = fieldWeight in 582, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.8927383 = idf(docFreq=2461, maxDocs=44421)
               0.0546875 = fieldNorm(doc=582)
         0.09573432 = weight(abstract_txt:random in 582) [ClassicSimilarity], result of:
           0.09573432 = score(doc=582,freq=1.0), product of:
             0.26804087 = queryWeight, product of:
               2.0034664 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.020485207 = queryNorm
             0.35716316 = fieldWeight in 582, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.0546875 = fieldNorm(doc=582)
         0.15652148 = weight(abstract_txt:distribution in 582) [ClassicSimilarity], result of:
           0.15652148 = score(doc=582,freq=3.0), product of:
             0.2952536 = queryWeight, product of:
               2.5752819 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.020485207 = queryNorm
             0.53012556 = fieldWeight in 582, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0546875 = fieldNorm(doc=582)
         0.13020763 = weight(abstract_txt:citations in 582) [ClassicSimilarity], result of:
           0.13020763 = score(doc=582,freq=1.0), product of:
             0.44657406 = queryWeight, product of:
               4.088823 = boost
               5.331567 = idf(docFreq=583, maxDocs=44421)
               0.020485207 = queryNorm
             0.29157007 = fieldWeight in 582, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.331567 = idf(docFreq=583, maxDocs=44421)
               0.0546875 = fieldNorm(doc=582)
       0.24 = coord(6/25)
   

3. Joss, M.W.; Wszola, S.: ¬The engines that can : text search and retrieval software, their strategies, and vendors (1996) 0.12

   0.12369058 = sum of:
     0.12369058 = product of:
       0.44175208 = sum of:
         0.06490488 = weight(abstract_txt:amounts in 5191) [ClassicSimilarity], result of:
           0.06490488 = score(doc=5191,freq=1.0), product of:
             0.15019986 = queryWeight, product of:
               1.0604771 = boost
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.020485207 = queryNorm
             0.43212348 = fieldWeight in 5191, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9139757 = idf(docFreq=119, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
         0.016533103 = weight(abstract_txt:results in 5191) [ClassicSimilarity], result of:
           0.016533103 = score(doc=5191,freq=1.0), product of:
             0.07604386 = queryWeight, product of:
               1.0671209 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.020485207 = queryNorm
             0.21741535 = fieldWeight in 5191, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
         0.07906154 = weight(abstract_txt:sort in 5191) [ClassicSimilarity], result of:
           0.07906154 = score(doc=5191,freq=1.0), product of:
             0.17131476 = queryWeight, product of:
               1.132567 = boost
               7.3839793 = idf(docFreq=74, maxDocs=44421)
               0.020485207 = queryNorm
             0.4614987 = fieldWeight in 5191, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3839793 = idf(docFreq=74, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
         0.02316805 = weight(abstract_txt:than in 5191) [ClassicSimilarity], result of:
           0.02316805 = score(doc=5191,freq=1.0), product of:
             0.095225714 = queryWeight, product of:
               1.1941494 = boost
               3.8927383 = idf(docFreq=2461, maxDocs=44421)
               0.020485207 = queryNorm
             0.24329615 = fieldWeight in 5191, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.8927383 = idf(docFreq=2461, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
         0.0563065 = weight(abstract_txt:software in 5191) [ClassicSimilarity], result of:
           0.0563065 = score(doc=5191,freq=3.0), product of:
             0.11935113 = queryWeight, product of:
               1.3368874 = boost
               4.3580413 = idf(docFreq=1545, maxDocs=44421)
               0.020485207 = queryNorm
             0.4717718 = fieldWeight in 5191, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.3580413 = idf(docFreq=1545, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
         0.047047976 = weight(abstract_txt:relevance in 5191) [ClassicSimilarity], result of:
           0.047047976 = score(doc=5191,freq=1.0), product of:
             0.1527057 = queryWeight, product of:
               1.5121998 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.020485207 = queryNorm
             0.30809575 = fieldWeight in 5191, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
         0.15473002 = weight(abstract_txt:random in 5191) [ClassicSimilarity], result of:
           0.15473002 = score(doc=5191,freq=2.0), product of:
             0.26804087 = queryWeight, product of:
               2.0034664 = boost
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.020485207 = queryNorm
             0.5772628 = fieldWeight in 5191, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.5309834 = idf(docFreq=175, maxDocs=44421)
               0.0625 = fieldNorm(doc=5191)
       0.28 = coord(7/25)
   

4. Leydesdorff, L.; Bornmann, L.; Mutz, R.; Opthof, T.: Turning the tables on citation analysis one more time : principles for comparing sets of documents (2011) 0.12

   0.11684821 = sum of:
     0.11684821 = product of:
       0.58424103 = sum of:
         0.08834038 = weight(abstract_txt:distributions in 485) [ClassicSimilarity], result of:
           0.08834038 = score(doc=485,freq=2.0), product of:
             0.14641339 = queryWeight, product of:
               1.0470247 = boost
               6.82627 = idf(docFreq=130, maxDocs=44421)
               0.020485207 = queryNorm
             0.60336274 = fieldWeight in 485, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.82627 = idf(docFreq=130, maxDocs=44421)
               0.0625 = fieldNorm(doc=485)
         0.14686981 = weight(abstract_txt:ranks in 485) [ClassicSimilarity], result of:
           0.14686981 = score(doc=485,freq=3.0), product of:
             0.1795006 = queryWeight, product of:
               1.1593097 = boost
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.020485207 = queryNorm
             0.8182135 = fieldWeight in 485, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.0625 = fieldNorm(doc=485)
         0.03530619 = weight(abstract_txt:evaluation in 485) [ClassicSimilarity], result of:
           0.03530619 = score(doc=485,freq=1.0), product of:
             0.1261039 = queryWeight, product of:
               1.3741869 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.020485207 = queryNorm
             0.279977 = fieldWeight in 485, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=485)
         0.10327739 = weight(abstract_txt:distribution in 485) [ClassicSimilarity], result of:
           0.10327739 = score(doc=485,freq=1.0), product of:
             0.2952536 = queryWeight, product of:
               2.5752819 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.020485207 = queryNorm
             0.34979215 = fieldWeight in 485, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0625 = fieldNorm(doc=485)
         0.2104473 = weight(abstract_txt:citations in 485) [ClassicSimilarity], result of:
           0.2104473 = score(doc=485,freq=2.0), product of:
             0.44657406 = queryWeight, product of:
               4.088823 = boost
               5.331567 = idf(docFreq=583, maxDocs=44421)
               0.020485207 = queryNorm
             0.47124836 = fieldWeight in 485, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.331567 = idf(docFreq=583, maxDocs=44421)
               0.0625 = fieldNorm(doc=485)
       0.2 = coord(5/25)
   

5. Cai, F.; Wang, S.; Rijke, M.de: Behavior-based personalization in web search (2017) 0.12

   0.11506813 = sum of:
     0.11506813 = product of:
       0.4109576 = sum of:
         0.06246608 = weight(abstract_txt:distributions in 4527) [ClassicSimilarity], result of:
           0.06246608 = score(doc=4527,freq=1.0), product of:
             0.14641339 = queryWeight, product of:
               1.0470247 = boost
               6.82627 = idf(docFreq=130, maxDocs=44421)
               0.020485207 = queryNorm
             0.42664188 = fieldWeight in 4527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.82627 = idf(docFreq=130, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
         0.016533103 = weight(abstract_txt:results in 4527) [ClassicSimilarity], result of:
           0.016533103 = score(doc=4527,freq=1.0), product of:
             0.07604386 = queryWeight, product of:
               1.0671209 = boost
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.020485207 = queryNorm
             0.21741535 = fieldWeight in 4527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4786456 = idf(docFreq=3724, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
         0.03392621 = weight(abstract_txt:user in 4527) [ClassicSimilarity], result of:
           0.03392621 = score(doc=4527,freq=3.0), product of:
             0.08514211 = queryWeight, product of:
               1.1291555 = boost
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.020485207 = queryNorm
             0.3984657 = fieldWeight in 4527, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6808684 = idf(docFreq=3042, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
         0.03530619 = weight(abstract_txt:evaluation in 4527) [ClassicSimilarity], result of:
           0.03530619 = score(doc=4527,freq=1.0), product of:
             0.1261039 = queryWeight, product of:
               1.3741869 = boost
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.020485207 = queryNorm
             0.279977 = fieldWeight in 4527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.479632 = idf(docFreq=1368, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
         0.08148948 = weight(abstract_txt:relevance in 4527) [ClassicSimilarity], result of:
           0.08148948 = score(doc=4527,freq=3.0), product of:
             0.1527057 = queryWeight, product of:
               1.5121998 = boost
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.020485207 = queryNorm
             0.53363746 = fieldWeight in 4527, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.929532 = idf(docFreq=872, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
         0.10327739 = weight(abstract_txt:distribution in 4527) [ClassicSimilarity], result of:
           0.10327739 = score(doc=4527,freq=1.0), product of:
             0.2952536 = queryWeight, product of:
               2.5752819 = boost
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.020485207 = queryNorm
             0.34979215 = fieldWeight in 4527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5966744 = idf(docFreq=447, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
         0.07795915 = weight(abstract_txt:relevant in 4527) [ClassicSimilarity], result of:
           0.07795915 = score(doc=4527,freq=1.0), product of:
             0.2694114 = queryWeight, product of:
               2.8405638 = boost
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.020485207 = queryNorm
             0.2893684 = fieldWeight in 4527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6298943 = idf(docFreq=1177, maxDocs=44421)
               0.0625 = fieldNorm(doc=4527)
       0.28 = coord(7/25)