Document (#44351)

Author

Usbeck, R.
Yan, X.
Perevalov, A.
Jiang, L.
Schulz, J.
Kraft, A.
Möller, C.
Huang, J.
Reineke, J.
Ngonga Ngomo, A.-C.
Saleem, M.
Both, A.

Title

QALD-10 - The 10th challenge on question answering over linked data: : shifting from DBpedia to Wikidata as a KG for KGQA

Source

Semantic Web. 1 (2023), S.1-15 [DOI 10.3233/SW-233471]

Year

2023

Abstract

Knowledge Graph Question Answering (KGQA) has gained attention from both industry and academia over the past decade. Researchers proposed a substantial amount of benchmarking datasets with different properties, pushing the development in this field forward. Many of these benchmarks depend on Freebase, DBpedia, or Wikidata. However, KGQA benchmarks that depend on Freebase and DBpedia are gradually less studied and used, because Freebase is defunct and DBpedia lacks the structural validity of Wikidata. Therefore, research is gravitating toward Wikidata-based benchmarks. That is, new KGQA benchmarks are created on the basis of Wikidata and existing ones are migrated. We present a new, multilingual, complex KGQA benchmarking dataset as the 10th part of the Question Answering over Linked Data (QALD) benchmark series. This corpus formerly depended on DBpedia. Since QALD serves as a base for many machine-generated benchmarks, we increased the size and adjusted the benchmark to Wikidata and its ranking mechanism of properties. These measures foster novel KGQA developments by more demanding benchmarks. Creating a benchmark from scratch or migrating it from DBpedia to Wikidata is non-trivial due to the complexity of the Wikidata knowledge graph, mapping issues between different languages, and the ranking mechanism of properties using qualifiers. We present our creation strategy and the challenges we faced that will assist other researchers in their future work. Our case study, in the form of a conference challenge, is accompanied by an in-depth analysis of the created benchmark.

Content

Vgl.: https://www.researchgate.net/publication/376009186_QALD-10_-_The_10th_challenge_on_question_answering_over_linked_data_Shifting_from_DBpedia_to_Wikidata_as_a_KG_for_KGQA.

Theme

Semantic Web
Retrievalstudien

Object

DBpedia
Wikidata

Similar documents (author)

1. Jiang, Y.; Meng, R.; Huang, Y.; Lu, W.; Liu, J.: Generating keyphrases for readers : a controllable keyphrase generation framework (2023) 0.72

   0.7249984 = sum of:
     0.7249984 = product of:
       1.8124961 = sum of:
         0.7222954 = weight(author_txt:huang in 2014) [ClassicSimilarity], result of:
           0.7222954 = score(doc=2014,freq=1.0), product of:
             0.32192877 = queryWeight, product of:
               7.179679 = idf(docFreq=91, maxDocs=44421)
               0.044838883 = queryNorm
             2.2436497 = fieldWeight in 2014, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.179679 = idf(docFreq=91, maxDocs=44421)
               0.3125 = fieldNorm(doc=2014)
         1.0902007 = weight(author_txt:jiang in 2014) [ClassicSimilarity], result of:
           1.0902007 = score(doc=2014,freq=1.0), product of:
             0.4235983 = queryWeight, product of:
               1.1470892 = boost
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.044838883 = queryNorm
             2.5736663 = fieldWeight in 2014, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.235732 = idf(docFreq=31, maxDocs=44421)
               0.3125 = fieldNorm(doc=2014)
       0.4 = coord(2/5)
   

2. Möller, E.: ¬Die heilige Familie der Inquisition (1998) 0.65

   0.64816934 = sum of:
     0.64816934 = product of:
       3.2408466 = sum of:
         3.2408466 = weight(author_txt:möller in 1636) [ClassicSimilarity], result of:
           3.2408466 = score(doc=1636,freq=1.0), product of:
             0.55169904 = queryWeight, product of:
               1.3090951 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.044838883 = queryNorm
             5.874302 = fieldWeight in 1636, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.625 = fieldNorm(doc=1636)
       0.2 = coord(1/5)
   

3. Möller, E.: Goldgräberstimmung (1999) 0.65

   0.64816934 = sum of:
     0.64816934 = product of:
       3.2408466 = sum of:
         3.2408466 = weight(author_txt:möller in 5136) [ClassicSimilarity], result of:
           3.2408466 = score(doc=5136,freq=1.0), product of:
             0.55169904 = queryWeight, product of:
               1.3090951 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.044838883 = queryNorm
             5.874302 = fieldWeight in 5136, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.625 = fieldNorm(doc=5136)
       0.2 = coord(1/5)
   

4. Möller, G.: Automatic classification of the World Wide Web using Universal Decimal Classification (1999) 0.65

   0.64816934 = sum of:
     0.64816934 = product of:
       3.2408466 = sum of:
         3.2408466 = weight(author_txt:möller in 1494) [ClassicSimilarity], result of:
           3.2408466 = score(doc=1494,freq=1.0), product of:
             0.55169904 = queryWeight, product of:
               1.3090951 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.044838883 = queryNorm
             5.874302 = fieldWeight in 1494, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.625 = fieldNorm(doc=1494)
       0.2 = coord(1/5)
   

5. Möller, E.: ¬Die heimliche Medienrevolution : wie Weblogs, Wikis und freie Software die Welt verändern (2006) 0.65

   0.64816934 = sum of:
     0.64816934 = product of:
       3.2408466 = sum of:
         3.2408466 = weight(author_txt:möller in 267) [ClassicSimilarity], result of:
           3.2408466 = score(doc=267,freq=1.0), product of:
             0.55169904 = queryWeight, product of:
               1.3090951 = boost
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.044838883 = queryNorm
             5.874302 = fieldWeight in 267, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.398883 = idf(docFreq=9, maxDocs=44421)
               0.625 = fieldNorm(doc=267)
       0.2 = coord(1/5)
   

Similar documents (content)

1. Yu, M.; Sun, A.: Dataset versus reality : understanding model performance from the perspective of information need (2023) 0.15

   0.15414453 = sum of:
     0.15414453 = product of:
       0.48170167 = sum of:
         0.01626911 = weight(abstract_txt:researchers in 2075) [ClassicSimilarity], result of:
           0.01626911 = score(doc=2075,freq=2.0), product of:
             0.04390054 = queryWeight, product of:
               1.0876545 = boost
               4.791714 = idf(docFreq=1001, maxDocs=44421)
               0.00842341 = queryNorm
             0.3705902 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.791714 = idf(docFreq=1001, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.015207842 = weight(abstract_txt:created in 2075) [ClassicSimilarity], result of:
           0.015207842 = score(doc=2075,freq=1.0), product of:
             0.05287889 = queryWeight, product of:
               1.1937056 = boost
               5.2589273 = idf(docFreq=627, maxDocs=44421)
               0.00842341 = queryNorm
             0.2875976 = fieldWeight in 2075, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2589273 = idf(docFreq=627, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.0043938938 = weight(abstract_txt:from in 2075) [ClassicSimilarity], result of:
           0.0043938938 = score(doc=2075,freq=1.0), product of:
             0.029117025 = queryWeight, product of:
               1.2526927 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.00842341 = queryNorm
             0.15090463 = fieldWeight in 2075, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.03823482 = weight(abstract_txt:question in 2075) [ClassicSimilarity], result of:
           0.03823482 = score(doc=2075,freq=3.0), product of:
             0.07760088 = queryWeight, product of:
               1.771067 = boost
               5.2016807 = idf(docFreq=664, maxDocs=44421)
               0.00842341 = queryNorm
             0.49271113 = fieldWeight in 2075, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.2016807 = idf(docFreq=664, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.031860277 = weight(abstract_txt:properties in 2075) [ClassicSimilarity], result of:
           0.031860277 = score(doc=2075,freq=1.0), product of:
             0.09910618 = queryWeight, product of:
               2.0014837 = boost
               5.878422 = idf(docFreq=337, maxDocs=44421)
               0.00842341 = queryNorm
             0.3214762 = fieldWeight in 2075, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.878422 = idf(docFreq=337, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.06490308 = weight(abstract_txt:answering in 2075) [ClassicSimilarity], result of:
           0.06490308 = score(doc=2075,freq=2.0), product of:
             0.12640607 = queryWeight, product of:
               2.260402 = boost
               6.6388726 = idf(docFreq=157, maxDocs=44421)
               0.00842341 = queryNorm
             0.5134491 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6388726 = idf(docFreq=157, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.11366871 = weight(abstract_txt:benchmark in 2075) [ClassicSimilarity], result of:
           0.11366871 = score(doc=2075,freq=2.0), product of:
             0.20214574 = queryWeight, product of:
               3.3006794 = boost
               7.270651 = idf(docFreq=83, maxDocs=44421)
               0.00842341 = queryNorm
             0.5623107 = fieldWeight in 2075, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.270651 = idf(docFreq=83, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
         0.19716395 = weight(abstract_txt:benchmarks in 2075) [ClassicSimilarity], result of:
           0.19716395 = score(doc=2075,freq=1.0), product of:
             0.42088428 = queryWeight, product of:
               5.8330812 = boost
               8.565973 = idf(docFreq=22, maxDocs=44421)
               0.00842341 = queryNorm
             0.46845168 = fieldWeight in 2075, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.565973 = idf(docFreq=22, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2075)
       0.32 = coord(8/25)
   

2. Otterbacher, J.; Erkan, G.; Radev, D.R.: Biased LexRank : passage retrieval using random walks with question-based priors (2009) 0.12

   0.1233653 = sum of:
     0.1233653 = product of:
       0.6168265 = sum of:
         0.030657815 = weight(abstract_txt:linked in 3450) [ClassicSimilarity], result of:
           0.030657815 = score(doc=3450,freq=1.0), product of:
             0.05891275 = queryWeight, product of:
               1.2599714 = boost
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.00842341 = queryNorm
             0.52039355 = fieldWeight in 3450, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.09375 = fieldNorm(doc=3450)
         0.07136549 = weight(abstract_txt:graph in 3450) [ClassicSimilarity], result of:
           0.07136549 = score(doc=3450,freq=2.0), product of:
             0.08212915 = queryWeight, product of:
               1.4876635 = boost
               6.553973 = idf(docFreq=171, maxDocs=44421)
               0.00842341 = queryNorm
             0.86894226 = fieldWeight in 3450, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.553973 = idf(docFreq=171, maxDocs=44421)
               0.09375 = fieldNorm(doc=3450)
         0.0655454 = weight(abstract_txt:question in 3450) [ClassicSimilarity], result of:
           0.0655454 = score(doc=3450,freq=3.0), product of:
             0.07760088 = queryWeight, product of:
               1.771067 = boost
               5.2016807 = idf(docFreq=664, maxDocs=44421)
               0.00842341 = queryNorm
             0.84464765 = fieldWeight in 3450, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.2016807 = idf(docFreq=664, maxDocs=44421)
               0.09375 = fieldNorm(doc=3450)
         0.11126243 = weight(abstract_txt:answering in 3450) [ClassicSimilarity], result of:
           0.11126243 = score(doc=3450,freq=2.0), product of:
             0.12640607 = queryWeight, product of:
               2.260402 = boost
               6.6388726 = idf(docFreq=157, maxDocs=44421)
               0.00842341 = queryNorm
             0.8801985 = fieldWeight in 3450, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6388726 = idf(docFreq=157, maxDocs=44421)
               0.09375 = fieldNorm(doc=3450)
         0.33799532 = weight(abstract_txt:benchmarks in 3450) [ClassicSimilarity], result of:
           0.33799532 = score(doc=3450,freq=1.0), product of:
             0.42088428 = queryWeight, product of:
               5.8330812 = boost
               8.565973 = idf(docFreq=22, maxDocs=44421)
               0.00842341 = queryNorm
             0.80306 = fieldWeight in 3450, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.565973 = idf(docFreq=22, maxDocs=44421)
               0.09375 = fieldNorm(doc=3450)
       0.2 = coord(5/25)
   

3. Bianchini, C.; Bargioni, S.: Automated classification using linked open data : a case study on faceted classification and Wikidata (2021) 0.12

   0.11549835 = sum of:
     0.11549835 = product of:
       0.96248627 = sum of:
         0.0075323894 = weight(abstract_txt:from in 1725) [ClassicSimilarity], result of:
           0.0075323894 = score(doc=1725,freq=1.0), product of:
             0.029117025 = queryWeight, product of:
               1.2526927 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.00842341 = queryNorm
             0.25869364 = fieldWeight in 1725, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.09375 = fieldNorm(doc=1725)
         0.030657815 = weight(abstract_txt:linked in 1725) [ClassicSimilarity], result of:
           0.030657815 = score(doc=1725,freq=1.0), product of:
             0.05891275 = queryWeight, product of:
               1.2599714 = boost
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.00842341 = queryNorm
             0.52039355 = fieldWeight in 1725, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.09375 = fieldNorm(doc=1725)
         0.9242961 = weight(abstract_txt:wikidata in 1725) [ClassicSimilarity], result of:
           0.9242961 = score(doc=1725,freq=3.0), product of:
             0.6281097 = queryWeight, product of:
               8.22818 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.00842341 = queryNorm
             1.471552 = fieldWeight in 1725, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.09375 = fieldNorm(doc=1725)
       0.12 = coord(3/25)
   

4. Bianchini, D.; Antonellis, V. De: Linked data services and semantics-enabled mashup (2012) 0.10

   0.103052296 = sum of:
     0.103052296 = product of:
       0.5152615 = sum of:
         0.009825045 = weight(abstract_txt:from in 1435) [ClassicSimilarity], result of:
           0.009825045 = score(doc=1435,freq=5.0), product of:
             0.029117025 = queryWeight, product of:
               1.2526927 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.00842341 = queryNorm
             0.337433 = fieldWeight in 1435, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1435)
         0.050582815 = weight(abstract_txt:linked in 1435) [ClassicSimilarity], result of:
           0.050582815 = score(doc=1435,freq=8.0), product of:
             0.05891275 = queryWeight, product of:
               1.2599714 = boost
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.00842341 = queryNorm
             0.85860556 = fieldWeight in 1435, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1435)
         0.026300184 = weight(abstract_txt:mechanism in 1435) [ClassicSimilarity], result of:
           0.026300184 = score(doc=1435,freq=1.0), product of:
             0.07618623 = queryWeight, product of:
               1.4328288 = boost
               6.312396 = idf(docFreq=218, maxDocs=44421)
               0.00842341 = queryNorm
             0.34520915 = fieldWeight in 1435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.312396 = idf(docFreq=218, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1435)
         0.011978318 = weight(abstract_txt:over in 1435) [ClassicSimilarity], result of:
           0.011978318 = score(doc=1435,freq=1.0), product of:
             0.05162558 = queryWeight, product of:
               1.4445553 = boost
               4.242705 = idf(docFreq=1734, maxDocs=44421)
               0.00842341 = queryNorm
             0.23202293 = fieldWeight in 1435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.242705 = idf(docFreq=1734, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1435)
         0.4165751 = weight(abstract_txt:dbpedia in 1435) [ClassicSimilarity], result of:
           0.4165751 = score(doc=1435,freq=5.0), product of:
             0.40527508 = queryWeight, product of:
               5.7238946 = boost
               8.405631 = idf(docFreq=26, maxDocs=44421)
               0.00842341 = queryNorm
             1.0278823 = fieldWeight in 1435, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               8.405631 = idf(docFreq=26, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1435)
       0.2 = coord(5/25)
   

5. Pattuelli, C.; Rubinow, S.: ¬The knowledge organization of DBpedia : a case study (2013) 0.10

   0.10249193 = sum of:
     0.10249193 = product of:
       0.51245964 = sum of:
         0.005021593 = weight(abstract_txt:from in 2776) [ClassicSimilarity], result of:
           0.005021593 = score(doc=2776,freq=1.0), product of:
             0.029117025 = queryWeight, product of:
               1.2526927 = boost
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.00842341 = queryNorm
             0.17246243 = fieldWeight in 2776, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.759399 = idf(docFreq=7646, maxDocs=44421)
               0.0625 = fieldNorm(doc=2776)
         0.020438544 = weight(abstract_txt:linked in 2776) [ClassicSimilarity], result of:
           0.020438544 = score(doc=2776,freq=1.0), product of:
             0.05891275 = queryWeight, product of:
               1.2599714 = boost
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.00842341 = queryNorm
             0.34692904 = fieldWeight in 2776, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5508647 = idf(docFreq=468, maxDocs=44421)
               0.0625 = fieldNorm(doc=2776)
         0.024763625 = weight(abstract_txt:challenge in 2776) [ClassicSimilarity], result of:
           0.024763625 = score(doc=2776,freq=1.0), product of:
             0.06695538 = queryWeight, product of:
               1.343225 = boost
               5.9176426 = idf(docFreq=324, maxDocs=44421)
               0.00842341 = queryNorm
             0.36985266 = fieldWeight in 2776, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9176426 = idf(docFreq=324, maxDocs=44421)
               0.0625 = fieldNorm(doc=2776)
         0.036411744 = weight(abstract_txt:properties in 2776) [ClassicSimilarity], result of:
           0.036411744 = score(doc=2776,freq=1.0), product of:
             0.09910618 = queryWeight, product of:
               2.0014837 = boost
               5.878422 = idf(docFreq=337, maxDocs=44421)
               0.00842341 = queryNorm
             0.36740136 = fieldWeight in 2776, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.878422 = idf(docFreq=337, maxDocs=44421)
               0.0625 = fieldNorm(doc=2776)
         0.4258241 = weight(abstract_txt:dbpedia in 2776) [ClassicSimilarity], result of:
           0.4258241 = score(doc=2776,freq=4.0), product of:
             0.40527508 = queryWeight, product of:
               5.7238946 = boost
               8.405631 = idf(docFreq=26, maxDocs=44421)
               0.00842341 = queryNorm
             1.0507039 = fieldWeight in 2776, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.405631 = idf(docFreq=26, maxDocs=44421)
               0.0625 = fieldNorm(doc=2776)
       0.2 = coord(5/25)