Document (#43358)

Author

Zhang, Y.
Ren, P.
Rijke, M. de

Title

¬A taxonomy, data set, and benchmark for detecting and classifying malevolent dialogue responses

Source

Journal of the Association for Information Science and Technology. 72(2021) no.12, S.1477-1497

Year

2021

Abstract

Conversational interfaces are increasingly popular as a way of connecting people to information. With the increased generative capacity of corpus-based conversational agents comes the need to classify and filter out malevolent responses that are inappropriate in terms of content and dialogue acts. Previous studies on the topic of detecting and classifying inappropriate content are mostly focused on a specific category of malevolence or on single sentences instead of an entire dialogue. We make three contributions to advance research on the malevolent dialogue response detection and classification (MDRDC) task. First, we define the task and present a hierarchical malevolent dialogue taxonomy. Second, we create a labeled multiturn dialogue data set and formulate the MDRDC task as a hierarchical classification task. Last, we apply state-of-the-art text classification methods to the MDRDC task, and report on experiments aimed at assessing the performance of these approaches.

Content

Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24496.

Similar documents (author)

1. De Rijke, M. -> Rijke, M. de: 2.01

   2.013841 = sum of:
     2.013841 = product of:
       4.027682 = sum of:
         4.027682 = weight(author_txt:rijke in 5116) [ClassicSimilarity], result of:
           4.027682 = score(doc=5116,freq=2.0), product of:
             0.87729204 = queryWeight, product of:
               1.3519822 = boost
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.07495637 = queryNorm
             4.5910387 = fieldWeight in 5116, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.375 = fieldNorm(doc=5116)
       0.5 = coord(1/2)
   

2. Li, X.; Rijke, M.de: Characterizing and predicting downloads in academic search (2019) 1.90

   1.8986673 = sum of:
     1.8986673 = product of:
       3.7973347 = sum of:
         3.7973347 = weight(author_txt:rijke in 103) [ClassicSimilarity], result of:
           3.7973347 = score(doc=103,freq=1.0), product of:
             0.87729204 = queryWeight, product of:
               1.3519822 = boost
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.07495637 = queryNorm
             4.3284726 = fieldWeight in 103, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.5 = fieldNorm(doc=103)
       0.5 = coord(1/2)
   

3. DeRijke, M. -> Rijke, M. de: 1.66

   1.6613339 = sum of:
     1.6613339 = product of:
       3.3226678 = sum of:
         3.3226678 = weight(author_txt:rijke in 117) [ClassicSimilarity], result of:
           3.3226678 = score(doc=117,freq=1.0), product of:
             0.87729204 = queryWeight, product of:
               1.3519822 = boost
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.07495637 = queryNorm
             3.7874136 = fieldWeight in 117, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.4375 = fieldNorm(doc=117)
       0.5 = coord(1/2)
   

4. Meij, E.; Rijke, M. de: Thesaurus-based feedback to support mixed search and browsing environments (2007) 1.66

   1.6613339 = sum of:
     1.6613339 = product of:
       3.3226678 = sum of:
         3.3226678 = weight(author_txt:rijke in 3432) [ClassicSimilarity], result of:
           3.3226678 = score(doc=3432,freq=1.0), product of:
             0.87729204 = queryWeight, product of:
               1.3519822 = boost
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.07495637 = queryNorm
             3.7874136 = fieldWeight in 3432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.4375 = fieldNorm(doc=3432)
       0.5 = coord(1/2)
   

5. Cai, F.; Rijke, M. de: Learning from homologous queries and semantically related terms for query auto completion (2016) 1.66

   1.6613339 = sum of:
     1.6613339 = product of:
       3.3226678 = sum of:
         3.3226678 = weight(author_txt:rijke in 3971) [ClassicSimilarity], result of:
           3.3226678 = score(doc=3971,freq=1.0), product of:
             0.87729204 = queryWeight, product of:
               1.3519822 = boost
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.07495637 = queryNorm
             3.7874136 = fieldWeight in 3971, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.656945 = idf(docFreq=20, maxDocs=44421)
               0.4375 = fieldNorm(doc=3971)
       0.5 = coord(1/2)
   

Similar documents (content)

1. ChatGPT : Optimizing language models for dalogue (2022) 0.14

   0.1417251 = sum of:
     0.1417251 = product of:
       1.1810424 = sum of:
         0.25564075 = weight(abstract_txt:inappropriate in 1837) [ClassicSimilarity], result of:
           0.25564075 = score(doc=1837,freq=1.0), product of:
             0.25680283 = queryWeight, product of:
               2.3852322 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013519134 = queryNorm
             0.99547476 = fieldWeight in 1837, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.125 = fieldNorm(doc=1837)
         0.28943163 = weight(abstract_txt:conversational in 1837) [ClassicSimilarity], result of:
           0.28943163 = score(doc=1837,freq=1.0), product of:
             0.27896118 = queryWeight, product of:
               2.486009 = boost
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.013519134 = queryNorm
             1.0375338 = fieldWeight in 1837, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.30027 = idf(docFreq=29, maxDocs=44421)
               0.125 = fieldNorm(doc=1837)
         0.6359701 = weight(abstract_txt:dialogue in 1837) [ClassicSimilarity], result of:
           0.6359701 = score(doc=1837,freq=1.0), product of:
             0.68000376 = queryWeight, product of:
               6.7227516 = boost
               7.48196 = idf(docFreq=67, maxDocs=44421)
               0.013519134 = queryNorm
             0.935245 = fieldWeight in 1837, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.48196 = idf(docFreq=67, maxDocs=44421)
               0.125 = fieldNorm(doc=1837)
       0.12 = coord(3/25)
   

2. Sood, S.O.; Churchill, E.F.; Antin, J.: Automatic identification of personal insults on social news sites (2012) 0.08

   0.08204682 = sum of:
     0.08204682 = product of:
       0.4102341 = sum of:
         0.0681313 = weight(abstract_txt:detection in 976) [ClassicSimilarity], result of:
           0.0681313 = score(doc=976,freq=3.0), product of:
             0.09290672 = queryWeight, product of:
               1.0144703 = boost
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.013519134 = queryNorm
             0.73333013 = fieldWeight in 976, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.0625 = fieldNorm(doc=976)
         0.054637067 = weight(abstract_txt:labeled in 976) [ClassicSimilarity], result of:
           0.054637067 = score(doc=976,freq=1.0), product of:
             0.11565951 = queryWeight, product of:
               1.1318958 = boost
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.013519134 = queryNorm
             0.4723958 = fieldWeight in 976, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.558333 = idf(docFreq=62, maxDocs=44421)
               0.0625 = fieldNorm(doc=976)
         0.032004185 = weight(abstract_txt:content in 976) [ClassicSimilarity], result of:
           0.032004185 = score(doc=976,freq=3.0), product of:
             0.070734374 = queryWeight, product of:
               1.2518319 = boost
               4.1796083 = idf(docFreq=1847, maxDocs=44421)
               0.013519134 = queryNorm
             0.45245588 = fieldWeight in 976, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1796083 = idf(docFreq=1847, maxDocs=44421)
               0.0625 = fieldNorm(doc=976)
         0.1807653 = weight(abstract_txt:inappropriate in 976) [ClassicSimilarity], result of:
           0.1807653 = score(doc=976,freq=2.0), product of:
             0.25680283 = queryWeight, product of:
               2.3852322 = boost
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.013519134 = queryNorm
             0.70390695 = fieldWeight in 976, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.963798 = idf(docFreq=41, maxDocs=44421)
               0.0625 = fieldNorm(doc=976)
         0.074696235 = weight(abstract_txt:task in 976) [ClassicSimilarity], result of:
           0.074696235 = score(doc=976,freq=1.0), product of:
             0.2436196 = queryWeight, product of:
               3.6733043 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.013519134 = queryNorm
             0.3066101 = fieldWeight in 976, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=976)
       0.2 = coord(5/25)
   

3. Ding, W.; Chen, C.: Dynamic topic detection and tracking : a comparison of HDP, C-word, and cocitation methods (2014) 0.08

   0.07815025 = sum of:
     0.07815025 = product of:
       0.48843908 = sum of:
         0.049169533 = weight(abstract_txt:detection in 2502) [ClassicSimilarity], result of:
           0.049169533 = score(doc=2502,freq=1.0), product of:
             0.09290672 = queryWeight, product of:
               1.0144703 = boost
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.013519134 = queryNorm
             0.5292355 = fieldWeight in 2502, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.774214 = idf(docFreq=137, maxDocs=44421)
               0.078125 = fieldNorm(doc=2502)
         0.15132456 = weight(abstract_txt:generative in 2502) [ClassicSimilarity], result of:
           0.15132456 = score(doc=2502,freq=3.0), product of:
             0.13629574 = queryWeight, product of:
               1.2287313 = boost
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.013519134 = queryNorm
             1.1102662 = fieldWeight in 2502, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.20496 = idf(docFreq=32, maxDocs=44421)
               0.078125 = fieldNorm(doc=2502)
         0.0841734 = weight(abstract_txt:hierarchical in 2502) [ClassicSimilarity], result of:
           0.0841734 = score(doc=2502,freq=2.0), product of:
             0.13295348 = queryWeight, product of:
               1.7162503 = boost
               5.7302055 = idf(docFreq=391, maxDocs=44421)
               0.013519134 = queryNorm
             0.6331042 = fieldWeight in 2502, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7302055 = idf(docFreq=391, maxDocs=44421)
               0.078125 = fieldNorm(doc=2502)
         0.20377159 = weight(abstract_txt:detecting in 2502) [ClassicSimilarity], result of:
           0.20377159 = score(doc=2502,freq=2.0), product of:
             0.23970598 = queryWeight, product of:
               2.3044655 = boost
               7.694134 = idf(docFreq=54, maxDocs=44421)
               0.013519134 = queryNorm
             0.8500897 = fieldWeight in 2502, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.694134 = idf(docFreq=54, maxDocs=44421)
               0.078125 = fieldNorm(doc=2502)
       0.16 = coord(4/25)
   

4. Li, Y.; Belkin, N.J.: ¬A faceted approach to conceptualizing tasks in information seeking (2008) 0.08

   0.07609144 = sum of:
     0.07609144 = product of:
       0.4755715 = sum of:
         0.045217317 = weight(abstract_txt:advance in 3442) [ClassicSimilarity], result of:
           0.045217317 = score(doc=3442,freq=1.0), product of:
             0.101951346 = queryWeight, product of:
               1.0627038 = boost
               7.0962973 = idf(docFreq=99, maxDocs=44421)
               0.013519134 = queryNorm
             0.44351858 = fieldWeight in 3442, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0962973 = idf(docFreq=99, maxDocs=44421)
               0.0625 = fieldNorm(doc=3442)
         0.059160236 = weight(abstract_txt:classification in 3442) [ClassicSimilarity], result of:
           0.059160236 = score(doc=3442,freq=6.0), product of:
             0.096797995 = queryWeight, product of:
               1.7935339 = boost
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.013519134 = queryNorm
             0.61117214 = fieldWeight in 3442, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.0625 = fieldNorm(doc=3442)
         0.072409034 = weight(abstract_txt:classifying in 3442) [ClassicSimilarity], result of:
           0.072409034 = score(doc=3442,freq=1.0), product of:
             0.1758173 = queryWeight, product of:
               1.9736111 = boost
               6.58948 = idf(docFreq=165, maxDocs=44421)
               0.013519134 = queryNorm
             0.4118425 = fieldWeight in 3442, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.58948 = idf(docFreq=165, maxDocs=44421)
               0.0625 = fieldNorm(doc=3442)
         0.29878494 = weight(abstract_txt:task in 3442) [ClassicSimilarity], result of:
           0.29878494 = score(doc=3442,freq=16.0), product of:
             0.2436196 = queryWeight, product of:
               3.6733043 = boost
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.013519134 = queryNorm
             1.2264404 = fieldWeight in 3442, product of:
               4.0 = tf(freq=16.0), with freq of:
                 16.0 = termFreq=16.0
               4.9057617 = idf(docFreq=893, maxDocs=44421)
               0.0625 = fieldNorm(doc=3442)
       0.16 = coord(4/25)
   

5. Price, L.; Robinson, L.: Tag analysis as a tool for investigating information behaviour : comparing fan-tagging on Tumblr, Archive of Our Own and Etsy (2021) 0.07

   0.07433882 = sum of:
     0.07433882 = product of:
       0.46461764 = sum of:
         0.021133056 = weight(abstract_txt:classification in 1340) [ClassicSimilarity], result of:
           0.021133056 = score(doc=1340,freq=1.0), product of:
             0.096797995 = queryWeight, product of:
               1.7935339 = boost
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.013519134 = queryNorm
             0.21832122 = fieldWeight in 1340, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9921594 = idf(docFreq=2228, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1340)
         0.10188976 = weight(abstract_txt:taxonomy in 1340) [ClassicSimilarity], result of:
           0.10188976 = score(doc=1340,freq=3.0), product of:
             0.16733019 = queryWeight, product of:
               1.9253865 = boost
               6.428468 = idf(docFreq=194, maxDocs=44421)
               0.013519134 = queryNorm
             0.6089144 = fieldWeight in 1340, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.428468 = idf(docFreq=194, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1340)
         0.063357905 = weight(abstract_txt:classifying in 1340) [ClassicSimilarity], result of:
           0.063357905 = score(doc=1340,freq=1.0), product of:
             0.1758173 = queryWeight, product of:
               1.9736111 = boost
               6.58948 = idf(docFreq=165, maxDocs=44421)
               0.013519134 = queryNorm
             0.36036217 = fieldWeight in 1340, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.58948 = idf(docFreq=165, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1340)
         0.27823693 = weight(abstract_txt:dialogue in 1340) [ClassicSimilarity], result of:
           0.27823693 = score(doc=1340,freq=1.0), product of:
             0.68000376 = queryWeight, product of:
               6.7227516 = boost
               7.48196 = idf(docFreq=67, maxDocs=44421)
               0.013519134 = queryNorm
             0.40916967 = fieldWeight in 1340, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.48196 = idf(docFreq=67, maxDocs=44421)
               0.0546875 = fieldNorm(doc=1340)
       0.16 = coord(4/25)