Document (#44381)

Author

Wei, Y.

Title

Analysis of accelerating ideation process with Large Language Model

Source

Applied and computational engineering. 102(2024) no.1, p.162-168 [Proceedings of the 2nd International Conference on Machine Learning and Automation. DOI: 10.54254/2755-2721/102/20241198]

Year

2024

Abstract

Large Language Models (LLMs) have rapidly advanced, demonstrating near-human intelligence in language comprehension, problem-solving, and autonomous decision-making, which can be leveraged in scientific discovery. Previous research of LLM for science often focuses on experiment execution part and the first stage of research, i.e., idea generation with large language models is lack of research. This paper fills this gap by surveying the use of LLM-based agent systems in research idea generation. This research first proposes three stages in research idea generation pipeline, pre-ideation (knowledge preparation), ideation (generation and iteration), post-ideation (evaluation), and provide detailed summary in different methods in each stage, then it summarizes different metrics and angles of evaluating generated research ideas, finally it discusses limitations and ethical concerns of existing works and suggest potential solutions and future directions. These results aim to provide a solid foundation for future research in improving LLM-based ideation systems and fostering responsible AI usage in scientific discovery.

Content

Vgl.: https://www.researchgate.net/publication/385727218_Analysis_of_Accelerating_Ideation_Process_with_Large_Language_Model.

Theme

Computerlinguistik

Similar documents (content)

1. Khorashadizadeh, H.; Amara, F.Z.; Ezzabady, M.; Ieng, F.; Tiwari, S.; Mihindukulasooriya, N.; Groppe, J.; Sahri, S.; Benamara, F.; Groppe, S.: Research trends for the interplay between Large Language Models and Knowledge Graphs (2024) 0.16

   0.16310355 = sum of:
     0.16310355 = product of:
       0.5096986 = sum of:
         0.017361635 = weight(abstract_txt:provide in 2335) [ClassicSimilarity], result of:
           0.017361635 = score(doc=2335,freq=1.0), product of:
             0.05536444 = queryWeight, product of:
               1.019692 = boost
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.013526713 = queryNorm
             0.3135882 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.020394627 = weight(abstract_txt:future in 2335) [ClassicSimilarity], result of:
           0.020394627 = score(doc=2335,freq=1.0), product of:
             0.06163788 = queryWeight, product of:
               1.0759135 = boost
               4.23524 = idf(docFreq=1747, maxDocs=44421)
               0.013526713 = queryNorm
             0.33087814 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.23524 = idf(docFreq=1747, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.19980796 = weight(abstract_txt:llms in 2335) [ClassicSimilarity], result of:
           0.19980796 = score(doc=2335,freq=4.0), product of:
             0.14110716 = queryWeight, product of:
               1.1511 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.013526713 = queryNorm
             1.4160016 = fieldWeight in 2335, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.026527043 = weight(abstract_txt:models in 2335) [ClassicSimilarity], result of:
           0.026527043 = score(doc=2335,freq=1.0), product of:
             0.07344514 = queryWeight, product of:
               1.1744515 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013526713 = queryNorm
             0.36118174 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.035303 = weight(abstract_txt:large in 2335) [ClassicSimilarity], result of:
           0.035303 = score(doc=2335,freq=1.0), product of:
             0.10172054 = queryWeight, product of:
               1.6927916 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.013526713 = queryNorm
             0.3470587 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.06748216 = weight(abstract_txt:language in 2335) [ClassicSimilarity], result of:
           0.06748216 = score(doc=2335,freq=3.0), product of:
             0.11956372 = queryWeight, product of:
               2.1191819 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.013526713 = queryNorm
             0.5644033 = fieldWeight in 2335, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.09492111 = weight(abstract_txt:generation in 2335) [ClassicSimilarity], result of:
           0.09492111 = score(doc=2335,freq=1.0), product of:
             0.2164823 = queryWeight, product of:
               2.8515418 = boost
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.013526713 = queryNorm
             0.43847054 = fieldWeight in 2335, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
         0.0479011 = weight(abstract_txt:research in 2335) [ClassicSimilarity], result of:
           0.0479011 = score(doc=2335,freq=2.0), product of:
             0.13721792 = queryWeight, product of:
               3.2106204 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013526713 = queryNorm
             0.34908777 = fieldWeight in 2335, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.078125 = fieldNorm(doc=2335)
       0.32 = coord(8/25)
   

2. Xiao, L.: Effects of rationale awareness in online ideation crowdsourcing tasks (2014) 0.14

   0.14255768 = sum of:
     0.14255768 = product of:
       0.8909855 = sum of:
         0.0282424 = weight(abstract_txt:large in 2329) [ClassicSimilarity], result of:
           0.0282424 = score(doc=2329,freq=1.0), product of:
             0.10172054 = queryWeight, product of:
               1.6927916 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.013526713 = queryNorm
             0.27764696 = fieldWeight in 2329, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=2329)
         0.110371 = weight(abstract_txt:idea in 2329) [ClassicSimilarity], result of:
           0.110371 = score(doc=2329,freq=3.0), product of:
             0.17498623 = queryWeight, product of:
               2.220246 = boost
               5.8265367 = idf(docFreq=355, maxDocs=44421)
               0.013526713 = queryNorm
             0.63074106 = fieldWeight in 2329, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.8265367 = idf(docFreq=355, maxDocs=44421)
               0.0625 = fieldNorm(doc=2329)
         0.07593689 = weight(abstract_txt:generation in 2329) [ClassicSimilarity], result of:
           0.07593689 = score(doc=2329,freq=1.0), product of:
             0.2164823 = queryWeight, product of:
               2.8515418 = boost
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.013526713 = queryNorm
             0.35077643 = fieldWeight in 2329, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.0625 = fieldNorm(doc=2329)
         0.67643523 = weight(abstract_txt:ideation in 2329) [ClassicSimilarity], result of:
           0.67643523 = score(doc=2329,freq=2.0), product of:
             0.7953617 = queryWeight, product of:
               6.11091 = boost
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.013526713 = queryNorm
             0.850475 = fieldWeight in 2329, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.0625 = fieldNorm(doc=2329)
       0.16 = coord(4/25)
   

3. Kau, A.; He, X.; Nambissan, A.; Astudillo, A.; Yin, H.; Aryani, A.: Combining graphs and Large Language Models (2024) 0.10

   0.09883088 = sum of:
     0.09883088 = product of:
       0.41179535 = sum of:
         0.013889308 = weight(abstract_txt:provide in 2336) [ClassicSimilarity], result of:
           0.013889308 = score(doc=2336,freq=1.0), product of:
             0.05536444 = queryWeight, product of:
               1.019692 = boost
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.013526713 = queryNorm
             0.25087056 = fieldWeight in 2336, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.0625 = fieldNorm(doc=2336)
         0.17871368 = weight(abstract_txt:llms in 2336) [ClassicSimilarity], result of:
           0.17871368 = score(doc=2336,freq=5.0), product of:
             0.14110716 = queryWeight, product of:
               1.1511 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.013526713 = queryNorm
             1.2665104 = fieldWeight in 2336, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0625 = fieldNorm(doc=2336)
         0.021221634 = weight(abstract_txt:models in 2336) [ClassicSimilarity], result of:
           0.021221634 = score(doc=2336,freq=1.0), product of:
             0.07344514 = queryWeight, product of:
               1.1744515 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013526713 = queryNorm
             0.28894538 = fieldWeight in 2336, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=2336)
         0.0282424 = weight(abstract_txt:large in 2336) [ClassicSimilarity], result of:
           0.0282424 = score(doc=2336,freq=1.0), product of:
             0.10172054 = queryWeight, product of:
               1.6927916 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.013526713 = queryNorm
             0.27764696 = fieldWeight in 2336, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=2336)
         0.06233735 = weight(abstract_txt:language in 2336) [ClassicSimilarity], result of:
           0.06233735 = score(doc=2336,freq=4.0), product of:
             0.11956372 = queryWeight, product of:
               2.1191819 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.013526713 = queryNorm
             0.52137345 = fieldWeight in 2336, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0625 = fieldNorm(doc=2336)
         0.10739097 = weight(abstract_txt:generation in 2336) [ClassicSimilarity], result of:
           0.10739097 = score(doc=2336,freq=2.0), product of:
             0.2164823 = queryWeight, product of:
               2.8515418 = boost
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.013526713 = queryNorm
             0.49607277 = fieldWeight in 2336, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.612423 = idf(docFreq=440, maxDocs=44421)
               0.0625 = fieldNorm(doc=2336)
       0.24 = coord(6/25)
   

4. Ghali, M.-K.; Farrag, A.; Won, D.; Jin, Y.: Enhancing knowledge retrieval with in-context learning and semantic search through Generative AI (2024) 0.09

   0.091561764 = sum of:
     0.091561764 = product of:
       0.3270063 = sum of:
         0.012153144 = weight(abstract_txt:provide in 2367) [ClassicSimilarity], result of:
           0.012153144 = score(doc=2367,freq=1.0), product of:
             0.05536444 = queryWeight, product of:
               1.019692 = boost
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.013526713 = queryNorm
             0.21951173 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.051336948 = weight(abstract_txt:execution in 2367) [ClassicSimilarity], result of:
           0.051336948 = score(doc=2367,freq=1.0), product of:
             0.11482819 = queryWeight, product of:
               1.0383955 = boost
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.013526713 = queryNorm
             0.44707617 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.175107 = idf(docFreq=33, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.13986558 = weight(abstract_txt:llms in 2367) [ClassicSimilarity], result of:
           0.13986558 = score(doc=2367,freq=4.0), product of:
             0.14110716 = queryWeight, product of:
               1.1511 = boost
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.013526713 = queryNorm
             0.99120116 = fieldWeight in 2367, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               9.06241 = idf(docFreq=13, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.01856893 = weight(abstract_txt:models in 2367) [ClassicSimilarity], result of:
           0.01856893 = score(doc=2367,freq=1.0), product of:
             0.07344514 = queryWeight, product of:
               1.1744515 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013526713 = queryNorm
             0.2528272 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.04280261 = weight(abstract_txt:large in 2367) [ClassicSimilarity], result of:
           0.04280261 = score(doc=2367,freq=3.0), product of:
             0.10172054 = queryWeight, product of:
               1.6927916 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.013526713 = queryNorm
             0.4207863 = fieldWeight in 2367, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.038569268 = weight(abstract_txt:language in 2367) [ClassicSimilarity], result of:
           0.038569268 = score(doc=2367,freq=2.0), product of:
             0.11956372 = queryWeight, product of:
               2.1191819 = boost
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.013526713 = queryNorm
             0.32258338 = fieldWeight in 2367, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1709876 = idf(docFreq=1863, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
         0.023709835 = weight(abstract_txt:research in 2367) [ClassicSimilarity], result of:
           0.023709835 = score(doc=2367,freq=1.0), product of:
             0.13721792 = queryWeight, product of:
               3.2106204 = boost
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.013526713 = queryNorm
             0.17278963 = fieldWeight in 2367, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.159582 = idf(docFreq=5124, maxDocs=44421)
               0.0546875 = fieldNorm(doc=2367)
       0.28 = coord(7/25)
   

5. Cohan, A.; Young, S.; Yates, A.; Goharian, N.: Triaging content severity in online mental health forums (2017) 0.09

   0.088538185 = sum of:
     0.088538185 = product of:
       0.5533637 = sum of:
         0.013889308 = weight(abstract_txt:provide in 4930) [ClassicSimilarity], result of:
           0.013889308 = score(doc=4930,freq=1.0), product of:
             0.05536444 = queryWeight, product of:
               1.019692 = boost
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.013526713 = queryNorm
             0.25087056 = fieldWeight in 4930, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.013929 = idf(docFreq=2180, maxDocs=44421)
               0.0625 = fieldNorm(doc=4930)
         0.021221634 = weight(abstract_txt:models in 4930) [ClassicSimilarity], result of:
           0.021221634 = score(doc=4930,freq=1.0), product of:
             0.07344514 = queryWeight, product of:
               1.1744515 = boost
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.013526713 = queryNorm
             0.28894538 = fieldWeight in 4930, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.623126 = idf(docFreq=1185, maxDocs=44421)
               0.0625 = fieldNorm(doc=4930)
         0.039940782 = weight(abstract_txt:large in 4930) [ClassicSimilarity], result of:
           0.039940782 = score(doc=4930,freq=2.0), product of:
             0.10172054 = queryWeight, product of:
               1.6927916 = boost
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.013526713 = queryNorm
             0.3926521 = fieldWeight in 4930, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4423513 = idf(docFreq=1420, maxDocs=44421)
               0.0625 = fieldNorm(doc=4930)
         0.47831196 = weight(abstract_txt:ideation in 4930) [ClassicSimilarity], result of:
           0.47831196 = score(doc=4930,freq=1.0), product of:
             0.7953617 = queryWeight, product of:
               6.11091 = boost
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.013526713 = queryNorm
             0.60137665 = fieldWeight in 4930, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.622026 = idf(docFreq=7, maxDocs=44421)
               0.0625 = fieldNorm(doc=4930)
       0.16 = coord(4/25)