A Persuasive AI Explanation of Organic Reaction Mechanisms with Minor Inaccuracies but High Circuit Explanation Complexity

A Persuasive AI Explanation of Organic Reaction Mechanisms with Minor Inaccuracies but High Circuit Explanation Complexity

Mohammadesmail Nikfar1

1) Ph.D. Graduate in Mathematics, Doctor of Philosophy (Ph.D.) in Pure Mathematics, Mathematics Teacher, District 4 Department of Education, Tehran, Iran

IOI Code : ADAE-ACADDC
Publication : The 3rd International Conference on New Research in Educational Systems and Pedagogy, Tbilisi, Georgia(https://confms.ir/3rd)
Abstract :
    The chemistry education research community values ​​and emphasizes the construction of explanations and mechanistic reasoning to support students' learning of organic chemistry. Emerging technologies—specifically large language models and generative artificial intelligence—are uniquely positioned to advance the teaching and learning of chemistry. Generative AI-based chatbots, such as ChatGPT, have the potential to assist students in learning mechanistic reasoning through the responses they generate. This study examines the extent to which ChatGPT provides accurate explanations for a variety of reaction mechanisms. It further classifies mechanistic reasoning based on the complexity levels of the provided explanations. Additionally, the study investigates the impact of prompt engineering on the accuracy of reaction mechanisms and the complexity of explanations, utilizing three distinct types of prompting strategies. The findings indicate that one-quarter of the responses contain entirely accurate explanations of reaction mechanisms, while the majority offer largely accurate descriptions of chemical phenomena, including the identification of nucleophiles and electrophiles. The responses demonstrate high levels of explanatory complexity. While prompt engineering plays a significant role in eliciting higher levels of explanatory complexity, it does not necessarily ensure the accuracy of the described mechanisms. These results are discussed within the frameworks of mechanistic reasoning and prompt engineering, with a focus on how these new technologies can be integrated into the chemistry classroom.

Keywords : organic chemistry mechanical reasoning generative artificial intelligence chemistry education research testing and evaluation role