پیام رسان بله
Design and Validation of Large Bilingual (Persian-English) Language Models for Medical Specializations Using Transfer Learning and Fine-Tuning Optimization
Design and Validation of Large Bilingual (Persian-English) Language Models for Medical Specializations Using Transfer Learning and Fine-Tuning Optimization
Mahdi Khoursha1
1) MSc in Computer Engineering, Iran University of Science and Technology, Tehran, Iran
IOI Code
:
ABGZ-ABHBZK
Publication :
The third international conference on new approaches in engineering and basic sciences(icnabs.ir)
Abstract :
Recent advances in the field of large language models have promised to revolutionize the applications of artificial intelligence in medicine. However, most of these models are focused on English and do not perform reliably in low-resource languages such as Persian. This study designs and validates large bilingual (Persian-English) language models for medical specializations using a transfer learning and fine-tuning optimization approach. The present study uses the PersianMedQA specialized dataset consisting of 20,785 expert-validated multiple-choice medical questions from 14 years of Iranian national medical exams. The performance of the top 40 large language models, including general models, Persian fine-tuned models, and medical specialized models, was evaluated in two zero-shot and chain of thought modes. The results showed that open-source closed-source models such as GPT-4.1 outperformed the other models with 83.09% accuracy in Persian and 80.7% in English, while Persian fine-tuned models such as Dorna performed very poorly with 34.9% accuracy in Persian. It was also found that between 3 and 10% of questions could only be answered correctly in Persian due to the presence of contextual cultural and clinical cues that are lost in translation. The findings of this study indicate that simply increasing the model size without sufficient linguistic and domain matching is not sufficient to achieve reliable clinical performance in Persian, and that the development of localized models with transfer learning approaches and targeted fine-tuning is essential.
Keywords : large language models medicine persian language transfer learning fine tuning bilingual assessment