Optimization of transformer models for focused applications in language processing and new technologies

Optimization of transformer models for focused applications in language processing and new technologies

Kaveh Kavianpour1 Behnam Barzegar2

1) Master of Science in Computer Engineering, Amirkabir University
2) Assistant Professor, Babol Branch, Azad University

IOI Code : AADG-AACAFE
Publication : The second international conference on new approaches in engineering and basic sciences(icnabs.ir/2nd)
Abstract :
    Transformer-based language models, such as BERT and GPT, have revolutionized the field of natural language processing and are widely used in various projects. However, effective utilization of these models in specialized areas requires careful tuning and optimization. In the present study, various methods for improving the performance of these models in specific areas are investigated. Methods such as fine-tuning, transfer learning, model compression, and structural modification are investigated and analyzed to determine the effect of each on the efficiency of the model in specific applications. Experimental results show that the use of these techniques can significantly improve the accuracy and processing speed in specialized projects. Finally, by comparing the results, optimal solutions for the development and application of transformer models in focused areas are presented, which can play an important role in advancing language technologies and making language-based systems intelligent.

Keywords : large language models transformers fine tuning transfer learning model compression architectural optimization