Deep Learning in Artificial Intelligence: From Fundamentals to Advanced Applications and Modern Challenges

Deep Learning in Artificial Intelligence: From Fundamentals to Advanced Applications and Modern Challenges

Yaldamesgar1

1) Master s degree in Software Engineering, Islamic Azad University, Ayatollah Amoli Branch

IOI Code : ABEB-ACAGEE
Publication : 4th International Conference on Recent Advances in Engineering, Innovation and Technology - Belgium(eitconf.com)
Abstract :
    Background and Objective Deep learning, as a fundamental pillar of recent advances in artificial intelligence, has brought about revolutionary changes across various domains. However, the pace of evolution is such that providing an up-to-date and coherent picture of concepts ranging from fundamentals to the most advanced architectures seems essential, especially for graduate students. This article aims to present a structured and practical overview of the most significant developments in deep learning between 2020 and 2026. Review Method A systematic search was conducted in Web of Science, Scopus, IEEE Xplore, and arXiv databases using relevant keywords. Out of 387 initial papers, 75 were selected for final analysis based on journal credibility, novelty (2024–2026), and direct relevance to the topic. Findings The review revealed that: (1) The Transformer architecture, through the introduction of linear attention mechanisms and successor architectures such as Nexusformer, has largely overcome its scalability limitations. (2) Diffusion models, moving from U-Net structures to Transformer-based architectures (DiT) and offering fast sampling methods, have become the gold standard in generative models. (3) Self-supervised learning (SSL), by providing more robust theoretical frameworks such as latent distribution matching and LeJEPA, has taken a major step toward more reliable learning. (4) Graph neural networks (GNNs), through the integration of domain knowledge and self-supervised pre-training, have become a key tool in drug discovery and molecular design. (5) Concepts such as Green AI and Out-of-Distribution (OOD) Generalization have emerged as the most important challenges facing the field. Conclusion The era of massive, centralized foundation models is giving way to efficient, open-source, and specialized models. Future research will focus on environmentally sustainable AI, autonomous intelligent agent systems, and deeper integration with the empirical sciences.

Keywords : deep learning linear transformer diffusion models self supervised learning graph neural networks drug discovery green ai