Название: Attention Augmented Learning Machines: Theory and Applications
Автор: Guoqiang Zhong, Jinxuan Sun
Издательство: Nova Science Publishers, Inc.
Год: 2023
Страниц: 140
Язык: английский
Формат: pdf (true)
Размер: 35.6 MB

Deep Learning has developed for more than 10 years. Many novel models are proposed. Among others, the attention models have greatly impacted the Deep Learning area. Similar to the attention mechanism of human beings, the attention mechanism improves the performance of many Deep Learning models based on its discovery of important information hidden in data and motivates the emergence of many new Deep Learning models, like Transformer and its variants.

This book includes eight chapters and aims to introduce some interesting works on the attention mechanism. Chapter 1 is a review of the attention mechanism used in the Deep Learning area, while Chapters 2 and 3 present two models that integrate the attention mechanism into gated recurrent units (GRUs) and long short-term memory (LSTM), respectively, making them pay attention to important information in the sequences. Chapter 4 designs a multi-attention fusion mechanism and uses it for industrial surface defect detection. Chapter 5 enhances Transformer for object detection applications. Moreover, Chapter 6 proposes a dual-path architecture called dual-path mutual attention network (DPMAN) for medical image classification, and Chapter 7 proposes a novel graph model called attention-gated graph neural network (AGGNN) for text classification. In addition, Chapter 8 combines the generative adversarial networks (GAN), LSTM, and the attention mechanism to build a generative model for stock price prediction. These chapters introduce new designs of the attention mechanism and demonstrate their effectiveness using extensive experiments and ablation studies on various applications.

This book can be used by college students (undergraduate or graduate) chosen to major in Computer Science, Artificial Intelligence, electrical engineering, and mathematics, or others who study or have the potential to use Deep Learning algorithms. It could be of special interest to professors who research pattern recognition, Machine Learning, computer vision, neural language processing (NLP), and related fields, or engineers who apply Deep Learning models to their products. On the other hand, the reader is assumed to be already familiar with basic computer programming, Machine Learning, pattern recognition, and computer vision.