Attentions, attentions!

Section contains Attention-based models in theory and implementation.

The Motif

A foundational reference is the influential paper “Attention Is All You Need” by Vaswani et al. (2017), which introduced the Transformer architecture and sparked widespread adoption of attention mechanisms in modern deep learning.

Intro

This section explores attention mechanisms in applications as standalone models or as components of hybrid models, for e.g. Graph Attention Network (GAT). The Q,K,V learning algorithm is trialed across different machine learning problems primarily dealing with time-series like datasets and NLP.