Attention-mechanism

Published on
November 21, 2024
Understanding Scaled Dot-Product Attention in Neural Networks for Causal Discovery
Causal-Discovery Transformer-Neural-Network Attention-Mechanism Layer-Normalization Deep-Learning ADIA-Lab
This post explores a neural network model designed for the Causal Discovery Challenge organized by ADIA Lab. It highlights the use of a Transformer-based architecture with two layers of scaled dot-product attention and layer normalization, achieving a multi-balanced accuracy of 47.986%.

Understanding Scaled Dot-Product Attention in Neural Networks for Causal Discovery