Hoang Van Thuy
Network Science Lab
E-mail: hoangvanthuy90@gmail.com

1
• GFF, graph neural networks are trained
greedily layer by layer, using both positive
and negative samples
• Solve the noise from neighbours

2
The fundamentals: Permutation equivariance and invariance
➢ node feature matrix.
➢ adjacency matrix, A
➢ Neighbours of nodes
➢ Features
➢ Hidden state

3
Graph Neural Networks
➢ Main Idea: Pass massages between pairs of nodes and agglomerate
➢ Alternative Interpretation: Pass massages between nodes to refine node
(and possibly edge) representations

4
Graph Neural Networks
➢ Convolutional

5
Graph Neural Networks
➢ Attentional

6
Graph Neural Networks
➢ Message-passing

7
Graph Neural Networks
➢ Graph Convolutional Networks (GCNs)
Kipf & Welling (ICLR 2017), related previous works by D
uvenaud et al. (NIPS 2015) and Li et al. (ICLR 2016)

8
Tasks
➢ Node classification.
➢ Graph classification.
➢ Link prediction

9
GNNs without a graph: Deep Sets and Transformers
➢ Deep Sets

10
GNNs without a graph: Deep Sets and Transformers
➢ To reverse-engineer why Transformers appear here,
let us consider the NLP perspective.

11
GNNs beyond permutation equivariance: Geometric Graphs
➢ We have assumed our graphs to be a discrete, unordered, collection of
nodes and edges—hence, only susceptible to permutation symmetries.
➢ But in many cases, this is not the entire story!

12