Chapter 4 better.pptx

4
Chapter 4 better.pptx


Chapter 4 better.pptx

Chapter 4 better.pptx

  • 2. AGENDA
    • Abstract
    • Background
    and Problem Definition
    • Supervised Graph Neural Networks
    • General Frame work of Graph Neural Networks (GNN)
    • Graph Convolutional Networks (GCN)
    • Graph Attention Networks
    • Neural Message Passing Networks(MPNN)
    • Continuous Graph Neural Networks
    • Multi-Scale Spectral Graph Convolutional Networks
    • Unsupervised Graph Neural Networks
    • Variational Graph Auto-Encoders
    • Deep Graph Infomax
    • Over-smoothing Problem
  • 4. BACKGROUND AND PROBLEM DEFINITION
    • A fundamental task on graphs is node classification, which tries to
    classify the nodes into a few predefined categories.
    • To make effective prediction, a critical problem is to have very effective
    node representations, which largely determine the performance of node
    classification.
  • 5. BACKGROUND AND PROBLEM DEFINITION
    • N is the total number of nodes.
    • C is the number of features for each node.
    • F is the dimension of node representations.
    The goal of graph neural networks is
    to learn effective node representations
    (denoted as H ∈ R N×F) by combining
    the graph structure information and the node attributes, which are further used for
    node classification.
  • 6. Supervised Graph Neural Networks
    • General Frame work of Graph Neural Networks (GNN)
    • Graph Convolutional Networks (GCN)
    • Graph Attention Networks
    • Neural Message Passing Networks(MPNN)
    • Continuous Graph Neural Networks
    • Multi-Scale Spectral Graph Convolutional Networks
  • 7. 1- General Frame work of Graph Neural Networks (GNN)
    • The essential idea of graph neural networks is to iteratively update the
    node representations by combining the representations of their neighbors
    and their own representations.
  • 8. 1- General Frame work of Graph Neural Networks (GNN)
    Starting from the initial node representation H0 = X,
    in each layer we have two important functions:
    • AGGREGATE : which tries to aggregate the information from the neighbours
    of each node;
    • COMBINE : which tries to update the node representations by combining the
    aggregated information from neighbors with the current node
    representations.
  • 10. 2- Graph Convolutional Networks (GCN)
    • Which is now the most popular graph neural network architecture due to its
    simplicity and effectiveness in a variety of tasks and applications.
    • Specifically, the node representations in each layer is updated according to a
    propagation rule.
  • 11. 2- Graph Convolutional Networks (GCN)
    • AGGREGATE function is defined as the weighted average of the
    neighbor node representations. The weight of the neighbor j is
    determined by the weight of the edge between i and j (i.e. Ai j
    normalized by the degrees of the two nodes)
    • COMBINE function is defined as the summation of the aggregated
    messages and the node representation itself, in which the node
    representation is normalized by its own degree.
  • 12. 3- Graph Attention Networks
    • In GCNs, for a target node i, the importance of a neighbor j is determined
    by the weight of their edge Ai j (normalized by their node degrees).
    • However, in practice, the input graph may be noisy.
    • The edge weights may not be able to reflect the true strength between two
    nodes.
    • As a result, a more principled approach might be to automatically learn the
    importance of each neighbor. Graph Attention Networks (a.k.a. GAT) is built
    on this idea and try to learn the importance of each neighbor based on the
    Attention mechanism.
  • 13. 3- Graph Attention Networks
    • Graph Attention Layer: defnes how to transfer the hidden node representations at layer k −1 (denoted
    as H k−1) to the new node representations H.
    • In order to guarantee suffcient expressive power to transform the lower-level node representations to
    higher-level node representations, a shared linear transformation is applied to every node.
    • Afterwards, self-attention is defned on the nodes, which measures the attention coeffcients for any
    pair of nodes through a shared attentional mechanism (indicates the relationship strength between
    node i and j).
    • And to make the coeffcients comparable across different nodes, the attention coeffcients are usually
    normalized with the softmax function:
    • The new node representation is a linear combination of the neighboring node representations with the
    weights determined by the attention coeffcients (with a potential nonlinear transformation)
  • 14. 4- Neural Message Passing Networks(MPNN)
    • MPNN is actually very general, provides a general framework of
    graph neural networks, and can be used for the task of node
    classifcation as well.
    • The essential idea of MPNN is formulating existing graph neural
    networks as a general framework of neural message passing
    among nodes.
    • In MPNNs, there are two important functions including
    Message and Updating function:
  • 15. 4- Neural Message Passing Networks(MPNN)
    • The MPNN framework is very similar to the general framework
    • The AGGREGATE function defned here is simply a summation of all the
    messages from the neighbors.
    • The COMBINE function is the same as the node Updating function: which
    combines the aggregated messages from the neighbors and the node
    representation itself.
  • 16. 5- Continuous Graph Neural Networks
    • The above graph neural networks iteratively update the node representations with different
    kinds of graph convolutional layers. Essentially, these approaches model the discrete
    dynamics of node representations with GNNs.
    • Xhonneux et al (2020) proposed the continuous graph neural networks (CGNNs), which
    generalizes existing graph neural networks with discrete dynamics to continuous settings, i.e.,
    trying to model the continuous dynamics of node representations.
    • The key idea is how to characterize the continuous dynamics of node representations, i.e. the
    derivatives of node representation w.r.t. time.
    • The derivatives of the node representations are defned as a combination of the node
    representation itself, the representations of its neighbors, and the initial status of the nodes.
  • 17. 5- Continuous Graph Neural Networks
    • Specifcally, two different variants of node dynamics are
    introduced.
    • The frst model assumes that different dimensions of node
    presentations (a.k.a. feature channels) are independent;
    • The second model is more fexible, which allows different feature
    channels to interact with each other.
  • 20. VARIATIONAL GRAPH AUTO-ENCODERS
    The
    VGAE model consists of an encoder, a decoder, and a prior .
    • Encoder
    The goal of the encoder is to learn a distribution of latent variables associated with each node
    conditioning on the node features X and the adjacency matrix A. (qφ (Z|X,A)).
    • Decoder
    Given sampled latent variables, the decoder aims at predicting the connectivities among nodes.
    Aopts a simple dot-product based predictor.
    • Prior
    The prior distributions over the latent variables are simply set to independent zero-mean Gaussians with unit variances,
    This prior is fixed throughout the learning.
  • 21. DEEP GRAPH INFOMAX
    Deep Graph Infomax is an unsupervised learning framework that
    learns graph representations via the principle of mutual information
    maximization.
  • 22. OVER-SMOOTHING PROBLEM
    • Training
    deep graph neural networks by stacking multiple layers of graph neural networks
    usually yields inferior results, which is a common problem observed in several different
    graph neural network architectures.
    • This is mainly due to the problem of over-smoothing.
    • The graph convolutional network is a special case of Laplacian smoothing:
    • The Laplacian smoothing will push nodes belonging to the same clusters to take similar
    representations, which are benefcial for downstream tasks such as node classifcation.
    • However, when the GCNs go deep, the node representations suffer from the problem of
    over-smoothing, i.e., all the nodes will have similar representations. As a result, the
    performance on downstream tasks suffer as well.
  • 23. OVER-SMOOTHING PROBLEM
    • PairNorm:
    a method for alleviating the problem of over-smoothing
    when GNNs go deep.
    • The essential idea of PairNorm is to keep the total pairwise squared
    distance (TPSD) of node representations unchanged, which is the
    same as that of the original node feature X.


Disasters Expo USA, is proud to be supported by Inergency for their next upcoming edition on March 6th & 7th 2024!

The leading event mitigating the world’s most costly disasters is returning to the Miami Beach

Convention Center and we want you to join us at the industry’s central platform for emergency management professionals.
Disasters Expo USA is proud to provide a central platform for the industry to connect and
engage with the industry’s leading professionals to better prepare, protect, prevent, respond
and recover from the disasters of today.
Hosting a dedicated platform for the convergence of disaster risk reduction, the keynote line up for Disasters Expo USA 2024 will provide an insight into successful case studies and
programs to accurately prepare for disasters. Featuring sessions from the likes of The Federal Emergency Management Agency,
NASA, The National Aeronautics and Space Administration, NOAA, The National Oceanic and Atmospheric Administration, TSA and several more this event is certainly providing you with the knowledge
required to prepare, respond and recover to disasters.
With over 50 hours worth of unmissable content, exciting new features such as their Disaster
Resilience Roundtable, Emergency Response Live, an Immersive Hurricane Simulation and
much more over just two days, you are guaranteed to gain an all-encompassing insight into
the industry to tackle the challenges of disasters.
By uniting global disaster risk management experts, well experienced emergency
responders and the leading innovators from the world, the event is the hub of the solutions
that provide attendees with tools that they can use to protect the communities and mitigate
the damage from disasters.
Tickets for the event are $119, but we have been given the promo code: HUGI100 that will
enable you to attend the event for FREE!

So don’t miss out and register today: https://shorturl.at/aikrW

And in case you missed it, here is our ultimate road trip playlist is the perfect mix of podcasts, and hidden gems that will keep you energized for the entire journey

-

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More