Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mathstodon.xyz is one of the many independent Mastodon servers you can use to participate in the fediverse.
A Mastodon instance for maths people. We have LaTeX rendering in the web interface!

Administered by:

Server stats:

2.8K
active users

mathstodon.xyz: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#graphneuralnetworks

0 posts0 participants0 posts today
Public
💧🌏 Greg Cocks

Interpretable Physics-Informed Graph Neural Networks For Flood Forecasting
--
doi.org/10.1111/mice.13484 <-- shared paper
--
github.com/MehdiTaghizadehUVa/ <-- shared GitHub repository
--
#GIS #spatial #mapping #FloodForecasting #GraphNeuralNetworks #PhysicsInformedAI #Hydrology #ClimateResilience #ScientificMachineLearning #CivilEngineering #HydroGraphNet #PhyiscsNeMo #hydrology #water #flood #flooding #spatialanalysis #spatiotemporal #climatechange #model #modeling #machinelearning #AI #extremeweather #prediction #forecasting #planning #humanimpacts #loss #lossoflife #publicsafety #cost #economics #infrastructure #risk #hazard #disaster #mitigation #preparedness #naturaldisaster #HydroGraphNet #floodforecasting #dynamics #remotesensing #earthobservation

Public
Victoria Stuart 🇨🇦 🏳️‍⚧️

Interpretable Graph Neural Networks for Tabular Data
arxiv.org/abs/2308.08945
Discussion: news.ycombinator.com/item?id=3

* GNN essentially deep NN black-box models
* IGNNet: Interpretable Graph Neural Network for tab data
* notable HN comment, resp. to critique: " Right, the significance of orig. article & related research is ChatGPT-like models don't handle tabular data well & there's need for things that do"

Interpretable Graph Neural Networks for Tabular Data Figure 1: An overview of our proposed approach. Each data instance is represented as a graph by embedding the feature values into a higher dimensionality, and the edge between two features (nodes) is the correlation value. Multiple iterations of message passing are then applied. Finally, the learned node representation is projected into a single value, and a whole graph representation is obtained by concatenating the projected values. Figure 2: IGNNet default architecture. It starts with the embedding layer, a linear transformation from one dimension to 64 dimensions. A Relu activation function follows each message-passing layer and each green block as well. The feedforward network at the end has no activation functions between layers to ensure a linear transformation into a single value. A sigmoid activation function follows the feedforward network to obtain the final value for each feature between 0 and 1. Article: https://arxiv.org/pdf/2308.08945.pdf Discussion (Hacker News): https://news.ycombinator.com/item?id=37269376
#DataProcessing#GraphNeuralNetworks#GNN
Continued thread
Public *
Victoria Stuart 🇨🇦 🏳️‍⚧️

Addendae 4

Graph Structure f. Point Clouds: Geometric Attention All You Need
arxiv.org/abs/2307.16662

* partly intersect my interest in knowledge graphs, fully-connected networks, and transformers in NLP and ML
* length of that discussion - though brief - exceeds the Mastodon 500-char limit, so I moved that discussion here:

Fully-connected graphs: Graph neural networks, transformers
persagen.com/docs/gnn-transfor

arXiv.orgGraph Structure from Point Clouds: Geometric Attention is All You NeedThe use of graph neural networks has produced significant advances in point cloud problems, such as those found in high energy physics. The question of how to produce a graph structure in these problems is usually treated as a matter of heuristics, employing fully connected graphs or K-nearest neighbors. In this work, we elevate this question to utmost importance as the Topology Problem. We propose an attention mechanism that allows a graph to be constructed in a learned space that handles geometrically the flow of relevance, providing one solution to the Topology Problem. We test this architecture, called GravNetNorm, on the task of top jet tagging, and show that it is competitive in tagging accuracy, and uses far fewer computational resources than all other comparable models.
#GNN#attention#transformers
Public
Victoria Stuart 🇨🇦 🏳️‍⚧️

Hierarchical GNNs for Large Graph Generation
arxiv.org/abs/2306.11412

Large graphs are present in a variety of domains, including social networks, civil infrastructure, & the physical sciences. ... Graph generation is similarly widespread, with applications in drug discovery, network analysis & synthetic datasets among others. While GNN (Graph Neural Network) models have been applied in these domains their high in-memory costs restrict them to small graphs. ...

arXiv.orgSize Matters: Large Graph Generation with HiGGsLarge graphs are present in a variety of domains, including social networks, civil infrastructure, and the physical sciences to name a few. Graph generation is similarly widespread, with applications in drug discovery, network analysis and synthetic datasets among others. While GNN (Graph Neural Network) models have been applied in these domains their high in-memory costs restrict them to small graphs. Conversely less costly rule-based methods struggle to reproduce complex structures. We propose HIGGS (Hierarchical Generation of Graphs) as a model-agnostic framework of producing large graphs with realistic local structures. HIGGS uses GNN models with conditional generation capabilities to sample graphs in hierarchies of resolution. As a result HIGGS has the capacity to extend the scale of generated graphs from a given GNN model by quadratic order. As a demonstration we implement HIGGS using DiGress, a recent graph-diffusion model, including a novel edge-predictive-diffusion variant edge-DiGress. We use this implementation to generate categorically attributed graphs with tens of thousands of nodes. These HIGGS generated graphs are far larger than any previously produced using GNNs. Despite this jump in scale we demonstrate that the graphs produced by HIGGS are, on the local scale, more realistic than those from the rule-based model BTER.
#GNN#graphs#GraphNeuralNetworks
Public
Dominic Boutet

#arxivfeed :

"CI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric Diagnosis"
arxiv.org/abs/2301.01642

arXiv.orgCI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric DiagnosisThere is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which,in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs. However, most of the existing GNN explainers are either post-hoc in which another interpretive model needs to be created to explain a well-trained GNN, or do not consider the causal relationship between the extracted explanation and the decision, such that the explanation itself contains spurious correlations and suffers from weak faithfulness. In this work, we propose a granger causality-inspired graph neural network (CI-GNN), a built-in interpretable model that is able to identify the most influential subgraph (i.e., functional connectivity within brain regions) that is causally related to the decision (e.g., major depressive disorder patients or healthy controls), without the training of an auxillary interpretive network. CI-GNN learns disentangled subgraph-level representations α and \b{eta} that encode, respectively, the causal and noncausal aspects of original graph under a graph variational autoencoder framework, regularized by a conditional mutual information (CMI) constraint. We theoretically justify the validity of the CMI regulation in capturing the causal relationship. We also empirically evaluate the performance of CI-GNN against three baseline GNNs and four state-of-the-art GNN explainers on synthetic data and three large-scale brain disease datasets. We observe that CI-GNN achieves the best performance in a wide range of metrics and provides more reliable and concise explanations which have clinical evidence.The source code and implementation details of CI-GNN are freely available at GitHub repository (https://github.com/ZKZ-Brain/CI-GNN/).
#Neuroscience#Neuro#ComputationalNeuroscience
Public
Rαhul Bhαdαni

ROLAND: A new framework to repurpose static GNNs to dynamic GNNs.

In real life, interaction graphs are often dynamic: edges and nodes change with time 🕸️

ROLAND ia built with PyG @PyTorch GraphGym to efficiently explore the GNN design space.

arxiv.org/abs/2208.07239

github.com/snap-stanford/rolan

arXiv.orgROLAND: Graph Learning Framework for Dynamic GraphsGraph Neural Networks (GNNs) have been successfully applied to many real-world static graphs. However, the success of static graphs has not fully translated to dynamic graphs due to the limitations in model design, evaluation settings, and training strategies. Concretely, existing dynamic GNNs do not incorporate state-of-the-art designs from static GNNs, which limits their performance. Current evaluation settings for dynamic GNNs do not fully reflect the evolving nature of dynamic graphs. Finally, commonly used training methods for dynamic GNNs are not scalable. Here we propose ROLAND, an effective graph representation learning framework for real-world dynamic graphs. At its core, the ROLAND framework can help researchers easily repurpose any static GNN to dynamic graphs. Our insight is to view the node embeddings at different GNN layers as hierarchical node states and then recurrently update them over time. We then introduce a live-update evaluation setting for dynamic graphs that mimics real-world use cases, where GNNs are making predictions and being updated on a rolling basis. Finally, we propose a scalable and efficient training approach for dynamic GNNs via incremental training and meta-learning. We conduct experiments over eight different dynamic graph datasets on future link prediction tasks. Models built using the ROLAND framework achieve on average 62.7% relative mean reciprocal rank (MRR) improvement over state-of-the-art baselines under the standard evaluation settings on three datasets. We find state-of-the-art baselines experience out-of-memory errors for larger datasets, while ROLAND can easily scale to dynamic graphs with 56 million edges. After re-implementing these baselines using the ROLAND training strategy, ROLAND models still achieve on average 15.5% relative MRR improvement over the baselines.
#graph#graphneuralnetworks#gnn
Public
Dominic Boutet

#arxivfeed :

"On the Ability of Graph Neural Networks to Model Interactions Between Vertices"
arxiv.org/abs/2211.16494

arXiv.orgOn the Ability of Graph Neural Networks to Model Interactions Between VerticesGraph neural networks (GNNs) are widely used for modeling complex interactions between entities represented as vertices of a graph. Despite recent efforts to theoretically analyze the expressive power of GNNs, a formal characterization of their ability to model interactions is lacking. The current paper aims to address this gap. Formalizing strength of interactions through an established measure known as separation rank, we quantify the ability of certain GNNs to model interaction between a given subset of vertices and its complement, i.e. between the sides of a given partition of input vertices. Our results reveal that the ability to model interaction is primarily determined by the partition's walk index -- a graph-theoretical characteristic defined by the number of walks originating from the boundary of the partition. Experiments with common GNN architectures corroborate this finding. As a practical application of our theory, we design an edge sparsification algorithm named Walk Index Sparsification (WIS), which preserves the ability of a GNN to model interactions when input edges are removed. WIS is simple, computationally efficient, and in our experiments has markedly outperformed alternative methods in terms of induced prediction accuracy. More broadly, it showcases the potential of improving GNNs by theoretically analyzing the interactions they can model.
#MachineLearning#DeepLearning#GraphNeuralNetworks
SearchLive feeds
About