图神经网络自监督学习
Graph Neural Networks: Self-supervised Learning
地址：https://tylersnetwork.github.io/papers/ssl_for_gnns.pdf
导读：自监督学习 (SSL)试图在未标记的数据上创建和利用特定的任务来辅助机器学习任务。虽然最初应用于图像和文本领域，最近越来越的研究在图机器学习上利用 SSL提高图神经网络 (GNN) 的性能。对于
在本文中，作者总结了最近的发展，将 SSL 应用于 GNN，通过不同的训练策略对它们进行分类，总结了用于构建自监督学习任务的策略和数据类型，最后讨论未来方向的开放挑战。
ICLR2022图学习领域一览
ICLR 2022投稿已经截止，根据Open Review网址收录的论文结果，来看看今年的ICLR关于图学习领域的研究热点。
ICLR 2022网址：https://openreview.net/group?id=ICLR.cc/2022/Conference
1.Scalability
- Sampling Before Training: Rethinking the Effect of Edges in the Process of Training Graph Neural Networks
- SpSC: A Fast and Provable Algorithm for Sampling-Based GNN Training
- Revisiting Layer-wise Sampling in Fast Training for Graph Convolutional Networks
- Large-Scale Representation Learning on Graphs via Bootstrapping
- EXACT: Scalable Graph Neural Networks Training via Extreme Activation Compression
- PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication
- Increase and Conquer: Training Graph Neural Networks on Growing Graphs
- LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs
- Embedding Compression with Hashing for Efficient Representation Learning in Graph
- Locality-Based Mini Batching for Graph Neural Networks
- Coarformer: Transformer for large graph via graph coarsening
- Inductive Lottery Ticket Learning for Graph Neural Networks
- IGLU: Efficient GCN Training via Lazy Updates
- Graph Attention Multi-layer Perceptron
- Learn Locally, Correct Globally: A Distributed Algorithm for Training Graph Neural Networks
- Full-Precision Free Binary Graph Neural Networks
- Graph-less Neural Networks: Teaching Old MLPs New Tricks Via Distillation
- D2-GCN: Data-Dependent GCNs for Boosting Both Efficiency and Scalability
- Adaptive Filters for Low-Latency and Memory-Efficient Graph Neural Networks
2.Oversmoothing/Depth
- DeeperGCN: All You Need to Train Deeper GCNs
- Understanding Graph Learning with Local Intrinsic Dimensionality
- RankedDrop: Enhancing Deep Graph Convolutional Networks Training
- DEEP GRAPH TREE NETWORKS
- Evaluating Deep Graph Neural Networks
- How Frequency Effect Graph Neural Networks
- Revisiting Over-smoothing in BERT from the Perspective of Graph
- Towards Feature Overcorrelation in Deeper Graph Neural Networks
- Tackling Oversmoothing of GNNs with Contrastive Learning
- Implicit vs Unfolded Graph Neural Networks
- Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective
3.Explainability
- Explainability in Graph Convolutional Network for Representation Learning
- FlowX: Towards Explainable Graph Neural Networks via Message Flows
- On Theoretically and Empirically Analyzing GNN Explanation Methods
- Task-Agnostic Graph Neural Explanations
- Deconfounding to Explanation Evaluation in Graph Neural Networks
- DEGREE: Decomposition Based Explanation for Graph Neural Networks
- Discovering Invariant Rationales for Graph Neural Networks
- Interpreting Graph Neural Networks via Unrevealed Causal Learning
- Explainable GNN-Based Models over Knowledge Graphs
4.Self-Supervision
- ESCo: Towards Provably Effective and Scalable Contrastive Representation Learning
- m-mix: Generating hard negatives via multiple samples mixing for contrastive learning
- Rethinking Temperature in Graph Contrastive Learning
- Graph Barlow Twins: A self-supervised representation learning framework for graphs
- Automated Self-Supervised Learning for Graphs
- Self-Supervised Representation Learning via Latent Graph Prediction
- Interrogating Paradigms in Self-supervised Graph Representation Learning
- Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis
- Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction
- SS-MAIL: Self-Supervised Multi-Agent Imitation Learning
- Learning Graph Augmentations to Learn Graph Representations
- Robust Graph Data Learning with Latent Graph Convolutional Representation
5. Adversarial Attacks / Robustness
- GARNET: A Spectral Approach to Robust and Scalable Graph Neural Networks
- Defending Graph Neural Networks via Tensor-Based Robust Graph Aggregation
- A General Unified Graph Neural Network Framework Against Adversarial Attacks
- Beyond Message Passing Paradigm: Training Graph Data with Consistency Constraints
- Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks
- Bandits for Black-box Attacks to Graph Neural Networks with Structure Perturbation
- Understanding and Improving Graph Injection Attack by Promoting Unnoticeability
6.Heterophily
- On the Relationship between Heterophily and Robustness of Graph Neural Networks
- Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
- An Interpretable Graph Generative Model with Heterophily
- Is Heterophily A Real Nightmare For Graph Neural Networks on Performing Node Classification?
- Is Homophily a Necessity for Graph Neural Networks?
- Graph Information Matters: Understanding Graph Filters from Interaction Probability
- GCN-SL: Graph Convolutional Network with Structure Learning for Disassortative Graphs
7. Heterogeneous Graphs
- Equivariant Heterogeneous Graph Networks
- Molecular Graph Representation Learning via Heterogeneous Motif Graph Construction
- R-GSN: The Relation-based Graph Similar Network for Heterogeneous Graph
8. Multi-Relational Graphs
- NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs
- Neural Methods for Logical Reasoning over Knowledge Graphs
- Time-aware Relational Graph Attention Network for Temporal Knowledge Graph Embeddings
- A Topological View of Rule Learning in Knowledge Graphs
- CareGraph: A Graph-based Recommender System for Diabetes Self-Care
- Relational Multi-Task Learning: Modeling Relations between Data and Tasks
- Inductive Relation Prediction Using Analogy Subgraph Embeddings
9. Hyper-relational Knowledge Graphs
- Message Function Search for Hyper-relational Knowledge Graph
- Query Embedding on Hyper-Relational Knowledge Graphs
10. Hypergraphs
- You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks
- Hypergraph Convolutional Networks via Equivalency between Hypergraphs and Undirected Graphs
- On the Expressiveness and Learning of Relational Neural Networks on Hypergraphs
- A molecular hypergraph convolutional network with functional group information
- Efficient Training and Inference of Hypergraph Reasoning Networks
- FEATURE-AUGMENTED HYPERGRAPH NEURAL NETWORKS
- GENERALIZING LINK PREDICTION FOR HYPERGRAPHS
11. Link Prediction
- Revisiting Virtual Nodes in Graph Neural Networks for Link Prediction
- Counterfactual Graph Learning for Link Prediction
- Benchmarking Graph Neural Networks on Dynamic Link Prediction
- Few-shot graph link prediction with domain adaptation
- Neural Link Prediction with Walk Pooling
12. Graph Classification
- GiG: Graph in Graph, a model boosting graph classification and representation learning
- G-Mixup: Graph Augmentation for Graph Classification
- Structural Optimization Makes Graph Classification Simpler and Better
- Intrusion-Free Graph Mixup
- The Infinite Contextual Graph Markov Model
- Geometric Random Walk Graph Neural Networks via Implicit Layers
- Wasserstein Weisfeiler-Lehman Subtree Distance for Graph-Structured Data
- Adaptive Graph Capsule Convolutional Networks
13. Expressivity
- Expressiveness and Approximation Properties of Graph Neural Networks
- Counting Substructures with Higher-Order Graph Neural Networks: Possibility and Impossibility Results
- A New Perspective on "How Graph Neural Networks Go Beyond Weisfeiler-Lehman?"
- Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks
- On Locality in Graph Learning via Graph Neural Network
- On the Effect of Input Perturbations for Graph Neural Networks
- Local Permutation Equivariance For Graph Neural Networks
- How Attentive are Graph Attention Networks?
14. Subgraphs
- Equivariant Subgraph Aggregation Networks
- Count-GNN: Graph Neural Networks for Subgraph Isomorphism Counting
- From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness
- NeuroSED: Learning Subgraph Similarity via Graph Neural Networks
- Learning Representations of Partial Subgraphs by Subgraph InfoMax
- GLASS: GNN with Labeling Tricks for Subgraph Representation Learning
15. Equivariance
- Frame Averaging for Invariant and Equivariant Network Design
- Geometric and Physical Quantities improve E(3) Equivariant Message Passing
- Symmetry-driven graph neural networks
16. Generalisability
- Towards Distribution Shift of Node-Level Prediction on Graphs: An Invariance Perspective
- A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs
- Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning
17. Graph Generative Models: Evaluation Metrics
- On Evaluation Metrics for Graph Generative Models
- Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions
18. Proteins
- Fast fixed-backbone protein sequence and rotamer design
- Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking
- De novo design of protein target specific scaffold-based Inhibitors via Reinforcement Learning
- An Effective GCN-based Hierarchical Multi-label classification for Protein Function Prediction
- Geometric Transformers for Protein Interface Contact Prediction
- Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design
- Granger causal inference on DAGs identifies genomic loci regulating transcription
19. Molecules
- Spanning Tree-based Graph Generation for Molecules
- Molecular Graph Generation via Geometric Scattering
- Generating Realistic 3D Molecules with an Equivariant Conditional Likelihood Model
- Chemical-Reaction-Aware Molecule Representation Learning
- Relative Molecule Self-Attention Transformer
- Differentiable Scaffolding Tree for Molecule Optimization
- MS2-Transformer: An End-to-End Model for MS/MS-assisted Molecule Identification
- Graph Piece: Efficiently Generating High-Quality Molecular Graphs with Substructures
- Pre-training Molecular Graph Representation with 3D Geometry
- Spherical Message Passing for 3D Molecular Graphs
- GraphEBM: Towards Permutation Invariant and Multi-Objective Molecular Graph Generation
- Data-Efficient Graph Grammar Learning for Molecular Generation
- Learning to Extend Molecular Scaffolds with Structural Motifs
- Stepping Back to SMILES Transformers for Fast Molecular Representation Inference
- An Autoregressive Flow Model for 3D Molecular Geometry Generation from Scratch
- Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations
- GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation
- 3D-Transformer: Molecular Representation with Transformer in 3D Space
- Crystal Diffusion Variational Autoencoder for Periodic Material Generation
- Knowledge Guided Geometric Editing for Unsupervised Drug Design
- Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery
- MoReL: Multi-omics Relational Learning
20. Molecular Property Prediction
- 3D Pre-training improves GNNs for Molecular Property Prediction
- Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond
21. Retrosynthesis
- SemiRetro: Semi-template framework boosts deep retrosynthesis prediction
- Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction
22. Time Series
- Evaluating the Robustness of Time Series Anomaly and Intrusion Detection Methods against Adversarial Attacks
- Spatiotemporal Representation Learning on Time Series with Dynamic Graph ODEs
- Multivariate Time Series Forecasting with Latent Graph Inference
- TAMP-S2GCNets: Coupling Time-Aware Multipersistence Knowledge Representation with Spatio-Supra Graph Convolutional Networks for Time-Series Forecasting
- Neural graphical modelling in continuous-time: consistency guarantees and algorithms
- Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks
- Graph-Guided Network for Irregularly Sampled Multivariate Time Series
- GAETS: A Graph Autoencoder Time Series Approach Towards Battery Parameter Estimation
- Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series
- Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future
- Causal discovery from conditionally stationary time-series
23. PDE
- Message Passing Neural PDE Solvers
- Learning Time-dependent PDE Solver using Message Passing Graph Neural Networks
- Learning to Solve PDE-constrained Inverse Problems with Graph Networks
24. Physics
- SiT: Simulation Transformer for Particle-based Physics Simulation
- Predicting Physics in Mesh-reduced Space with Temporal Attention
- KINet: Keypoint Interaction Networks for Unsupervised Forward Modeling
- Constraint-based graph network simulator
- Boundary Graph Neural Networks for 3D Simulations
- Constrained Graph Mechanics Networks
- Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions
- Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks
25. Dynamic / Temporal Graphs
- Dynamic Graph Representation Learning via Graph Transformer Networks
- Metric Learning on Temporal Graphs via Few-Shot Examples
- Online graph nets
- Space-Time Graph Neural Networks
- Convolutional Neural Network Dynamics: A Graph Perspective
26. Traffic
- Simpler can be better: Multi-level Abstraction with Graph Convolution Recurrent Neural Network cells for Traffic Prediction
- Learning to Remember Patterns: Pattern Matching Memory Networks for Traffic Forecasting
- A multi-domain splitting framework for time-varying graph structure
27. Combinatorial Optimisation
- Learning to Solve Combinatorial Problems via Efficient Exploration
- Graph Neural Network Guided Local Search for the Traveling Salesperson Problem
- Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient
- Learning to Solve an Order Fulfillment Problem in Milliseconds with Edge-Feature-Embedded Graph Attention
- Neural Models for Output-Space Invariance in Combinatorial Problems
- What’s Wrong with Deep Learning in Tree Search for Combinatorial Optimization
- Mind Your Solver! On Adversarial Attack and Defense for Combinatorial Optimization
28. Natural Language Processing
- Crossformer: Transformer with Alternated Cross-Layer Guidance
- Constituency Tree Representation for Argument Unit Recognition
- Learning Object-Oriented Dynamics for Planning from Text
- Fact-driven Logical Reasoning- Fact-driven Logical Reasoning
29. Language Modelling
- GNN-LM: Language Modeling based on Global Contexts via GNN
- Understanding Knowledge Integration in Language Models with Graph Convolutions
- GreaseLM: Graph REASoning Enhanced Language Models
30. Question Answering
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
- GNN is a Counter? Revisiting GNN for Question Answering
31. Computer Vision
- Graph Similarities and Dual Approach for Sequential Text-to-Image Retrieval
- Breaking Down Questions for Outside-Knowledge VQA
- Towards Generic Interface for Human-Neural Network Knowledge Exchange
- Revisiting Skeleton-based Action Recognition
- Unified Recurrence Modeling for Video Action Anticipation
- From Graph Local Embedding to Deep Metric Learning
- Ada-NETS: Face Clustering via Adaptive Neighbour Discovery in the Structure Space
- SSR-GNNs: Stroke-based Sketch Representation with Graph Neural Networks
- Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation
32. Point Clouds
- Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework
- Concentric Spherical GNN for 3D Representation Learning
33. Fairness
- Fair Node Representation Learning via Adaptive Data Augmentation
- Generalized Demographic Parity for Group Fairness
34. Privacy/Federated Learning
- Node-Level Differentially Private Graph Neural Networks
- Federated Learning with Heterogeneous Architectures using Graph HyperNetworks
- Federated Inference through Aligning Local Representations and Learning a Consensus Graph
35. Programming
- GRAPHIX: A Pre-trained Graph Edit Model for Automated Program Repair
- Exploring General Intelligence of Program Analysis for Multiple Tasks
- ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection
36. Multi-Agent Learning
- Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
- LPMARL: Linear Programming based Implicit Task Assigment for Hiearchical Multi-Agent Reinforcement Learning
- ScheduleNet: Learn to solve multi-agent scheduling problems with reinforcement learning
- Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agents
- A Multi-Agent Koopman Operator Approach for Distributed Representation Learning of Networked Dynamical Systems
- Multi-Agent Decentralized Belief Propagation on Graphs
- Context-Aware Sparse Deep Coordination Graphs
- Learning to Solve Multi-Robot Task Allocation with a Covariant-Attention based Neural Architecture
37. Stochastic Block Models
- LEARNING GUARANTEES FOR GRAPH CONVOLUTIONAL NETWORKS ON THE STOCHASTIC BLOCK MODEL
- DIGRAC: Digraph Clustering Based on Flow Imbalance
38. Architecture Search
- AutoCoG: A Unified Data-Modal Co-Search Framework for Graph Neural Networks
- A Transferable General-Purpose Predictor for Neural Architecture Search
39. Healthcare
- Using Graph Representation Learning with Schema Encoders to Measure the Severity of Depressive Symptoms
- Deep Representations for Time-varying Brain Datasets
- A Biologically Interpretable Graph Convolutional Network to Link Genetic Risk Pathways and Imaging Phenotypes of Disease
40. Miscellaneous
- Neural Structured Prediction for Inductive Node Classification
- Embedding models through the lens of Stable Coloring
- GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification
- PI-GNN: Towards Robust Semi-Supervised Node Classification against Noisy Labels
- PERSONALIZED LAB TEST RESPONSE PREDICTION WITH KNOWLEDGE AUGMENTATION
- Learning Scenario Representation for Solving Two-stage Stochastic Integer Programs
- WeaveNet: A Differentiable Solver for Non-linear Assignment Problems
- Factored World Models for Zero-Shot Generalization in Robotic Manipulation
- On Exploring Node-feature and Graph-structure Diversities for Node Drop Graph Pooling
- Learning Graph Structure from Convolutional Mixtures
- Large-Scale Adversarial Attacks on Graph Neural Networks via Graph Coarsening
- Graph Condensation for Graph Neural Networks
- Graph Kernel Neural Networks
- Multiresolution Equivariant Graph Variational Autoencoder
- Backpropagation-free Graph Convolutional Networks
- Graph Neural Networks with Learnable Structural and Positional Representations
- NAFS: A Simple yet Tough-to-Beat Baseline for Graph Representation Learning
- SpecTRA: Spectral Transformer for Graph Representation Learning
- A Deep Latent Space Model for Directed Graph Representation Learning
- Convergent Graph Solvers
- Effective Polynomial Filter Adaptation for Graph Neural Networks
- G3: Representation Learning and Generation for Geometric Graphs
- On the Unreasonable Effectiveness of Feature Propagation in Learning on Graphs with Missing Node Features
- PF-GNN: Differentiable particle filtering based approximation of universal graph representations
- Learning Graph Representations for Influence Maximization
- Connecting Graph Convolution and Graph PCA
- p-Laplacian Based Graph Neural Networks
- PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs
- Topological Graph Neural Networks
- Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations
- Top-N: Equivariant Set and Graph Generation without Exchangeability
- An Analysis of Attentive Walk-Aggregating Graph Neural Networks
- Input Convex Graph Neural Networks: An Application to Optimal Control and Design Optimization
- Spiking Graph Convolutional Networks
- Trading Quality for Efficiency of Graph Partitioning: An Inductive Method across Graphs
- Graph Auto-Encoder via Neighborhood Wasserstein Reconstruction
- Personalized PageRank meets Graph Attention Networks
- Understanding over-squashing and bottlenecks on graphs via curvature
- Weakly Supervised Graph Clustering
- Graph Tree Neural Networks
- Open Set Domain Adaptation with Zero-shot Learning on Graph
- Graph Convolutional Memory using Topological Priors
- Learning to Pool in Graph Neural Networks for Extrapolation
- Edge Partition Modulated Graph Convolutional Networks
- Local Augmentation for Graph Neural Networks
- GRAND++: Graph Neural Diffusion with A Source Term
- Learning to Schedule Learning rate with Graph Neural Networks
- Genome Sequence Reconstruction Using Gated Graph Convolutional Network
- Graph Convolutional Networks via Adaptive Filter Banks
- SONG: Self-Organizing Neural Graphs
- Information Gain Propagation: a New Way to Graph Active Learning with Soft Labels
- Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
- GIR Framework: Learning Graph Positional Embeddings with Anchor Indication and Path Encoding
- Why Propagate Alone? Parallel Use of Labels and Features on Graphs
- Stabilized Self-training with Negative Sampling on Few-labeled Graph Data
- Accelerating Optimization using Neural Reparametrization
- Convergence of Generalized Belief Propagation Algorithm on Graphs with Motifs
- Convolutional Networks on Enhanced Message-Passing Graph Improve Semi-Supervised Classification with Few Labels
- Efficient Ensembles of Graph Neural Networks
- Learning to Infer the Structure of Network Games
- Convergent Boosted Smoothing for Modeling GraphData with Tabular Node Features
- Neural Relational Inference with Node-Specific Information
- Triangle and Four Cycle Counting with Predictions in Graph Streams
- Neurally boosted supervised spectral clustering
- Structure-Aware Transformer Policy for Inhomogeneous Multi-Task Reinforcement Learning
- Reasoning-Modulated Representations
- Know Your Action Set: Learning Action Relations for Reinforcement Learning
- Second-Order Unsupervised Feature Selection via Knowledge Contrastive Distillation
内容中包含的图片若涉及版权问题，请及时与我们联系删除
评论
沙发等你来抢