Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules

K Irie, J Schmidhuber
[IDSIA]

图像作为权重矩阵: 基于突触学习规则的序列图像生成

要点:

  1. 将快速权重编程器(FWP)应用于图像生成，在六个不同的图像域可视化了迭代 FFP 过程；
  2. 由此产生的 FPA 通过执行基于自发明学习模式的 NN 控制学习规则序列来学习生成看起来像自然图像的权重矩阵；
  3. 本文研究的方法并不是生成图像的最佳方法，但其展示了 NN 如何以目标导向的方式学习控制突触权重学习规则序列，以生成复杂而有意义的权重模式；
  4. 在 FPA 输出端有一个额外的卷积 U-Net，可以对 FPA 生成的图像进行一步式的"去噪 "学习，以提高其质量。

一句话总结:
神经网络可以使用通用的突触学习规则和外积生成自然图像，从而实现复杂连接模式的人可理解的可视化。

Work on fast weight programmers has demonstrated the effectiveness of key/value outer product-based learning rules for sequentially generating a weight matrix (WM) of a neural net (NN) by another NN or itself. However, the weight generation steps are typically not visually interpretable by humans, because the contents stored in the WM of an NN are not. Here we apply the same principle to generate natural images. The resulting fast weight painters (FPAs) learn to execute sequences of delta learning rules to sequentially generate images as sums of outer products of self-invented keys and values, one rank at a time, as if each image was a WM of an NN. We train our FPAs in the generative adversarial networks framework, and evaluate on various image datasets. We show how these generic learning rules can generate images with respectable visual quality without any explicit inductive bias for images. While the performance largely lags behind the one of specialized state-of-the-art image generators, our approach allows for visualising how synaptic learning rules iteratively produce complex connection patterns, yielding human-interpretable meaningful images. Finally, we also show that an additional convolutional U-Net (now popular in diffusion models) at the output of an FPA can learn one-step ''denoising'' of FPA-generated images to enhance their quality.

https://openreview.net/forum?id=ddad0PNUvV
图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除