LG - 机器学习   CV - 计算机视觉   CL - 计算与语言   AS - 音频与语音 RO - 机器人

转自爱可可爱生活

摘要：关于任务结构的明确知识是人类基于模型行动的主要决定因素、无噪声逆向任意图像变换、支持可微虚拟目标插入的街道场景神经光场估计、生成式信息提取、减少在线持续学习突发表示变化的新见解、基于可变形卷积提升现代和历代手写文本识别、卷积引导视觉Transformer、基于跨模态Transformer的舞蹈风格迁移、面向模糊图像快速鲁棒场景重建的渐进去模糊辐射场

 

1、[LG] Explicit knowledge of task structure is a primary determinant of human model-based action

P Castro-Rodrigues, T Akam, I Snorasson…

[Champalimaud Centre for the Unknown & New York State Psychiatric Institute...]

关于任务结构的明确知识是人类基于模型行动的主要决定因素。通过指令获得的显性信息深刻地塑造了人类的选择行为。然而，这是在计算简单的任务中进行的研究，基于模型的系统和无模型的系统，分别产生目标导向的行动和习惯性的行动，是如何受到没有或存在指令的影响的，目前还不清楚。本文评估了在提供关于任务结构的信息之前和之后，健康志愿者和患有强迫症或其他疾病的人在一个计算上更复杂的决策任务的变体中的行为。最初的行为是无模型的，奖励直接强化了之前的行动。基于模型的控制，采用对每个动作产生的状态的预测，在少数参与者中随着经验的积累而出现，而在那些患有强迫症的人中则较少出现。提供任务结构信息强烈地增加了基于模型的控制，在所有群体中都是如此。因此，在人类中，明确的任务结构知识是基于模型的强化学习的主要决定因素，并且最容易从指导而不是经验中获得。

Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and in individuals suffering from obsessive-compulsive or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of participants, and less in those with obsessive-compulsive disorder. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, in humans, explicit task structural knowledge is a primary determinant of model-based reinforcement learning and is most readily acquired from instruction rather than experience.

https://nature.com/articles/s41562-022-01346-2

 

2、[CV] Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise

A Bansal, E Borgnia, H Chu, J S. Li, H Kazemi, F Huang, M Goldblum, J Geiping, T Goldstein

[University of Maryland & New York University]

Cold Diffusion：无噪声逆向任意图像变换。标准的扩散模型涉及一个图像变换——加入高斯噪声——和一个逆向这种退化的图像恢复算子。本文观察到，扩散模型的生成行为，并不强烈依赖于图像退化的选择，事实上，通过改变这种选择，可以构建整个生成模型族。即使用完全确定的退化(如模糊、掩码等)，作为扩散模型基础的训练和测试时更新规则也可以很容易地泛化以创建生成模型。这些完全确定的模型的成功使人们对社区对扩散模型的理解产生了疑问，这种理解依赖于梯度朗文动力学或变分推理中的噪声，并为逆向任意过程的泛化扩散模型铺平了道路。

Standard diffusion models involve an image transform – adding Gaussian noise – and an image restoration operator that inverts this degradation. We observe that the generative behavior of diffusion models is not strongly dependent on the choice of image degradation, and in fact an entire family of generative models can be constructed by varying this choice. Even when using completely deterministic degradations (e.g., blur, masking, and more), the training and test-time update rules that underlie diffusion models can be easily generalized to create generative models. The success of these fully deterministic models calls into question the community’s understanding of diffusion models, which relies on noise in either gradient Langevin dynamics or variational inference, and paves the way for generalized diffusion models that invert arbitrary processes. Our code is available at github.com/arpitbansal297/Cold-Diffusion-Models.

https://arxiv.org/abs/2208.09392

 

3、[CV] Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion

Z Wang, W Chen, D Acuna, J Kautz, S Fidler

[NVIDIA]

支持可微虚拟目标插入的街道场景神经光场估计。本文考虑了户外照明估计这一具有挑战性的问题，目的是将逼真的虚拟目标插入照片中。现有的户外照明估计工作通常将场景照明简化为环境图，这无法捕捉到户外场景中空间变化的照明效果。本文提出一种神经方法，从单一图像中估计5D HDR光场，并提出一种可微的目标插入公式，使基于图像的损失得到端到端的训练，以鼓励真实性。设计了一种针对室外场景的混合照明表示，其中包含一个处理太阳极端强度的HDR天穹，以及一个模拟周围场景的空间变化外观的体照明表示。有了估计的照明，阴影感知物体插入是完全可微的，这使得在合成图像上进行对抗训练，为照明预测提供额外的监督信号。通过实验证明，所提出的混合照明表示法比现有的户外照明估计方法具有更好的性能。本文进一步展示了所提出的AR物体插入在自主驾驶应用中的好处，在增强数据上进行训练时，获得了3D目标检测器的性能提升。

We consider the challenging problem of outdoor lighting estimation for the goal of photorealistic virtual object insertion into photographs. Existing works on outdoor lighting estimation typically simplify the scene lighting into an environment map which cannot capture the spatially-varying lighting effects in outdoor scenes. In this work, we propose a neural approach that estimates the 5D HDR light field from a single image, and a differentiable object insertion formulation that enables end-to-end training with image-based losses that encourage realism. Specifically, we design a hybrid lighting representation tailored to outdoor scenes, which contains an HDR sky dome that handles the extreme intensity of the sun, and a volumetric lighting representation that models the spatially-varying appearance of the surrounding scene. With the estimated lighting, our shadow-aware object insertion is fully differentiable, which enables adversarial training over the composited image to provide additional supervisory signal to the lighting prediction. We experimentally demonstrate that our hybrid lighting representation is more performant than existing outdoor lighting estimation methods. We further show the benefits of our AR object insertion in an autonomous driving application, where we obtain performance gains for a 3D object detector when trained on our augmented data.

https://arxiv.org/abs/2208.09480

 

4、[CL] GenIE: Generative Information Extraction

M Josifoski, N D Cao, M Peyrard, F Petroni, R West

[Ecole Polytechnique Fédérale de Lausanne & University of Amsterdam & Meta AI]

GenIE：生成式信息提取。文本的结构化和基础化表示，通常是通过封闭式信息提取来实现的，即从知识库模式中提取一套详尽的(主体、关系、客体)三联体，与预先定义的实体和关系集相一致。大多数现有的工作是容易积累错误的管道，而且所有的方法都只适用于不切实际的少量实体和关系。本文提出GenIE(生成式信息提取)，是第一个端到端的封闭式信息提取的自回归方案。GenIE通过自回归地生成文本形式的关系和实体，自然而然地利用了来自预训练的Transformer的语言知识。由于采用了新的双级约束生成策略，只生成与预定义知识库模式一致的三联体。实验表明，GenIE在封闭式信息提取方面是最先进的，比基线的训练数据点更少，可扩展到之前无法管理的实体和关系数量。通过这项工作，封闭式信息提取在现实场景中变得实用，为下游任务提供了新的机会。本文工作为实现信息提取核心任务的统一的端到端方法铺平了道路。

Structured and grounded representation of text is typically formalized by closed information extraction, the problem of extracting an exhaustive set of (subject, relation, object) triplets that are consistent with a predefined set of entities and relations from a knowledge base schema. Most existing works are pipelines prone to error accumulation, and all approaches are only applicable to unrealistically small numbers of entities and relations. We introduce GenIE (generative information extraction), the first end-to-end autoregressive formulation of closed information extraction. GenIE naturally exploits the language knowledge from the pre-trained transformer by autoregressively generating relations and entities in textual form. Thanks to a new bi-level constrained generation strategy, only triplets consistent with the predefined knowledge base schema are produced. Our experiments show that GenIE is state-of-the-art on closed information extraction, generalizes from fewer training data points than baselines, and scales to a previously unmanageable number of entities and relations. With this work, closed information extraction becomes practical in realistic scenarios, providing new opportunities for downstream tasks. Finally, this work paves the way towards a unified end-to-end approach to the core tasks of information extraction.

https://arxiv.org/abs/2112.08340

 

5、[LG] New Insights on Reducing Abrupt Representation Change in Online Continual Learning

L Caccia, R Aljundi, N Asadi, T Tuytelaars...

[McGill University & Toyota Motor Europe & Concordia University & KU Leuven]

减少在线持续学习突发表示变化的新见解。在在线持续学习范式中，智能体必须从不断变化的分布中学习，同时遵守内存和计算限制。经验重放(ER)，即存储过去数据的一小部分并与新数据一起重放，已经成为一种简单而有效的学习策略。本文关注的是，当之前未观察到的类别出现在传入的数据流中，而新的类别必须与之前的类别区分开来时，观察到的数据的表示会发生变化。本文对该问题有了新的认识，表明应用ER会使新增加的类的表示与之前的类明显重叠，从而导致高度破坏性的参数更新。基于这一经验分析，本文提出一种新方法，通过保护学习到的表示不被急剧调整以适应新类别来缓解这一问题。用非对称的更新规则，可以推动新类别适应旧类别(而不是相反)，特别是在任务边界更有效，因为大部分的遗忘通常发生在那里。经验结果显示，在标准的持续学习基准上，比强基线有明显的改善。

In the online continual learning paradigm, agents must learn from a changing distribution while respecting memory and compute constraints. Experience Replay (ER), where a small subset of past data is stored and replayed alongside new data, has emerged as a simple and effective learning strategy. In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones. We shed new light on this question by showing that applying ER causes the newly added classes’ representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates. Based on this empirical analysis, we propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes. We show that using an asymmetric update rule pushes new classes to adapt to the older ones (rather than the reverse), which is more effective especially at task boundaries, where much of the forgetting typically occurs. Empirical results show significant gains over strong baselines on standard continual learning benchmarks 1.

https://arxiv.org/abs/2203.03798

 

另外几篇值得关注的论文：

 

[CV] Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions

基于可变形卷积提升现代和历代手写文本识别

S Cascianelli, M Cornia, L Baraldi, R Cucchiara

[University of Modena and Reggio Emilia]

https://arxiv.org/abs/2208.08109

 

[CV] Conviformers: Convolutionally guided Vision Transformer

Conviformers：卷积引导视觉Transformer

M Vaishnav, T Fel, I F Rodrıguez, T Serre

[Universite de Toulouse & Brown University]

https://arxiv.org/abs/2208.08900

 

[LG] Dance Style Transfer with Cross-modal Transformer

基于跨模态Transformer的舞蹈风格迁移
W Yin, H Yin, K Baraka, D Kragic, M Björkman

[KTH Royal Institute of Technology & Vrije Universiteit Amsterdam] https://arxiv.org/abs/2208.09406

 

[CV] PDRF: Progressively Deblurring Radiance Field for Fast and Robust Scene Reconstruction from Blurry Images

PDRF：面向模糊图像快速鲁棒场景重建的渐进去模糊辐射场

C Peng, R Chellappa

[Johns Hopkins University]
https://arxiv.org/abs/2208.08049

 

 

 

内容中包含的图片若涉及版权问题，请及时与我们联系删除