来自今天的爱可可AI前沿推介
LG] ClimaX: A foundation model for weather and climate
T Nguyen, J Brandstetter, A Kapoor, J K. Gupta, A Grover
[Microsoft]
ClimaX: 天气和气候的基础模型
要点:
-
ClimaX是一个用于天气和气候科学的深度学习模型，具有灵活性和通用性，可以用跨越不同变量、时空覆盖和物理基础的异质数据集进行训练； -
ClimaX用新的编码块和聚合块扩展了 Transformer 架构，允许有效利用可用计算，同时保持通用性； -
ClimaX在来自 CMIP6 的气候数据集上用自监督学习目标进行预训练，可微调以解决广泛的气候和天气任务，包括那些涉及大气变量和预训练中未见的时空尺度。
一句话总结:
ClimaX 是一个深度学习模型，可以在异构数据集上进行训练，具有灵活性和通用性，可针对各种天气和气候任务进行微调，使其成为天气和气候建模的基础模型。
摘要：
大多数最先进的天气和气候建模方法，是基于大气的物理信息数值模型。这些方法旨在模拟非线性动态和多变量间复杂相互作用，这对近似计算具有挑战性。此外，许多这样的数值模型是计算密集型的，特别是在以细粒度的空间和时间分辨率对大气现象进行建模时。最近基于机器学习的数据驱动的方法，旨在通过用深度神经网络学习数据驱动的函数映射，来直接解决下游的预测或预报任务。然而，这些网络是用策划和同质的气候数据集来训练特定的时空任务，因此缺乏数值模型的通用性。本文开发并展示了ClimaX，一个用于天气和气候科学的灵活和通用的深度学习模型，可以用跨越不同变量、时空覆盖和物理基础的异质数据集来训练。ClimaX用新的编码块和聚合块扩展了 Transformer 架构，允许有效利用可用的计算，同时保持通用性。ClimaX 在来自 CMIP6 的气候数据集上用自监督学习目标进行预训练。预训练的ClimaX可以微调以解决广泛的气候和天气任务，包括那些涉及大气变量和预训练中未见的时空尺度的任务。与现有的数据驱动基线相比，ClimaX的这种通用性使其在天气预报和气候预测的基准上有更高的性能，即使在较低的分辨率和计算预算下进行预训练。
Most state-of-the-art approaches for weather and climate modeling are based on physics-informed numerical models of the atmosphere. These approaches aim to model the non-linear dynamics and complex interactions between multiple variables, which are challenging to approximate. Additionally, many such numerical models are computationally intensive, especially when modeling the atmospheric phenomenon at a fine-grained spatial and temporal resolution. Recent data-driven approaches based on machine learning instead aim to directly solve a downstream forecasting or projection task by learning a data-driven functional mapping using deep neural networks. However, these networks are trained using curated and homogeneous climate datasets for specific spatiotemporal tasks, and thus lack the generality of numerical models. We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings. ClimaX extends the Transformer architecture with novel encoding and aggregation blocks that allow effective use of available compute while maintaining general utility. ClimaX is pre-trained with a self-supervised learning objective on climate datasets derived from CMIP6. The pre-trained ClimaX can then be fine-tuned to address a breadth of climate and weather tasks, including those that involve atmospheric variables and spatio-temporal scales unseen during pretraining. Compared to existing data-driven baselines, we show that this generality in ClimaX results in superior performance on benchmarks for weather forecasting and climate projections, even when pretrained at lower resolutions and compute budgets.
论文链接：https://arxiv.org/abs/2301.10343
内容中包含的图片若涉及版权问题，请及时与我们联系删除
评论
沙发等你来抢