PAPER

LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence

ML SSLRO AutoML Generative Models DM
2025年09月03日
我们认为,要实现通用智能的进步,需要基于语言、物理世界和结构化数据的互补性基础模型。本报告介绍了LimiX,这是我们大型结构化数据模型(LDMs)系列中的首个成果。LimiX将结构化数据视为变量与缺失性(missingness)的联合分布,因此通过基于查询的条件预测,一个模型即可胜任广泛的表格任务。LimiX采用带掩码的联合分布建模进行预训练,其训练目标具有情景化和上下文依赖的特点:模型会根据特定数据集的上下文信息,预测用户指定的查询子集,从而在推理阶段实现快速、无需训练的适应能力。我们在10个大型结构化数据基准任务上对LimiX进行了全面评估,涵盖样本量、特征维度、类别数量、类别型与数值型特征比例、缺失值比例以及样本与特征比等多种复杂场景。实验表明,仅使用一个模型和统一的接口,LimiX在分类、回归、缺失值填补和数据生成等多种任务中始终优于包括梯度提升树、深度表格网络、近期的表格基础模型以及自动化集成方法在内的多种强大基线方法,且优势显著,同时无需针对不同任务设计专门的模型架构或进行单独训练。所有LimiX模型均已按照Apache 2.0协议公开发布,供公众使用。
We argue that progress toward general intelligence requires complementary foundation models grounded in language, the physical world, and structured data. This report presents LimiX, the first installment of our large structured-data models (LDMs). LimiX treats structured data as a joint distribution over variables and missingness, thus capable of addressing a wide range of tabular tasks through query-based conditional prediction via a single model. LimiX is pretrained using masked joint-distribution modeling with an episodic, context-conditional objective, where the model predicts for query subsets conditioned on dataset-specific contexts, supporting rapid, training-free adaptation at inference. We evaluate LimiX across 10 large structured-data benchmarks with broad regimes of sample size, feature dimensionality, class number, categorical-to-numerical feature ratio, missingness, and sample-to-feature ratios. With a single model and a unified interface, LimiX consistently surpasses strong baselines including gradient-boosting trees, deep tabular networks, recent tabular foundation models, and automated ensembles, as shown in Figure 1 and Figure 2. The superiority holds across a wide range of tasks, such as classification, regression, missing value imputation, and data generation, often by substantial margins, while avoiding task-specific architectures or bespoke training per task. All LimiX models are publicly accessible under Apache 2.0.
许愿