Statistical Learning Theory in Lean 4: Empirical Processes from Scratch

AK's Picks ML NNGTB Other ML
我们提出了首个基于经验过程理论的统计学习理论(SLT)在 Lean 4 中的全面形式化工作。本项目的端到端形式化基础设施填补了当前最新版 Lean 4 Mathlib 数学库中的若干空白,完整构建了高斯—利普希茨集中性理论,首次对达德利(Dudley)关于次高斯过程的熵积分定理进行了形式化,并将其应用于最小二乘(稀疏)回归问题,得到了一个紧致的收敛速率界。该项目采用人机协同的工作流程:人类研究者负责设计证明策略,人工智能代理则具体执行战术层面的证明构造,最终产出经人类严格验证的、面向统计学习理论的 Lean 4 工具箱。除实现本身外,形式化过程还揭示并修正了标准统计学习理论教科书中隐含的假设与缺失的技术细节,从而推动研究者对整套理论达成细致入微、逐行可验的理解。本工作不仅确立了一个可复用的形式化基础,也为未来机器学习理论的形式化研究铺平了道路。相关代码已开源,地址为:https://github.com/YuanheZ/lean-stat-learning-theory
We present the first comprehensive Lean 4 formalization of statistical learning theory (SLT) grounded in empirical process theory. Our end-to-end formal infrastructure implement the missing contents in latest Lean 4 Mathlib library, including a complete development of Gaussian Lipschitz concentration, the first formalization of Dudley's entropy integral theorem for sub-Gaussian processes, and an application to least-squares (sparse) regression with a sharp rate. The project was carried out using a human-AI collaborative workflow, in which humans design proof strategies and AI agents execute tactical proof construction, leading to the human-verified Lean 4 toolbox for SLT. Beyond implementation, the formalization process exposes and resolves implicit assumptions and missing details in standard SLT textbooks, enforcing a granular, line-by-line understanding of the theory. This work establishes a reusable formal foundation and opens the door for future developments in machine learning theory. The code is available at https://github.com/YuanheZ/lean-stat-learning-theory
许愿