Program-as-Weights: A Programming Paradigm for Fuzzy Functions

AK's Picks LLM LoRA/Adapter MQIA
许多日常编程任务难以通过简洁、基于规则的方式实现,例如:针对重要日志行触发告警、修复格式错误的 JSON 数据,或依据用户意图对搜索结果进行排序。目前,这类任务正越来越多地交由大型语言模型(LLM)API 来完成,但由此也付出了牺牲本地化执行能力、结果不可复现性以及高昂调用成本的代价。为此,我们提出“模糊函数式编程”(fuzzy-function programming):即根据自然语言描述的规格说明,将此类函数编译为一个紧凑、可本地执行的神经网络构件。我们以“程序即权重”(Program-as-Weights, PAW)作为该范式的具体实现——其核心是一个在 FuzzyBench 数据集(我们开源的包含 1000 万个样本的大规模基准数据集)上训练得到的 40 亿参数编译器;该编译器能为一个冻结不动、轻量级的解释器生成参数高效的适配器(adapter)。实验表明,仅含 6 亿参数的 Qwen3 解释器在执行 PAW 编译生成的程序时,性能可媲美直接对 320 亿参数的 Qwen3-32B 模型进行提示工程(prompting)的效果;同时,其推理内存占用仅为后者的约二十分之一(注:原文“one fiftieth”即 1/50),且在配备 M3 芯片的 MacBook 上可达每秒 30 个 token 的推理速度。PAW 从根本上重塑了基础模型的角色:它不再作为每次输入都需调用的问题求解器,而是转变为一种“工具构建器”——仅需在定义函数时调用一次,即可产出一个体积小、可复用的神经构件;此后,每次调用该函数时,均可低成本、离线地执行该构件,无需联网或依赖外部服务。
Many everyday programming tasks resist clean rule-based implementation, such as alerting on important log lines, repairing malformed JSON, or ranking search results by intent, and are increasingly outsourced to large language model APIs at the cost of locality, reproducibility, and price. We propose fuzzy-function programming: compiling such a function from a natural-language specification into a compact, locally-executable neural artifact. We instantiate this paradigm with Program-as-Weights (PAW), in which a 4B compiler trained on FuzzyBench, a 10M-example dataset we release, emits parameter-efficient adapters for a frozen, lightweight interpreter. A 0.6B Qwen3 interpreter executing PAW programs matches the performance of direct prompting of Qwen3-32B, while using roughly one fiftieth of the inference memory and running at 30 tokens/s on a MacBook M3. PAW reframes the foundation model from a per-input problem solver into a tool builder: invoked once per function definition, it produces a small reusable artifact whose subsequent calls per function application are cheap and offline.
许愿