Some Simple Economics of AGI

Agent Human Agent Interaction AI Safety / AI Ethics EVA Other AI Safety
数千年来,人类认知一直是地球进步的主要驱动力。随着人工智能将认知能力从生物载体中解耦,可度量任务执行的边际成本趋近于零,从而吸纳一切可通过指标衡量的劳动——包括创造性、分析性与创新性工作。此时,制约经济增长的瓶颈已不再是智能本身,而是人类的验证带宽(verification bandwidth):即在执行能力已然充沛的前提下,人类对结果进行验证、审计并承担相应责任的能力。我们将通用人工智能(AGI)转型建模为两条竞速下行的成本曲线之间的“碰撞”:一条是呈指数衰减的“自动化成本”曲线,另一条则是受生物学限制而难以提升的“验证成本”曲线。这种结构性不对称,在智能体实际可执行的任务与人类实际负担得起验证的任务之间,拉开了日益扩大的“可度量性鸿沟”(Measurability Gap)。它同时也推动技术变革的方向,从过去偏向技能(skill-biased)转向如今更偏向可度量性(measurability-biased)。经济租金正持续向三类稀缺资源转移:具备验证等级的真实基准(verification-grade ground truth)、具备密码学可追溯性的过程证据(cryptographic provenance),以及风险承保能力(liability underwriting)——即为结果提供保险保障的能力,而不仅仅是生成结果本身。当前以“人在回路中”(human-in-the-loop)为特征的均衡状态本质上并不稳定:一方面,它自下而上被侵蚀——因学徒制瓦解,“初级人才闭环”(Missing Junior Loop)不复存在;另一方面,它自内而外地瓦解——专家们将自身知识编码固化,却无意间加速了自身的过时(“编码者诅咒”,Codifier’s Curse)。在此背景下,未经充分验证的系统部署,对个体或企业而言反而成为一种私人性的理性选择,却由此产生“特洛伊木马式”的外部性。若放任这些力量不受约束地发展,整个经济体系将滑向“空心化经济”(Hollow Economy)。然而,倘若我们同步规模化提升验证能力与智能体的自主行动能力,原本威胁系统崩溃的这些力量,反而可转化为驱动无限探索与实验的催化剂,催生一种“增强型经济”(Augmented Economy)。本文据此为个人、企业、投资者及政策制定者推导出一套切实可行的行动指南。当下最根本的挑战,并非竞相部署最高度自主的系统,而是竞相筑牢对其监督与治理的根基。唯有使我们的验证带宽与执行能力同步扩展,我们所召唤出的智能,才能真正延续并守护最初赋予它意义的人性。
For millennia, human cognition was the primary engine of progress on Earth. As AI decouples cognition from biology, the marginal cost of measurable execution falls to zero, absorbing any labor capturable by metrics--including creative, analytical, and innovative work. The binding constraint on growth is no longer intelligence but human verification bandwidth: the capacity to validate, audit, and underwrite responsibility when execution is abundant. We model the AGI transition as the collision of two racing cost curves: an exponentially decaying Cost to Automate and a biologically bottlenecked Cost to Verify. This structural asymmetry widens a Measurability Gap between what agents can execute and what humans can afford to verify. It also drives a shift from skill-biased to measurability-biased technical change. Rents migrate to verification-grade ground truth, cryptographic provenance, and liability underwriting--the ability to insure outcomes rather than merely generate them. The current human-in-the-loop equilibrium is unstable: eroded from below as apprenticeship collapses (Missing Junior Loop) and from within as experts codify their obsolescence (Codifier's Curse). Unverified deployment becomes privately rational--a Trojan Horse externality. Unmanaged, these forces pull toward a Hollow Economy. Yet by scaling verification alongside agentic capabilities, the forces that threaten collapse become the catalyst for unbounded discovery and experimentation--an Augmented Economy. We derive a practical playbook for individuals, companies, investors, and policymakers. Today's defining challenge is not the race to deploy the most autonomous systems; it is the race to secure the foundations of their oversight. Only by scaling our bandwidth for verification alongside our capacity for execution can we ensure that the intelligence we have summoned preserves the humanity that initiated it.
许愿