来自爱可可的前沿推介
PyGlove: Efficiently Exchanging ML Ideas as Code
D Peng, X Dong, E Real, Y Lu, Q V. Le
[Google Research]
PyGlove: 以代码形式高效交流机器学习想法
要点:
-
介绍PyGlove库，用于通过符号块将复杂的机器学习(ML)思想以代码形式进行高效和可扩展的共享； -
演示如何在整个机器学习开发过程中使用符号化编程； -
将PyGlove的符号功能从 AutoML 扩展到机器学习，以解决将概念性规则应用于不同的机器学习设置的问题。
一句话总结:
PyGlove 库能通过符号块高效、可扩展以代码形式分享机器学习新想法，有可能改变机器学习程序的开发和组织方式。
摘要：
机器学习(ML)的复杂性和规模不断增加，是对多个团队之间需要更有效的合作。例如，当一个研究团队发明了像"ResNet"这样的新架构时，希望多个工程团队能采用它。然而，每个团队研究和理解该发明所需的努力，并不能随着团队或发明的数量很好地扩展。在本文中，我们介绍了 PyGlove 库的一个扩展，以轻松和可扩展地分享机器学习想法。PyGlove将想法表示为基于规则的符号块，使研究人员能为没有见过的模型写下规则。例如，一个发明者可以写下"增加跳接"的规则。这允许团队之间产生网络效应：在同一时间，任何团队可以向所有其他团队提交符号块。这样的网络效应使得用户可以通过更快地编写更少的代码来迅速克服采用 PyGlove 的成本，提供一个随时间扩展的收益。本文描述了通过符号块组织机器学习的新范式，并将其与现有的方法进行了比较。还对一个大型代码库进行了案例研究，PyGlove 使代码行数减少了80%。
论文地址：https://arxiv.org/abs/2302.01918
The increasing complexity and scale of machine learning (ML) has led to the need for more efficient collaboration among multiple teams. For example, when a research team invents a new architecture like "ResNet," it is desirable for multiple engineering teams to adopt it. However, the effort required for each team to study and understand the invention does not scale well with the number of teams or inventions. In this paper, we present an extension of our PyGlove library to easily and scalably share ML ideas. PyGlove represents ideas as symbolic rule-based patches, enabling researchers to write down the rules for models they have not seen. For example, an inventor can write rules that will "add skip-connections." This permits a network effect among teams: at once, any team can issue patches to all other teams. Such a network effect allows users to quickly surmount the cost of adopting PyGlove by writing less code quicker, providing a benefit that scales with time. We describe the new paradigm of organizing ML through symbolic patches and compare it to existing approaches. We also perform a case study of a large codebase where PyGlove led to an 80% reduction in the number of lines of code.
内容中包含的图片若涉及版权问题，请及时与我们联系删除
评论
沙发等你来抢