Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines

机器学习推理管道通常在数据科学和工业领域中广泛使用，由于其用户界面的特性，常常需要实时响应。然而，当某些输入特征需要在线聚合大量数据时，满足这一要求变得尤为具有挑战性。最近关于可解释机器学习的文献表明，大多数机器学习模型对输入变化表现出显著的韧性。这表明，机器学习模型可以有效地适应近似输入特征，对准确性的影响很小。在本文中，我们介绍了Biathlon，这是一种新型的ML服务系统，利用模型固有的韧性，并确定每个聚合特征的最佳近似程度。这种方法可以实现最大的加速，同时确保准确性损失有保障。我们在工业应用和数据科学竞赛中对Biathlon进行了评估，证明它能够实现实时延迟要求，实现5.3倍到16.6倍的加速，几乎没有准确性损失。

Biathlon: Approximating Aggregation Features for Real-time ML Inference Pipelines

Biathlon leverages the resilience of machine learning models to determine the optimal degree of approximation for each aggregation feature, enabling maximum speedup while ensuring a guaranteed bound on accuracy loss.

The paper evaluates Biathlon on real pipelines from industry applications and data science competitions, demonstrating its ability to achieve 5.3x to 16.6x speedup with almost no accuracy loss. The approach is novel in its use of model resilience to accommodate approximate input features. The paper also discusses the importance of interpretable machine learning in this context.

Related work includes literature on interpretable machine learning and other methods for optimizing ML inference pipelines for real-time responsiveness.