LogRCA: Log-based Root Cause Analysis for Distributed Services

为了帮助IT服务开发人员和运营商管理日益复杂的服务环境，越来越多的工作致力于利用人工智能来进行运营。特别是为了加速故障排除，日志异常检测受到了广泛关注，它处理的是指示系统故障原因的日志事件的识别。然而，故障往往在系统内广泛传播，这可能导致现有方法检测到大量异常。在这种情况下，用户很难快速识别故障的实际根本原因。我们提出了LogRCA，一种新的方法，用于识别描述根本原因的最小日志行集合。LogRCA使用半监督学习方法来处理罕见和未知的错误，并设计用于处理嘈杂的数据。我们在一个包含4430万条日志行的大规模生产日志数据集上评估了我们的方法，其中包含80个由专家标记了根本原因的故障。在检测候选根本原因的精度和召回率方面，LogRCA始终优于基于深度学习和统计分析的基准线。此外，我们调查了我们部署的数据平衡方法的影响，证明它在罕见故障的性能上有很大的改进。

LogRCA: Identifying Minimal Causes for Production System Failures via Root Cause Analysis on Logs

The paper proposes a semi-supervised learning approach called LogRCA to identify a minimal set of log lines that together describe a root cause of system failures. It handles rare and unknown errors and is designed to handle noisy data.

LogRCA outperforms baselines based on deep learning and statistical analysis in terms of precision and recall to detect candidate root causes. The approach was evaluated on a large-scale production log data set of 44.3 million log lines. The paper also investigated the impact of a deployed data balancing approach, which considerably improves performance on rare failures.

Related work includes log anomaly detection and root cause analysis methods such as DeepLog, LogMine, and LogCluster.