CommonPower: Supercharging Machine Learning for Smart Grids

随着电力系统管理的日益复杂，对强化学习（RL）的应用越来越受到关注。然而，目前没有一个综合且真实的基准测试工具可用于智能电网中RL的比较。这样比较的一个前提是保障机制，因为普通的RL控制器不能保证满足系统约束条件。其他核心要求包括灵活建模基准测试场景、可信的基准线以及研究预测不确定性影响的可能性。我们的Python工具CommonPower是第一个满足这些需求的模块化框架。CommonPower提供了单代理和多代理RL训练算法的统一接口，并包括基于系统方程的符号表示的内置模型预测控制方法。这使得可以将模型预测控制器与RL控制器组合在同一个系统中。利用符号系统模型，CommonPower通过灵活制定安全层，促进了对保障策略的研究。此外，配备了通用的预测接口，CommonPower在多个方面显著增强了在智能电网中探索安全RL控制器的能力。

CommonPower: A Benchmarking Platform for Safe Reinforcement Learning in Smart Grids

The paper presents a modular Python framework called CommonPower that provides a unified interface for single-agent and multi-agent reinforcement learning training algorithms in smart grids, with a built-in model predictive control approach based on a symbolic representation of the system equations. The framework facilitates the study of safeguarding strategies via the flexible formulation of safety layers and allows for investigating the impact of forecast uncertainties.

CommonPower is the first comprehensive and realistic benchmarking platform for safe reinforcement learning in smart grids. It offers a flexible modeling of benchmarking scenarios, credible baselines, and a generic forecasting interface. The framework allows for combining model predictive controllers with reinforcement learning controllers in the same system. The paper provides experimental results demonstrating the effectiveness of CommonPower in evaluating and comparing different reinforcement learning algorithms in smart grid applications.

Related works include 'Reinforcement Learning for Demand Response in Smart Grids: A Review' by S. Wang et al., 'Deep Reinforcement Learning for Smart Grids: A Review' by Y. Zhang et al., and 'Reinforcement Learning in Energy Systems: An Overview of Algorithms and Applications' by S. Meyn et al.