Neural Exploratory Landscape Analysis

ML DREM ML NN POA EA
2024年08月20日
最近的Meta-Black-Box Optimization(MetaBBO)研究表明,元训练的神经网络可以有效地指导黑盒优化器的设计,显著减少了对专家调优的需求,并在复杂问题分布中提供了稳健的性能。尽管它们取得了成功,但存在一个悖论:MetaBBO仍然依赖于人工设计的探索性地形分析特征,以通知元级代理程序有关低级优化进展的信息。为了解决这个问题,本文提出了神经探索性地形分析(NeurELA),这是一个全新的框架,通过一个两阶段的基于注意力的神经网络动态地分析地形特征,并以完全端到端的方式执行。NeurELA使用多任务神经进化策略在各种MetaBBO算法上进行预训练。广泛的实验表明,当NeurELA集成到不同甚至未见过的MetaBBO任务中时,它能够实现始终优越的性能,并且可以进行高效的微调以进一步提高性能。这一进展标志着MetaBBO算法更加自主和广泛适用的关键一步。NeurELA的源代码可在https://anonymous.4open.science/r/Neur-ELA-303C上访问。
Recent research in Meta-Black-Box Optimization (MetaBBO) have shown that meta-trained neural networks can effectively guide the design of black-box optimizers, significantly reducing the need for expert tuning and delivering robust performance across complex problem distributions. Despite their success, a paradox remains: MetaBBO still rely on human-crafted Exploratory Landscape Analysis features to inform the meta-level agent about the low-level optimization progress. To address the gap, this paper proposes Neural Exploratory Landscape Analysis (NeurELA), a novel framework that dynamically profiles landscape features through a two-stage, attention-based neural network, executed in an entirely end-to-end fashion. NeurELA is pre-trained over a variety of MetaBBO algorithms using a multi-task neuroevolution strategy. Extensive experiments show that NeurELA achieves consistently superior performance when integrated into different and even unseen MetaBBO tasks and can be efficiently fine-tuned for further performance boost. This advancement marks a pivotal step in making MetaBBO algorithms more autonomous and broadly applicable.The source code of NeurELA can be accessed at https://anonymous.4open.science/r/Neur-ELA-303C.
许愿