COMET: "Cone of experience" enhanced large multimodal model for mathematical problem generation

自动生成高质量的数学问题在许多教育场景中具有实际价值。大型多模态模型因其在跨模态数据场景中的广泛成功而为数学问题生成提供了一种新的技术方法。然而，传统的将问题解决与问题生成分离的方法以及单调数据结构的主流微调框架限制了大型多模态模型在数学问题生成中的应用。针对这些挑战，本文提出了COMET，一种“经验锥”增强的大型多模态模型，用于数学问题生成。首先，从相互促进的角度和应用逻辑的角度，我们将课程知识的学习过程统一为数学问题生成。其次，我们提出了一个由“经验锥”指导的三阶段微调框架。该框架将微调数据分为符号经验、图标经验和直接经验，以与教师职业成长中的经验相对应。该框架设计了几种细粒度的数据构建和注入方法。最后，我们构建了一个中文多模态数学问题数据集，以填补该领域中中文多模态数据的空白。结合客观和主观指标，对多个数据集的实验充分验证了所提出的框架和模型的有效性。

COMET: A Cone of Experience Enhanced Large Multimodal Model for Mathematical Problem Generation

The paper proposes a unified approach for mathematical problem generation and a three-stage fine-tuning framework guided by the 'Cone of Experience' to overcome the limitations of traditional methods in this field.

The proposed framework and model are evaluated on multiple datasets and shown to be effective in generating high-quality mathematical problems. The paper also introduces a Chinese multimodal mathematical problem dataset and provides fine-grained data construction and injection methods. The 'Cone of Experience' is a novel approach to guide the fine-tuning process.

Related work in this field includes 'Neural Problem Generation from Natural Language Textbooks' by Wang et al. and 'Math Word Problem Generation with Graph-to-Sequence Model' by Ling et al.