PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applications

发展智能儿科咨询系统为提高诊断效率提供了有前途的前景，特别是在医疗资源匮乏的中国。尽管中文医学大语言模型（LLMs）近年来取得了一定进展，但由于缺乏足够的指导数据和易受攻击的训练程序，其在儿科应用中的表现仍然不够理想。为解决上述问题，本文构建了PedCorpus，一个高质量的数据集，包含来自儿科教材、指南和知识图谱资源的30万个多任务指令，以满足多样化的诊断需求。在经过精心设计的PedCorpus的基础上，我们提出了PediatricsGPT，这是第一个基于系统化和强大的训练流程构建的中文儿科LLM助手。在连续的预训练阶段，我们引入了一种混合指令预训练机制，以减轻LLMs在医学领域适应中注入的内部知识不一致性。随后，我们采用全参数监督微调（SFT）来将一般医学知识模式纳入模型。之后，我们设计了直接跟随偏好优化，以增强生成类似于儿科医生的人性化响应。在参数高效的次级SFT阶段，我们提出了通用-特定专家混合策略，以解决医学普通专家和儿科专业技能之间的能力冲突。基于指标、GPT-4和不同医生下游任务的医生评估的广泛结果表明，PediatricsGPT始终优于以前的中文医学LLMs。我们的模型和数据集将开源供社区开发。

Developing an intelligent pediatric consultation system to improve diagnostic efficiency in China where healthcare resources are scarce.

Building PedCorpus, a high-quality dataset of over 300,000 multi-task instructions from pediatric textbooks, guidelines, and knowledge graph resources to fulfill diverse diagnostic demands. Proposing PediatricsGPT, the first Chinese pediatric LLM assistant built on a systematic and robust training pipeline that includes a hybrid instruction pre-training mechanism, full-parameter Supervised Fine-Tuning, direct following preference optimization, and a mixture of universal-specific experts strategy.

Extensive experiments show that PediatricsGPT consistently outperforms previous Chinese medical LLMs based on metrics, GPT-4, and doctor evaluations on distinct doctor downstream tasks. The model and dataset will be open-source for community development.

Recent related studies include advances in Large Language Models (LLMs) for Chinese medicine, but their performance is sub-optimal in pediatric applications due to inadequate instruction data and vulnerable training procedures. No previous studies have proposed a Chinese pediatric LLM assistant built on a systematic and robust training pipeline like PediatricsGPT.