• 首页
  • 期刊简介
  • 编委会
  • 投稿指南
  • 收录情况
  • 杂志订阅
  • 联系我们
引用本文:马艳雨,贺平安.基于H&E图像和基因表达数据的多模态深度学习模型预测胃癌生存风险[J].软件工程,2024,(8):37-41.【点击复制】
【打印本页】   【下载PDF全文】   【查看/发表评论】  【下载PDF阅读器】  
←前一篇|后一篇→ 过刊浏览
分享到: 微信 更多
基于H&E图像和基因表达数据的多模态深度学习模型预测胃癌生存风险
马艳雨, 贺平安
(浙江理工大学理学院, 浙江 杭州 310018)
zstumayanyu@163.com; pinganhe@zstu.edu.cn
摘 要: 胃癌作为高发恶性肿瘤,其致死率近年来居高不下,因此精准预测胃癌患者的生存风险对于治疗至关重要。文章提出了一种基于多模态深度学习的预测模型,旨在评估胃癌患者的生存风险。该模型整合了H&E(Hematoxylin-Eosin staining)染色图像和基因表达数据,首先,采用ResNet18卷积神经网络模型提取深层H&E图像信息,将其编码为一维特征向量。其次,采用多模态紧凑型双线性池化方法,将图像特征与基因表达数据进行融合,用于预测胃癌患者的风险分数。在TCGA的胃癌样本中,该模型的一致性指数(c-index)为0.70。在测试集上进行的Kaplan-Meier分析结果显示,模型成功地区分出高风险群和低风险群。结果表明,该模型在区分胃癌患者风险层次方面表现出色,具有显著优势。
关键词: 胃癌;H&E染色图像;基因表达;深度学习;多模态
中图分类号: TP391.41    文献标识码: A
Predicting Gastric Cancer Survival Risk with Multi-Modal Deep Learning Model Based on H&E Images and Gene Expression Dat
MA Yanyu, HE Pingan
(School of Science, Zhejiang Sci-Tech University, Hangzhou 310018, China)
zstumayanyu@163.com; pinganhe@zstu.edu.cn
Abstract: Gastric cancer, as a common malignant tumor, has a high mortality rate in recent years. Therefore, accurately predicting the survival risk of gastric cancer patients is crucial for treatment. This paper proposes a predictive model based on multi-modal deep learning to assess the survival risk of gastric cancer patients. The model integrates H&E ( Hematoxylin-Eosin staining) stained images and gene expression data. Firstly, the ResNet18 convolutional neural network model is used to extract deep H&E image information, encoding it into a one-dimensional feature vector. Secondly, a multi-modal compact bilinear pooling method is employed to merge image features with gene expression data for predicting the risk scores of gastric cancer patients. The model achieved a concordance index (c-index) of 0.70 in gastric cancer samples from TCGA. Kaplan-Meier analysis on the test set successfully differentiates between high-risk and low-risk groups. The results indicate that the model performs well in distinguishing risk levels of gastric cancer patients, demonstrating significant advantages.
Keywords: gastric cancer; H&E stained images; gene expression; deep learning; multi-modal


版权所有:软件工程杂志社
地址:辽宁省沈阳市浑南区新秀街2号 邮政编码:110179
电话:0411-84767887 传真:0411-84835089 Email:semagazine@neusoft.edu.cn
备案号:辽ICP备17007376号-1
技术支持:北京勤云科技发展有限公司

用微信扫一扫

用微信扫一扫