结合多尺度特征融合和注意力机制的肺腺癌病理图像分类胶囊网络

李思雨; 高静; 王云玲; 帕力旦·吐尔逊; 马玉花

doi:10.13568/j.cnki.651094.651316.2023.07.22.0001

您当前的位置：

首页 >

文章列表页 >

结合多尺度特征融合和注意力机制的肺腺癌病理图像分类胶囊网络

信息科学与技术 | 更新时间：2026-01-21

- 结合多尺度特征融合和注意力机制的肺腺癌病理图像分类胶囊网络
- 结合多尺度特征融合和注意力机制的肺腺癌病理图像分类胶囊网络
- 新疆大学学报（自然科学版中英文） 2024年41卷第3期页码：319-328
- 作者机构：
  
  1. 新疆大学软件学院
  2. 新疆临床基因检测与生物医学信息重点实验室
  3. 新疆克拉玛依市消化系统肿瘤精准医学临床研究中心
  4. 新疆克拉玛依市中心医院病理科
  5. 新疆医科大学第一附属医院影像中心
  6. 新疆师范大学计算机科学技术学院
- 作者简介：
- 基金信息：
  
  中央引导地方科技发展专项“新疆地区缺血性脑卒中风险筛查及多模态影像人工智能研究服务平台建设”（ZYYD2023D02）;新疆维吾尔自治区自然科学基金面上项目“极光激酶A抑制剂（MLN8237）逆转EGFR-TKI耐药的分子机制研究”（2021D01A24）;新疆维吾尔自治区自然科学基金地州科学基金“基于振动光谱结合多变量分析的肺癌早期诊断模型的研究”（2021D01F35）;新疆维吾尔自治区科技支疆项目“基于Prophet-ARMA（预测-自回归滑动平均模型）的医院门诊就诊量预测方法与管理对策研究”（2021E02078）
- DOI：10.13568/j.cnki.651094.651316.2023.07.22.0001
  中图分类号： TP391.41
- 纸质出版：2024
- 稿件说明：
移动端阅览
[1]李思雨,高静,王云玲,等.结合多尺度特征融合和注意力机制的肺腺癌病理图像分类胶囊网络[J].新疆大学学报(自然科学版)(中英文),2024,41(03):319-328.
[1]李思雨,高静,王云玲,等.结合多尺度特征融合和注意力机制的肺腺癌病理图像分类胶囊网络[J].新疆大学学报(自然科学版)(中英文),2024,41(03):319-328. DOI： 10.13568/j.cnki.651094.651316.2023.07.22.0001.

DOI：10.13568/j.cnki.651094.651316.2023.07.22.0001.

摘要

病理学家通过分析肺腺癌低级别组织和癌旁组织来确定病灶切除范围，然而，两者间的细胞形态差异较小，分析时依赖病理学家的主观经验，耗时且易误诊．故提出一种结合多尺度特征融合和通道自注意力的胶囊网络（Multi-Scale Feature Fusion with Self-Channel Attention for Capsule Network

MSCNet），用于帮助医生高效诊断疾病，为患者提供更好的治疗方案．首先，设计了多尺度特征融合模块来提升胶囊网络以捕捉同源图像不同尺度间的语义信息，试图减少模型计算量以提高处理速度及分类准确性．其次，通道自注意力（Self-Channel Attention

SCA）模块作为MSCNet的另一重要组件，可以寻找到更具代表性的特征，辅助识别组织病理学图像中的细微特征，降低误诊风险．实验结果表明，在肺腺癌低级别组织与癌旁组织的二分类任务中，MSCNet实现了99.34%的分类准确率、97.65%的F1-Score值和97.57%的精确度．

Abstract

Pathologists determine the extent of lesion resection by analyzing low-grade and paracancerous tissues of lung adenocarcinoma; however

the cellular morphology differences between the two are small

and the analysis relies on the pathologist's subjective experience

which is time-consuming and prone to misdiagnosis. Therefore

this study proposes a capsule network(Multi-Scale Feature Fusion with Self-Channel Attention for Capsule Network

MSCNet) that combines multi-scale feature fusion and attention mechanisms for helping physicians efficiently diagnose diseases and provide better treatment options for patients. Specifically

we first design a multi-scale feature fusion module to enhance the capsule network to capture semantic information between different scales of homologous images in an attempt to reduce the amount of model computation in order to improve processing speed and classification accuracy. Second

the self-channel attention(SCA) module

as another important component of MSCNet

can find more representative features to assist in identifying subtle features in histopathology images and reduce the risk of misdiagnosis. Experimental results show that MSCNet achieves 99.34% classification accuracy

97.65% F1-Score value

and 97.57% precision in the task of binary classification of low-grade tissues and paracancerous tissues of lung adenocarcinoma.

关键词

Keywords

references

ZHAO X Z,QI S L,ZHANG B H,et al.Deep CNN models for pulmonary nodule classification:Model modification,model integration,and transfer learning[J].Journal of X-Ray Science and Technology,2019,27(4):615-629.

NASRULLAH N,SANG J,ALAM M S,et al.Automated lung nodule detection and classification using deep learning combined with multiple strategies[J].Sensors,2019,19(17):3722.

CAO W M,WU R,CAO G T,et al.A comprehensive review of computer-aided diagnosis of pulmonary nodules based on computed tomography scans[J].IEEE Access,2020,8:154007-154023.

SUNG H,FERLAY J,SIEGEL R L,et al.Global cancer statistics 2020:GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J].CA:A Cancer Journal for Clinicians,2021,71(3):209-249.

NICHOLSON A G,TSAO M S,BEASLEY M B,et al.The 2021 WHO classification of lung tumors:Impact of advances since2015[J].Journal of Thoracic Oncology,2022,17(3):362-387.

闫睿，罗洞波，刘艳萍，等．蛋白磷酸酶2A抑癌因子在非小细胞肺癌组织中的表达及其对预后的影响[J]．新疆大学学报(自然科学版)，2017,34(2):213-217.YAN R,LUO D B,LIU Y P,et al.Expression of cancerous inhibitor of protein phosphatase 2A in non-small cell lung cancer on prognosis[J].Journal of Xinjiang University(Natural Science Edition),2017,34(2):213-217.(in Chinese)

WANG X,CHEN H,GAN C X,et al.Weakly supervised deep learning for whole slide lung cancer image analysis[J].IEEE Transactions on Cybernetics,2020,50(9):3950-3962.

KRIEGSMANN M,HAAG C,WEIS C A,et al.Deep learning for the classification of small-cell and non-small-cell lung cancer[J].Cancers,2020,12(6):1604.

YANG H,CHEN L L,CHENG Z Q,et al.Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images:A retrospective study[J].BMC Medicine,2021,19(1):80.

DING S S,GAO Z Y,WANG J,et al.Fractal graph convolutional network with MLP-mixer based multi-path feature fusion for classification of histopathological images[J].Expert Systems with Applications,2023,212:118793.

屈莹，陈晨，吕小毅．多特征融合结合机器学习算法快速筛查葡萄膜炎[J]．新疆大学学报(自然科学版)(中英文)，2021,38(4):439-449.QU Y,CHEN C,LYU X Y.Multi-feature fusion combined with machine learning algorithms to quickly screen uveitis[J].Journal of Xinjiang University(Natural Science Edition in Chinese and English),2021,38(4):439-449.(in Chinese)

胡红萍，乔世昌，孔慧华，等．基于加权平均樽海鞘群算法和BP神经网络的COVID-19预测[J]．新疆大学学报(自然科学版)(中英文)，2022,39(1):19-25.HU H P,QIAO S C,KONG H H,et al.Prediction of COVID-19 based on the weighted average salp swarm algorithm and BP neural network[J].Journal of Xinjiang University(Natural Science Edition in Chinese and English),2022,39(1):19-25.(in Chinese)

SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.December 4-9,2017.Long Beach,CA,USA.ACM,2017:3859-3869.

XIANG C Q,ZHANG L,TANG Y,et al.MS-CapsNet:A novel multi-scale capsule network[J].IEEE Signal Processing Letters,2018,25(12):1850-1854.

QAYYUM A,ANWAR S M,AWAIS M,et al.Medical image retrieval using deep convolutional neural network[J].Neurocomputing,2017,266(C):8-20.

KUMAR A,KIM J,LYNDON D,et al.An ensemble of fine-tuned convolutional neural networks for medical image classification[J].IEEE Journal of Biomedical and Health Informatics,2017,21(1):31-40.

XU Y,JIA Z P,WANG L B,et al.Large scale tissue histopathology image classification,segmentation,and visualization via deep convolutional activation features[J].BMC Bioinformatics,2017,18(1):281.

HUANG Y,ZHENG H,LIU C,et al.Epithelium-stroma classification via convolutional neural networks and unsupervised domain adaptation in histopathological images[J].IEEE Journal of Biomedical and Health Informatics,2017,21(6):1625-1632.

BEJNORDI B E,ZUIDHOF G,BALKENHOL M,et al.Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images[J].Journal of Medical Imaging,2017,4(4):044504.

KOMURA D,ISHIKAWA S.Machine learning methods for histopathological image analysis[J].Computational and Structural Biotechnology Journal,2018,16:34-42.

KHOSRAVI P,KAZEMI E,IMIELINSKI M,et al.Deep convolutional neural networks enable discrimination of heterogeneous digital pathology images[J].EBioMedicine,2018,27:317-328.

DABEER S,KHAN M M,ISLAM S.Cancer diagnosis in histopathological image:CNN based approach[J].Informatics in Medicine Unlocked,2019,16:100231.

WU Z J,WANG L,LI C R,et al.DeepLRHE:A deep convolutional neural network framework to evaluate the risk of lung cancer recurrence and metastasis from histopathology images[J].Frontiers in Genetics,2020,11:768.

MNIH V,HEESS N,GRAVES A,et al.Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2.December 8-13,2014.Montreal,Canada.ACM,2014:2204-2212.

HUANG W K,ZHOU F B.DA-CapsNet:Dual attention mechanism capsule network[J].Scientific Reports,2020,10:11383.

ADLA D,REDDY G V R,NAYAK P,et al.Deep learning-based computer aided diagnosis model for skin cancer detection and classification[J].Distributed and Parallel Databases,2022,40(4):717-736.

HUA Y S,MOU L C,ZHU X X.Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification[J].ISPRS Journal of Photogrammetry and Remote Sensing,2019,149:188-199.

ZHANG Z L,XIAO T H,FU Y,et al.3D multi-resolution attention capsule network for diagnosing multi-pathological types of pulmonary nodules[J].International Journal of Imaging Systems and Technology,2022,32(5):1727-1742.

XU R,CONG Z,YE X C,et al.Pulmonary textures classification via a multi-scale attention network[J].IEEE Journal of Biomedical and Health Informatics,2020,24(7):2041-2052.

QIU S,MA J L,MA Z P.IRCM-Caps:An X-ray image detection method for COVID-19[J].The Clinical Respiratory Journal,2023,17(5):364-373.

VAHADANE A,PENG T Y,SETHI A,et al.Structure-preserving color normalization and sparse stain separation for histological images[J].IEEE Transactions on Medical Imaging,2016,35(8):1962-1971.

ZHANG Z,YE S W,LIAO P,et al.Enhanced capsule network for medical image classification[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine&Biology Society(EMBS).July 20-24,2020.Montreal,QC,Canada.IEEE,2020:1544-1547.

PANG J M,LI C,SHI J P,et al.R2-CNN:Fast tiny object detection in large-scale remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(8):5512-5524.

DING Y F,MA Z Y,WEN S G,et al.AP-CNN:Weakly supervised attention pyramid convolutional neural network for fine-grained visual classification[J].IEEE Transactions on Image Processing,2021,30:2826-2836.

XI X M,MENG X J,QIN Z Y,et al.IA-net:Informative attention convolutional neural network for choroidal neovascularization segmentation in OCT images[J].Biomedical Optics Express,2020,11(11):6122-6136.

TOˇGAC AR M，¨OZKURT K B,ERGEN B,et al.BreastNet:A novel convolutional neural network model through histopathological images for the diagnosis of breast cancer[J].Physica A:Statistical Mechanics and Its Applications,2020,545:123592.

HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23,2018.Salt Lake City,UT,USA.IEEE,2018:7132-7141.

WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module[C]//European Conference on Computer Vision.Cham:Springer,2018:3-19.

HU J,SHEN L,ALBANIE S,et al.Gather-excite:Exploiting feature context in convolutional neural networks[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.December 3-8,2018.Montreal,Canada.ACM,2018:9423-9433.

CHEN Y P,KALANTIDIS Y,LI J S,et al.A2-Nets:Double attention networks[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.December 3-8,2018.Montreal,Canada.ACM,2018:350-359.

ROY A G,NAVAB N,WACHINGER C.Recalibrating fully convolutional networks with spatial and channel “squeeze and excitation” blocks[J].IEEE Transactions on Medical Imaging,2019,38(2):540-549.

FU J,LIU J,TIAN H J,et al.Dual attention network for scene segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).June 15-20,2019.Long Beach,CA,USA.IEEE,2019:3141-3149.

WANG Q L,WU B G,ZHU P F,et al.ECA-net:Efficient channel attention for deep convolutional neural networks[C]//2020IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).June 13-19,2020.Seattle,WA,USA.IEEE,2020:11531-11539.

JIA Z Y,CAI X Y,JIAO Z H.Multi-modal physiological signals based squeeze-and-excitation network with domain adversarial learning for sleep staging[J].IEEE Sensors Journal,2022,22(4):3464-3471.

LI X,WANG W H,HU X L,et al.Selective kernel networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).June 15-20,2019.Long Beach,CA,USA.IEEE,2019:510-519.

CHANDALIYA P K,NAIN N.AW-GAN:Face aging and rejuvenation using attention with wavelet GAN[J].Neural Computing and Applications,2023,35(3):2811-2825.

ZHU X Z,CHENG D Z,ZHANG Z,et al.An empirical study of spatial attention mechanisms in deep networks[C]//2019IEEE/CVF International Conference on Computer Vision(ICCV).October 27-November 2,2019.Seoul,Korea(South).IEEE,2019:6687-6696.

HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).June 27-30,2016.Las Vegas,NV,USA.IEEE,2016:770-778.

SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].3rd International Conference on Learning Representations,ICLR 2015-Conference Track Proceedings,2015:1-14.

HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).July 21-26,2017.Honolulu,HI.IEEE,2017:2261-2269.

NI J H,YAN Z Z,JIANG J H.TongueCaps:An improved capsule network model for multi-classification of tongue color[J].Diagnostics,2022,12(3):653.

WANG P,WANG J X,LI Y M,et al.Automatic classification of breast cancer histopathological images based on deep feature fusion and enhanced routing[J].Biomedical Signal Processing and Control,2021,65:102341.

AFSHAR P,HEIDARIAN S,NADERKHANI F,et al.COVID-CAPS:A capsule network-based framework for identification of COVID-19 cases from X-ray images[J].Pattern Recognition Letters,2020,138:638-643.

ZHOU P Y,CAO Y Z,LI M,et al.HCCANet:Histopathological image grading of colorectal cancer using CNN based on multichannel fusion attention mechanism[J].Scientific Reports,2022,12:15103.

MOREIRA A L,OCAMPO P S S,XIA Y H,et al.A grading system for invasive pulmonary adenocarcinoma:A proposal from the international association for the study of lung cancer pathology committee[J].Journal of Thoracic Oncology,2020,15(10):1599-1610.

浏览量

300

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

双注意力门融合网络的水下图像增强方法

具有互补特征学习框架和注意力特征融合模块的语音情感识别模型