Identification of carotid artery atherosclerosis is crucial for the diagnosis of the cerebral apoplexy and other vascular diseases.Intravascular optical tomography(IVOCT)has been employed to clinical coronary imaging ...Identification of carotid artery atherosclerosis is crucial for the diagnosis of the cerebral apoplexy and other vascular diseases.Intravascular optical tomography(IVOCT)has been employed to clinical coronary imaging for several years.Vessel morphological information on IVOCT images together with blood flow information on Doppler OCT(DOCT)images could provide a more accurate internal environment of arteries.Images integrated with fluid-structure interaction(FSI)could obtain the accurate mechanical responses and the quantitative material characters.A porcine carotid artery was imaged with an intravascular system(C7-XR,St.Jude Medical Inc.St.Paul,Minnesota,USA)in vivo,during which 120 images of one section and 600 images of a 5 mm/s pull back were captured within 6 s.Those images were then overlapped with Doppler phase changes to imply the changes in flow profiles.Segmentation and quantification of vessel structure was done in the software(MATLAB 2014b),including specifically the segmentation of lumen,imaging catheter,vessel wall and the guide wire.Appropriate interpolation functions are selected in the coordinate transformation algorithm to have smooth boundaries from images.A set of flow algorithms include image segmentation,three-dimensional/two-dimensional model reconstruction,inversion of material parameters,fitting of experimental velocity data and theoretical derivation based on simulation results is proposed.All steps are programmed to provide a theoretical basis for the future simplified process control.3D-reconstruction FSI model was built in SOLIDWORKS by lofting operation based on the segmentation results.Commercial finite element software(COMSOL 5.3,Sweden)numerically analyzed the entity model to obtain vessel stress/strain and flow shear stress data.Boundary conditions are from the OCT detection.Material of the artery was set to be the modified Mooney-Rivlin constitutive model and the parameters used were adjusted in an algorithm to match an ex vivo experiment.Wall shear stresses(WSS)and vessel deformations were chosen to measure the conditions of the artery and would serve as a target variables for future prediction.Thus,the geometric information together with the data of materials and other mechanical properties are possible to obtain during the imaging process.Segmentation process provided anatomically correct models of a two-layered artery.Numerical simulation permits reliable stress distribution in which the position of catheter and the artery curvature have a neglectable disturbance.Shear stress of the fluid is quite small compared with that of the wall at the same interface,which shows good agreement with the former studies.Moreover,a high flushing speed of 0.1 mps have little impact on the stress distributions and magnitudes,which denotes that the OCT imaging process brings little harm to the vessel.It is the first attempt to combine the OCT imaging and Doppler OCT within a full algorithm and a structural analysis.This study is helpful for the biomechanical property studies of carotid arteries and the development of medical imaging technology.展开更多
为去除导丝伪影以提高血管内光学相干断层扫描(IVOCT)的图像质量,辅助医师更加准确地诊断心血管疾病,降低误诊及漏诊的概率,针对IVOCT图像结构信息复杂且伪影区域占比大的难点,提出一种采用生成对抗网络(GAN)架构的基于Transformer的结...为去除导丝伪影以提高血管内光学相干断层扫描(IVOCT)的图像质量,辅助医师更加准确地诊断心血管疾病,降低误诊及漏诊的概率,针对IVOCT图像结构信息复杂且伪影区域占比大的难点,提出一种采用生成对抗网络(GAN)架构的基于Transformer的结构强化网络(SETN)。首先,GAN的生成器在提取纹理特征的原始图像(ORI)主干生成网络的基础上,并联了RTV(Relative Total Variation)图像强化生成网络用于获取图像的结构信息;其次,在ORI/RTV图像的伪影区域重建过程中,引入了分别关注时/空间域信息的Transformer编码器,用于捕获IVOCT图像序列的上下文信息以及纹理/结构特征之间的关联性;最后,利用结构特征融合模块将不同层次的结构特征融入ORI主干生成网络的解码阶段,配合判别器完成导丝伪影区域的图像重建。实验结果表明,SETN的导丝伪影去除结果在纹理和结构的重建上均十分优秀。此外,导丝伪影去除后IVOCT图像质量的提高,对于IVOCT图像的易损斑块分割及管腔轮廓线提取任务均具有积极意义。展开更多
Background Coronary artery calcification is a well-known marker of atherosclerotic plaque burden.High-resolution intravascular optical coherence tomography(OCT)imaging has shown the potential to characterize the detai...Background Coronary artery calcification is a well-known marker of atherosclerotic plaque burden.High-resolution intravascular optical coherence tomography(OCT)imaging has shown the potential to characterize the details of coronary calcification in vivo.In routine clinical practice,it is a time-consuming and laborious task for clinicians to review the over 250 images in a single pullback.Besides,the imbalance label distribution within the entire pullbacks is another problem,which could lead to the failure of the classifier model.Given the success of deep learning methods with other imaging modalities,a thorough understanding of calcified plaque detection using Convolutional Neural Networks(CNNs)within pullbacks for future clinical decision was required.Methods All 33 IVOCT clinical pullbacks of 33 patients were taken from Affiliated Drum Tower Hospital,Nanjing University between December 2017 and December 2018.For ground-truth annotation,three trained experts determined the type of plaque that was present in a B-Scan.The experts assigned the labels'no calcified plaque','calcified plaque'for each OCT image.All experts were provided the all images for labeling.The final label was determined based on consensus between the experts,different opinions on the plaque type were resolved by asking the experts for a repetition of their evaluation.Before the implement of algorithm,all OCT images was resized to a resolution of 300×300,which matched the range used with standard architectures in the natural image domain.In the study,we randomly selected 26 pullbacks for training,the remaining data were testing.While,imbalance label distribution within entire pullbacks was great challenge for various CNNs architecture.In order to resolve the problem,we designed the following experiment.First,we fine-tuned twenty different CNNs architecture,including customize CNN architectures and pretrained CNN architectures.Considering the nature of OCT images,customize CNN architectures were designed that the layers were fewer than 25 layers.Then,three with good performance were selected and further deep fine-tuned to train three different models.The difference of CNNs was mainly in the model architecture,such as depth-based residual networks,width-based inception networks.Finally,the three CNN models were used to majority voting,the predicted labels were from the most voting.Areas under the receiver operating characteristic curve(ROC AUC)were used as the evaluation metric for the imbalance label distribution.Results The imbalance label distribution within pullbacks affected both convergence during the training phase and generalization of a CNN model.Different labels of OCT images could be classified with excellent performance by fine tuning parameters of CNN architectures.Overall,we find that our final result performed best with an accuracy of 90%of'calcified plaque'class,which the numbers were less than'no calcified plaque'class in one pullback.Conclusions The obtained results showed that the method is fast and effective to classify calcific plaques with imbalance label distribution in each pullback.The results suggest that the proposed method could be facilitating our understanding of coronary artery calcification in the process of atherosclerosis andhelping guide complex interventional strategies in coronary arteries with superficial calcification.展开更多
基金supported by the National Natural Science Foundation of China ( 11602166)the Natural Science Foundation of Tianjin ( Grant 16JCYBJC40500)the Key Projects in the Tianjin Science & Technology Pillar Program ( 18YFZCSY00900)
文摘Identification of carotid artery atherosclerosis is crucial for the diagnosis of the cerebral apoplexy and other vascular diseases.Intravascular optical tomography(IVOCT)has been employed to clinical coronary imaging for several years.Vessel morphological information on IVOCT images together with blood flow information on Doppler OCT(DOCT)images could provide a more accurate internal environment of arteries.Images integrated with fluid-structure interaction(FSI)could obtain the accurate mechanical responses and the quantitative material characters.A porcine carotid artery was imaged with an intravascular system(C7-XR,St.Jude Medical Inc.St.Paul,Minnesota,USA)in vivo,during which 120 images of one section and 600 images of a 5 mm/s pull back were captured within 6 s.Those images were then overlapped with Doppler phase changes to imply the changes in flow profiles.Segmentation and quantification of vessel structure was done in the software(MATLAB 2014b),including specifically the segmentation of lumen,imaging catheter,vessel wall and the guide wire.Appropriate interpolation functions are selected in the coordinate transformation algorithm to have smooth boundaries from images.A set of flow algorithms include image segmentation,three-dimensional/two-dimensional model reconstruction,inversion of material parameters,fitting of experimental velocity data and theoretical derivation based on simulation results is proposed.All steps are programmed to provide a theoretical basis for the future simplified process control.3D-reconstruction FSI model was built in SOLIDWORKS by lofting operation based on the segmentation results.Commercial finite element software(COMSOL 5.3,Sweden)numerically analyzed the entity model to obtain vessel stress/strain and flow shear stress data.Boundary conditions are from the OCT detection.Material of the artery was set to be the modified Mooney-Rivlin constitutive model and the parameters used were adjusted in an algorithm to match an ex vivo experiment.Wall shear stresses(WSS)and vessel deformations were chosen to measure the conditions of the artery and would serve as a target variables for future prediction.Thus,the geometric information together with the data of materials and other mechanical properties are possible to obtain during the imaging process.Segmentation process provided anatomically correct models of a two-layered artery.Numerical simulation permits reliable stress distribution in which the position of catheter and the artery curvature have a neglectable disturbance.Shear stress of the fluid is quite small compared with that of the wall at the same interface,which shows good agreement with the former studies.Moreover,a high flushing speed of 0.1 mps have little impact on the stress distributions and magnitudes,which denotes that the OCT imaging process brings little harm to the vessel.It is the first attempt to combine the OCT imaging and Doppler OCT within a full algorithm and a structural analysis.This study is helpful for the biomechanical property studies of carotid arteries and the development of medical imaging technology.
文摘为去除导丝伪影以提高血管内光学相干断层扫描(IVOCT)的图像质量,辅助医师更加准确地诊断心血管疾病,降低误诊及漏诊的概率,针对IVOCT图像结构信息复杂且伪影区域占比大的难点,提出一种采用生成对抗网络(GAN)架构的基于Transformer的结构强化网络(SETN)。首先,GAN的生成器在提取纹理特征的原始图像(ORI)主干生成网络的基础上,并联了RTV(Relative Total Variation)图像强化生成网络用于获取图像的结构信息;其次,在ORI/RTV图像的伪影区域重建过程中,引入了分别关注时/空间域信息的Transformer编码器,用于捕获IVOCT图像序列的上下文信息以及纹理/结构特征之间的关联性;最后,利用结构特征融合模块将不同层次的结构特征融入ORI主干生成网络的解码阶段,配合判别器完成导丝伪影区域的图像重建。实验结果表明,SETN的导丝伪影去除结果在纹理和结构的重建上均十分优秀。此外,导丝伪影去除后IVOCT图像质量的提高,对于IVOCT图像的易损斑块分割及管腔轮廓线提取任务均具有积极意义。
基金supported in part by the National Natural Science Foundation of China ( NSFC ) ( 11772093)ARC ( FT140101152)
文摘Background Coronary artery calcification is a well-known marker of atherosclerotic plaque burden.High-resolution intravascular optical coherence tomography(OCT)imaging has shown the potential to characterize the details of coronary calcification in vivo.In routine clinical practice,it is a time-consuming and laborious task for clinicians to review the over 250 images in a single pullback.Besides,the imbalance label distribution within the entire pullbacks is another problem,which could lead to the failure of the classifier model.Given the success of deep learning methods with other imaging modalities,a thorough understanding of calcified plaque detection using Convolutional Neural Networks(CNNs)within pullbacks for future clinical decision was required.Methods All 33 IVOCT clinical pullbacks of 33 patients were taken from Affiliated Drum Tower Hospital,Nanjing University between December 2017 and December 2018.For ground-truth annotation,three trained experts determined the type of plaque that was present in a B-Scan.The experts assigned the labels'no calcified plaque','calcified plaque'for each OCT image.All experts were provided the all images for labeling.The final label was determined based on consensus between the experts,different opinions on the plaque type were resolved by asking the experts for a repetition of their evaluation.Before the implement of algorithm,all OCT images was resized to a resolution of 300×300,which matched the range used with standard architectures in the natural image domain.In the study,we randomly selected 26 pullbacks for training,the remaining data were testing.While,imbalance label distribution within entire pullbacks was great challenge for various CNNs architecture.In order to resolve the problem,we designed the following experiment.First,we fine-tuned twenty different CNNs architecture,including customize CNN architectures and pretrained CNN architectures.Considering the nature of OCT images,customize CNN architectures were designed that the layers were fewer than 25 layers.Then,three with good performance were selected and further deep fine-tuned to train three different models.The difference of CNNs was mainly in the model architecture,such as depth-based residual networks,width-based inception networks.Finally,the three CNN models were used to majority voting,the predicted labels were from the most voting.Areas under the receiver operating characteristic curve(ROC AUC)were used as the evaluation metric for the imbalance label distribution.Results The imbalance label distribution within pullbacks affected both convergence during the training phase and generalization of a CNN model.Different labels of OCT images could be classified with excellent performance by fine tuning parameters of CNN architectures.Overall,we find that our final result performed best with an accuracy of 90%of'calcified plaque'class,which the numbers were less than'no calcified plaque'class in one pullback.Conclusions The obtained results showed that the method is fast and effective to classify calcific plaques with imbalance label distribution in each pullback.The results suggest that the proposed method could be facilitating our understanding of coronary artery calcification in the process of atherosclerosis andhelping guide complex interventional strategies in coronary arteries with superficial calcification.
