Chinese Medical Sciences Journal ›› 2021, Vol. 36 ›› Issue (3): 210-217.doi: 10.24920/003968

所属专题: 人工智能与精准肿瘤学

• 论著 • 上一篇    下一篇

CT图像中多器官占位性病变的计算机辅助检测模型的外部和内部验证

徐潋滟1,闫轲2,吕乐2,张伟宏1,陈旭3,霍晓菲3,陆菁菁3()   

  1. 1中国医学科学院 北京协和医学院 北京协和医院 放射科,北京 100730,中国
    2PAII Inc.,贝塞斯达,马里兰州 20817,美国
    3北京和睦家医院放射科,北京 100015 中国
  • 收稿日期:2021-07-10 接受日期:2021-08-30 出版日期:2021-09-30 发布日期:2021-09-23

External and Internal Validation of a Computer Assisted Diagnostic Model for Detecting Multi-Organ Mass Lesions in CT images

Lianyan Xu1,Ke Yan2,Le Lu2,Weihong Zhang1,Xu Chen3,Xiaofei Huo3,Jingjing Lu3()   

  1. 1Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China
    2PAII Inc., Bethesda, MD 20817, USA
    3Department of Radiology, Beijing United Family Hospital, Beijing 100015, China
  • Received:2021-07-10 Accepted:2021-08-30 Online:2021-09-30 Published:2021-09-23

摘要:

目的我们开发了一种在实验室测试中具有较佳表现的通用病变检测模型ULDor。本研究旨在通过外部数据集和内部数据集对其检测性能进行测试并评估其在临床上的应用价值。
方法 通过卷积神经网络(convolutional neural network,CNN)构建通用病变检测模型(ULDor)。该模型经过DeepLesion数据集和其他5个特定器官的公共数据集对模型进行训练,其中DeepLesion数据集包括12,000多组CT扫描图像及其中80,000多个病变注释。验证测试集包括外部和内部验证数据集。外部验证数据集由一家综合医院回顾性收集的164组胸部(含上腹部)CT平扫检查图像组成,内部验证数据集由来自美国国家肺筛查试验(NLST)的187组胸部低剂量螺旋CT扫描图像组成。我们运行ULDor对这两个测试集的图像进行病变检测,记录并测量模型所检测出的所有肺外器官(包括肝、肾、胰腺、甲状腺、淋巴结、体壁、胸椎,等)的占位性病变;另由三名经过资格认证的放射科医生对两个测试集进行人工阅片,以此为标准对ULDor的检测结果进行验证分析,采用阳性预测值和灵敏度来评价模型的检测性能。
结果 在外部验证中,模型对所有病变的整体阳性预测值和敏感性分别为57.9%和67.0%,平均每组图像检测出2.1个病变,其中0.9个是假阳性。ULDor检出肝脏病变的能力最佳,阳性预测值为78.9%,敏感性为92.7%,其次是肾脏,阳性预测值为70.0%,敏感性为58.3%。在内部验证中,尽管图像的软组织噪声水平较高,ULDor仍实现了75.3%的阳性预测值和52.0%的灵敏度。
结论ULDor在外部真实数据的验证显示模型在多用途计算机辅助诊断方面对于某些器官占位病变具有较好的检测效能。通过进一步优化和迭代升级,ULDor或许可以很好地推广应用到外部数据。

关键词: 病变检测, 计算机辅助诊断, 卷积神经网络, 深度学习

Abstract:

Objective We developed a universal lesion detector (ULDor) which showed good performance in in-lab experiments. The study aims to evaluate the performance and its ability to generalize in clinical setting via both external and internal validation.
Methods The ULDor system consists of a convolutional neural network (CNN) trained on around 80K lesion annotations from about 12K CT studies in the DeepLesion dataset and 5 other public organ-specific datasets. During the validation process, the test sets include two parts: the external validation dataset which was comprised of 164 sets of non-contrasted chest and upper abdomen CT scans from a comprehensive hospital, and the internal validation dataset which was comprised of 187 sets of low-dose helical CT scans from the National Lung Screening Trial (NLST). We ran the model on the two test sets to output lesion detection. Three board-certified radiologists read the CT scans and verified the detection results of ULDor. We used positive predictive value (PPV) and sensitivity to evaluate the performance of the model in detecting space-occupying lesions at all extra-pulmonary organs visualized on CT images, including liver, kidney, pancreas, adrenal, spleen, esophagus, thyroid, lymph nodes, body wall, thoracic spine, etc.
Results In the external validation, the lesion-level PPV and sensitivity of the model were 57.9% and 67.0%, respectively. On average, the model detected 2.1 findings per set, and among them, 0.9 were false positives. ULDor worked well for detecting liver lesions, with a PPV of 78.9% and a sensitivity of 92.7%, followed by kidney, with a PPV of 70.0% and a sensitivity of 58.3%. In internal validation with NLST test set, ULDor obtained a PPV of 75.3% and a sensitivity of 52.0% despite the relatively high noise level of soft tissue on images.
Conclusions The performance tests of ULDor with the external real-world data have shown its high effectiveness in multiple-purposed detection for lesions in certain organs. With further optimisation and iterative upgrades, ULDor may be well suited for extensive application to external data.

Key words: lesion detection, computer-aided diagnosis, convolutional neural network, deep learning

Copyright © 2018 Chinese Academy of Medical Sciences. All right reserved.
 
www.cmsj.cams.cn
京公安备110402430088 京ICP备06002729号-1  Powered by Magtech.

Supervised by National Health & Family Plan Commission of PRC

9 Dongdan Santiao, Dongcheng district, Beijing, 100730 China

Tel: 86-10-65105897  Fax:86-10-65133074 

E-mail: cmsj@cams.cn  www.cmsj.cams.cn

Copyright © 2018 Chinese Academy of Medical Sciences

All right reserved.

京公安备110402430088  京ICP备06002729号-1