Deep Vision Transformers for Prognostic Modeling in Covid-19 Patients Using Large Multi-Institutional Chest Ct Dataset Publisher



Shiri I¹ ; Salimi Y¹ ; Sirjani N² ; Aval AH³ ; Mansouri Z¹ ; Amini M¹ ; Saberi A¹ ; Hajianfar G⁴ ; Pakbin M⁵ ; Oghli MG² ; Oveisi M⁶ ; Zaidi H¹ Show Affiliations
Source: 2022 IEEE NSS/MIC RTSD - IEEE Nuclear Science Symposium# Medical Imaging Conference and Room Temperature Semiconductor Detector Conference Published:2022

Abstract

The importance of prognosis is the assessment of the disease progression, providing more effective management, decreasing mortality, and lowering the time of hospital stay. In convolutional neural network (CNN)-based algorithms, explicit long-range and global relation modeling is a major challenge because of the locality of convolution operations. These challenges result in weak performance because of large inter/intra-patient variabilities, specifically in COVID-19 patients. Transformers successfully used in natural language processing (NLP) tasks could potentially address the limitation of CNN-based algorithms. In the current study, we evaluated deep transformers-based algorithms' performances in the prognostication of COVID-19 patients. Patient data from 19 centers were enrolled in this study. After inclusion and exclusion criteria, 2339 patients remained (1278 alive and 1061 deceased). We implemented a pure Transformer that consists Swin Transformer block. Images split into non-overlap patches (4 by 4) are used as tokens by the patch partitioner, followed by a linear embedding layer. These tokens pass through the patch merging layer and the Swin transformer block (feature representation learning). In addition, we implemented CNN-based algorithms, including ResNet-18, ResNet-50, ResNet-101, and DensNet, for comparison. Data were split into train/validation (70%) and test sets (30%), and all evaluations were performed on test sets. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were reported for the test sets (unseen during training). Regarding AUCs, 0.66, 0.75, 0.72, 0.77, and 0.81 were achieved by ResNet-18, Resnet-50, Resnet-101, DensNet, and Transformers.Considering all parameters Transformer significantly (p-value <0.05) outperformed all CNN-based models with Accuracy, Sensitivity, Specificity, and AUC of 0.75, 0.78, 0.71, and 0.81, respectively. © 2022 IEEE.