Skip to main content
Download PDF

Deep learning to estimate response of concurrent chemoradiotherapy in non-small-cell lung carcinoma

Journal of Translational Medicine volume 22, Article number: 896 (2024) Cite this article

Abstract

Background

Concurrent chemoradiotherapy (CCRT) is a crucial treatment for non-small cell lung carcinoma (NSCLC). However, the use of deep learning (DL) models for predicting the response to CCRT in NSCLC remains unexplored. Therefore, we constructed a DL model for estimating the response to CCRT in NSCLC and explored the associated biological signaling pathways.

Methods

Overall, 229 patients with NSCLC were recruited from six hospitals. Based on contrast-enhanced computed tomography (CT) images, a three-dimensional ResNet50 algorithm was used to develop a model and validate the performance in predicting response and prognosis. An associated analysis was conducted on CT image visualization, RNA sequencing, and single-cell sequencing.

Results

The DL model exhibited favorable predictive performance, with an area under the curve of 0.86 (95% confidence interval [CI] 0.79–0·92) in the training cohort and 0.84 (95% CI 0.75–0.94) in the validation cohort. The DL model (low score vs. high score) was an independent predictive factor; it was significantly associated with progression-free survival and overall survival in both the training (hazard ratio [HR] = 0.54 [0.36−0.80], P = 0.002; 0.44 [0.28−0.68], P < 0.001) and validation cohorts (HR = 0.46 [0.24−0.88], P = 0.008; 0.30 [0.14−0.60], P < 0.001). The DL model was also positively related to the cell adhesion molecules, the P53 signaling pathway, and natural killer cell-mediated cytotoxicity. Single-cell analysis revealed that differentially expressed genes were enriched in different immune cells.

Conclusion

The DL model demonstrated a strong predictive ability for determining the response in patients with NSCLC undergoing CCRT. Our findings contribute to understanding the potential biological mechanisms underlying treatment responses in these patients.

Key message

AbstractSection What is already known on this topic

The relationship between deep learning and biological signaling pathways in the context of concurrent chemoradiotherapy for non-small cell lung carcinoma has not been reported.

AbstractSection What this study adds

Based on contrast-enhanced computed tomography images, a three-dimensional ResNet50 algorithm was used to develop a robust model for response. The DL model demonstrated a strong predictive ability for determining the prognosis in patients with NSCLC undergoing CCRT. Positive associations between CAMs, NK cell-mediated cytotoxicity, and prediction models for CCRT were revealed in patients with NSCLC.

AbstractSection How this study might affect research, practice or policy

This approach is generalizable to any medical imaging analysis, including concurrent chemoradiotherapy, offering a novel noninvasive method to tailor cancer treatment.

Background

Approximately 75–80% of individuals who receive a diagnosis of early-stage and locally advanced non-small-cell lung cancer (NSCLC) are unsuitable for curative surgery [1]. Concurrent chemoradiotherapy (CCRT) is the main treatment strategy for patients diagnosed with locally advanced NSCLC [2, 3]. Moreover, CCRT combined with immunotherapy has further prolonged progression-free survival (PFS) and overall survival (OS). These underscore the urgent need to discover biomarkers for the management of unresectable NSCLC with CCRT [4].

Deep learning (DL) techniques have been extensively employed to analyze medical imaging and sequence data to enable the diagnosis, prognosis, and estimation of therapeutic outcomes in patients with various cancers [5,6,7,8,9,10]. The investigations have led to encouraging results, highlighting the potential effectiveness of machine learning-based approaches [11,12,13,14,15]. Nevertheless, the application of computed tomography (CT) image-based DL models to estimate the response of patients with NSCLC undergoing CCRT has rarely been reported. Additionally, most DL models inherently possess an opaque nature, commonly referred to as “black boxes,” which poses challenges in interpreting predictions [16]. The use of DL models for predicting response to CCRT in NSCLC and the potential interpretability of both bulk-RNA sequencing and single-cell sequencing analyses remain unexplored.

Therefore, we aimed to establish a DL model for predicting response before CCRT using a three-dimensional ResNet 50 algorithm with contrast-enhanced CT (CE-CT) images. Additionally, we analyzed the association of the DL model with OS and PFS. The CT images associated with a predictive response were visualized in the key areas using the gradient-weighted class activation mapping (CAM-Grad) method. Furthermore, bulk-RNA and single-cell sequencing investigations were conducted to elucidate the prediction model’s possible mechanism. The robust and non-invasive predictive model is expected to aid in tailoring precise CCRT strategies for patients with NSCLC.

Methods

Patients

The flowchart of the recruited patients is presented in Supplementary Fig. 1, Additional file 1. Patients who received CCRT at six different medical institutions, namely the Second Affiliated Hospital of Guizhou Medical University, Guiyang Pulmonary Hospital, Qiannan Prefecture Hospital of Traditional Chinese Medicine, Qiandongnan Prefecture People’s Hospital, Qiannan Prefecture People’s Hospital, and the First Affiliated Hospital of Zhengzhou University, were categorized into a training cohort (Dataset A, n = 156) and a validation cohort (Dataset B, n = 73). Additionally, bulk-RNA sequence data of 16 patients (Dataset C) with NSCLC were collected from the Second Affiliated Hospital of Guizhou Medical University, and single-cell sequencing data of 6 patients (GSE139555, Dataset D) were retrieved from GEO (https://www.ncbi.nlm.nih.gov/geo/) [17]. From June 2018 to October 2022, the patients were recruited, and Fig. 1 illustrates the study’s comprehensive flowchart. The institutional review board of the aforementioned five institutions (2023-LUNSHEN-02) and the Second Affiliated Hospital of Guizhou Medical University authorized this study. The study followed the Declaration of Helsinki. Because the patient data were collected retrospectively, informed consent was not required.

Fig. 1
figure 1

Flowchart of the DL model to estimate response. The CE-CT image databases were collected from 229 patients with NSCLC across six centers. CE-CT images were masked and prepared for DL model training. Images were trained to predict responses using the three-dimensional ResNet 50 model in the training cohort. The DL model was validated in the validation cohort. The OS and PFS were further analyzed in the two cohorts. Grad-CAM was visualized among six patients, including those with and without response to CCRT. The association between biological signaling pathways and the DL model was explored in patients with NSCLC. CCRT, concurrent chemoradiotherapy; CE-CT, contrast-enhanced computed tomography; DL, deep learning; Grad-CAM, gradient-weighted class activation mapping; NSCLC, non-small-cell lung carcinoma; OS, overall survival; PFS, progression-free survival

Full size image

CT image preparation process

As detailed in a previous study, CE-CT imaging was performed in hospital settings using multi-slice spiral CT scanners [18]. Following a routine CT scan, contrast media was introduced using a syringe. The CE-CT procedures were conducted immediately after the injection. Furthermore, non-contrast CT images were acquired using the picture archiving and communication system and subsequently adjusted to optimize window settings. All CT images were labeled using MATLAB 2018a (https://ww2.mathworks.cn/) and cropped to a size suitable for input into the DL model. In this study, we calculated an average and random distribution of plaques in each category through data augmentation. In addition to methods such as horizontal flip, vertical flip, horizontal and vertical flip, 90° rotation, and − 90° rotation, we applied other common data augmentation techniques. These included random rotation to introduce variability by rotating images within a specified range; scaling and cropping to adjust image sizes and fit different structures; brightness and contrast adjustments to simulate variations in lighting conditions; adding noise for increased model robustness; color transformations by randomly adjusting image colors; elastic transformations to introduce local shape distortions; and mirror transformations, involving not only horizontal and vertical flips but also diagonal flips or other mirror transformations. Notably, augmentation operations were performed exclusively on the training set to create a more diverse “new” training dataset. To ensure the reliability of model performance evaluation, data augmentation was not applied to the validation cohorts. During the real-time application of data augmentation, we paid special attention to minimizing memory usage. Additionally, we suggest maintaining class balance when considering data augmentation, adjusting parameters for optimization and ensuring that real-time augmentation does not significantly increase training time. We emphasize the importance of preserving the original distribution of the validation set for a better assessment of the model’s performance in real-world scenarios. Finally, we have mentioned other potential data augmentation methods, such as random rotation, scaling, and cropping, as well as the importance of systematically optimizing the model and augmentation hyper-parameters. These details and recommendations contribute to a more comprehensive understanding of the role and impact of data augmentation in our model training.

ResNet50model and implementation

We developed a deep convolutional neural network model aimed at accurately predicting response to CCRT (Fig. 2). To enhance the performance and stability of the model, we carefully selected ResNet50 as the underlying architecture, focusing on feature extraction from bounding box images [19]. ResNet50 comprises five essential modules, including Conv3D + Batch Norm + ReLu + Max Pool, Stage 2, Stage 3, Stage 4, and Conv5, collectively participating in the critical process of feature extraction. To further integrate these extracted features, we introduced a fully connected network with an additional hidden layer containing 1024 nodes. This hidden layer plays a crucial role in integrating features extracted from the convolutional network. By strategically employing the Softmax activation function, the output is transformed into a probability distribution, resulting in the final two classification results. This strategic design further reinforces the model’s performance, ensuring exceptional accuracy in the classification task. The provided description outlines the detailed structure of the ResNet50, elucidating its components such as inputs, outputs, and the composition of layers in each stage. The architectural design of ResNet50 aims to acquire deeper feature representations, significantly enhancing its performance across various tasks, including image classification and other computer vision applications. To ensure the correct implementation of the training program, we have set the maximum epoch to 200. This configuration not only aids the model in learning features more effectively but also provides enough training iterations to ensure thorough convergence of the model. It is noteworthy that we employed PyTorch 2.0 as the implementation tool for the DL framework (https://pytorch.org/) and combined it with Python 3.8 (https://www.python.org/). Such a choice ensures code flexibility and readability while leveraging the powerful features of PyTorch, thereby making the experimental design and implementation more efficient and convenient. The experimental setup was conducted in a Windows operating system environment, with system specifications comprising a 3.7-GHz Intel i7-12700KF CPU, NVIDIA GeForce RTX 3090, and 32 GB of RAM. This hardware configuration is meant to provide sufficient computational power for the experiment, ensuring efficient operation of the model during both training and inference stages while also guaranteeing the reproducibility and comparability of the experiment. The detailed context of ResNet50 is provided in the Supplementary Materials and Methods, Additional file 2.

Fig. 2
figure 2

ResNet50 model construction. The prepared three-dimensional computed tomography images were inputted into the deep learning model. After the fine-tuning and training process, the predictive score was generated for both response and no-response patients

Full size image

Visualization of grad-CAM

The class activation map serves as an indicator of the network’s attention distribution across different regions during the prediction of the target class [19]. Brighter regions on the map correspond to more salient features, aligning with the earlier discussed principles. Following these principles, Grad-CAM effectively generates class-specific activation maps for the target class, offering detailed explanations and visual insights into the decision-making process of deep neural networks. By employing this methodology, we can achieve a more profound understanding of the decision rationale and identify key regions of interest within the network. As previously outlined, integrating Grad-CAM with the ResNet50 architecture enhances the interpretability of the model, allowing for a comprehensive exploration of significant features and decision-making factors in DL processes. The detailed algorithm of Grad-CAM is described in the Supplementary Materials and Methods, Additional file 2.

Bulk-RNA-sequence analysis

Total RNA was extracted utilizing Trizol Reagent (Invitrogen Life Technologies); the concentration, quality, and integrity were evaluated utilizing a NanoDrop spectrophotometer (Thermo Scientific). Three micrograms of RNA were employed for sample preparations. The sequencing libraries were constructed as follows: mRNA was extracted from total RNA utilizing poly-T oligo-attached magnetic beads. Fragmentation was conducted utilizing divalent cations in an Illumina proprietary fragmentation buffer at an elevated temperature. First-strand cDNA was synthesized utilizing random oligonucleotides and a Super Script II kit. Subsequently, second-strand cDNA was generated utilizing DNA polymerase I and RNase H. Remaining overhangs were transformed into blunt ends via exonuclease/polymerase activities, and the enzymes were eliminated. Following adenylation of the 3′ ends of the DNA fragments, Illumina PE adapter oligonucleotides were ligated for hybridization. To select cDNA fragments of 400–500 bp in length, the library fragments were purified employing the AMPure XP system (Beckman Coulter, Beverly, CA, USA). DNA fragments with ligated adaptor molecules on both ends were amplified utilizing Illumina PCR Primer Cocktail in a 15-cycle PCR reaction. The products were purified (AMPure XP system) and quantified with the Agilent high-sensitivity DNA assay on a Bioanalyzer 2100 system (Agilent). The library was then sequenced on the NovaSeq 6000 platform (Illumina) at Shanghai Outdo Biotech Co., Ltd. (Shanghai, China). The raw FASTQ files were aligned to the hg38 genome reference utilizing STAR aligner (version 2.7.4a) with default parameters. Gene expression was quantified utilizing Salmon (version 1.3.0) to generate transcripts per kilobase million (TPM) values. The TPM matrix was then transformed utilizing log2 (TPM + 1) [20].

Gene set enrichment analysis (GSEA) in bulk-RNA-seq

The connection between the signal pathways, genomic expression, and the DL model was ascertained using the GSEA [21]. Radiological data from 16 patients with NSCLC were collected from our hospital. Additionally, the CT images based on DL model classification were classified into two groups (response vs. no-response) and subjected to differential gene expression analysis (fold change ≥ 1.5, P < 0.05) with the “edge” software. The MSigDB gene set database (https://ngdc.cncb.ac.cn/databasecommons/database/id/1077) was used to determine the specific biological pathways.

Single-cell sequencing analysis

A ScRNAseq investigation was conducted to find cell groups expressing genes closely linked to the DL model’s predicted response. This was expected to open up new avenues for research into the cellular signaling underpinnings of CCRT for lung cancer and strategies to develop anti-tumor mechanisms in response to treatment. We selected the NSCLC single-cell chip data (GSE139555) for further analysis. The detailed steps for ScRNAseq are performed in the TISCH2 (http://tisch.comp-genomics.org/gallery/). The raw data have been submitted to EGA (EGAS00001003993).

Statistical analysis

To evaluate the predictive capability of the DL model, the area under the receiver operating characteristic curve (AUC) was utilized as a primary metric. The AUC was calculated using the “pROC” package, while the receiver operating characteristic curve (ROC) was plotted utilizing the “ggplot2” package. Logistic multivariate analysis and Cox multivariate analysis were performed using the “rms” package. The least absolute shrinkage and selection operator (LASSO) algorithm, in conjunction with a five-fold cross-validation method, was utilized to identify the radiomics features with optimal non-zero coefficients. The PFS and OS curves of the DL predictive score (DLS) groups (DLS-low and DLS-high) were evaluated using the Kaplan–Meier method and compared utilizing the log-rank test. All statistical analyses were conducted utilizing R (version 3.5.2; http://www.R-project.org) and GraphPad Prism 9 (https://www.graphpad.com/). A P < 0.05 was considered statistically significant.

Results

Clinical characteristics of patients

Table 1 lists the fundamental clinical characteristics of individuals with NSCLC receiving CCRT in the training and validation groups; 88 (56.41%) patients in the training cohort and 38 (52.05%) in the validation cohort were aged ≤ 60 years. Among the two cohorts, 129 (82.69%) and 62 (84.93%) patients were male, respectively. The majority of patients in the training (61.53%) and validation (60.27%) cohorts were smokers. Furthermore, 94 (60.26%) and 45 (61.64%) individuals mainly had lung squamous carcinoma, respectively. The number of patients with low CEA levels was 133 (85.25%) in the training cohort and 64 (87.67%) in the validation cohort. Additionally, most patients in both cohorts had T3–4 (training, 66.67%; validation, 65.75), N2–3 (training, 87.18%; validation, 89.04%), and M0 stages (training: 76.92%; validation, 100.00%). A total of 53 (33.98) and 34 (46.58%) individuals in the training and validation cohorts experienced an objective response rate, including complete response and partial response, respectively.

Table 1 Characteristics of patients in the training and validation cohorts
Full size table

DL model development and validation in patients with NSCLC undergoing CCRT

The high predictive probabilities (> 0.5) were predominantly observed for the patients with CR or PR (Fig. 3A, B). The DL model showed high AUCs of 0·86 (95% confidence interval [CI] 0.98–1.00) (Fig. 3C) in the training cohort and 0.84 (95% CI 0.75–0.94) (Fig. 3D) in the validation cohort. On the other hand, as illustrated in Supplementary Fig. 2, Additional file 3, three radiomics characteristics (Supplementary Table 1, Additional file 4) were chosen and utilized in the LASSO approach to develop the radiomics model. We found that the DL model for predicting response demonstrated higher AUC values compared with the radiomics model in both the training and validation cohorts (0·86 vs. 0·81; 0·84 vs. 0·80, respectively). DeLong’s test for two ROC curves was not marked in either the training (P = 0·319) or validation cohort (P = 0.577). Additionally, we discovered that the DL model outperformed the T stage in terms of prediction (0.86 vs. 0.67; 0.84 vs. 0.70; DeLong’s test, P < 0.001 and P = 0.046, respectively) (Supplementary Fig. 3, Additional file 5). Decision curves indicated that the DL model outperformed the radiomics model in both cohorts (Fig. 3E–F). Multivariate logistic regression analysis revealed that the T stage (T3–4 vs. T1–2) and DL model (DLS-high vs. DLS-low) were independent predictive factors (OR: 6.70 [2.49–20.28], P < 0.001; 0.01 [95% CI 00.00–0.01], P < 0·001) (Supplementary Table 2, Additional file 6).

Fig. 3
figure 3

Predictive performance of the DL model. a, b The predictive score is compared between the response and no-response groups in the training and validation cohorts. c–d The predictive performances of the DL and radiomics models for estimating response are illustrated as ROC curves in the two cohorts. e–f Decision curves of the DL and radiomics models in the training and validation cohorts are presented. DL, deep learning; ROC, receiver operating characteristic curve

Full size image

PFS and OS analysis of the DL model in the training and validation cohorts

To investigate the connection between the DLS and prognosis, we analyzed the PFS and OS in the two cohorts. We discovered that the DLS-high group had a longer median PFS compared with the DLS-low group in the training cohort (13.6 vs. 7.40 months, respectively; hazard ratio (HR) = 0.54 [0.36–0.80], P = 0.002; Fig. 4A). In the validation cohort, individuals in the DLS-high group exhibited a markedly prolonged median PFS compared with those in the DLS-low group (17·30 vs. 6·60 months, respectively; HR = 0·46 [0.24–0.88], P = 0.008; Fig. 4B). The DLS-high group also showed longer median OS than the DLS-low group in the training and validation cohorts (32.00 vs. 12.00 months, hazard ratio (HR) = 0·44 [0.28–0.68], P < 0.001; 33.00 vs. 11·30 months, HR = 0·30 [0.14–0.60], P < 0.001) (Fig. 4C, D). We conducted further analyses to investigate the association between DLS and immunotherapy. We discovered that the combination group (CCRT + ICI) had longer PFS and OS than the CCRT-alone group in all DLS-high patients (16.60 vs. 6.80 months, HR = 0·35 [0.22–0.55], P < 0.001; 23.70 vs. 10.50 months, HR = 0.28 [0.17–0.45], P < 0.001) (Fig. 4E, F). Additionally, we observed that the combination group (CCRT + ICI) had similar PFS and OS as the CCRT-alone group (P = 0·370 and 0·162) (Supplementary Fig. 4, Additional file 7). Cox regression analysis revealed that DLS, N stage, and M stage were independent predictors of OS for NSCLC in the training cohort (P = 0.003, 0.005, and 0.003, respectively) (Supplementary Table 3, Additional file 8).

Fig. 4
figure 4

Association between the PFS, OS, and the DL model. a–b The DLS predicts the PFS in both the training and validation cohorts. c–d Additionally, the DLS predicts the OS in both the training and validation cohorts. e PFS curves for DLS-high patients are depicted here. f OS curves for DLS-high patients. CCRT, concurrent chemoradiotherapy; DL, deep learning; DLS, deep learning score; ICI, immune checkpoint inhibitor; OS, overall survival; PFS, progression-free survival

Full size image

Response and no-response CT image visualization using Grad-CAM

The activation map of the DL model, specifically from the final convolution layer, was generated for both response and no-response patients to better understand the regions influencing the DL model predictions in the CT images (Fig. 5). The gradient size within this layer served as a metric for the “importance” of each node or voxel concerning the final prediction layer. In Grad-CAM images, the red hues indicated a higher predictive score, and highlights represented the significant regions influencing the DL model’s prediction of response. Meanwhile, the blue hues indicated a poor response in the DL model.

Fig. 5
figure 5

Gradient-weighted Class Activation Mapping. Row 1 shows the CT images from response (CR + PR) and no-response patients (SD + PD) before CCRT. Row 2 shows the CT images from response (CR + PR) and no-response patients (SD + PD) after CCRT. In row 3, Grad-CAM images highlight darker red areas, indicating the regions with the most contribution to maximizing the outputs of the final prediction layer. The red color represents a high predictive score of response, whereas the blue color indicates a low predictive score of response, signifying no response. CCRT, concurrent chemoradiotherapy; CR, complete response; CT, computed tomography; Grad-CAM, Gradient-weighted Class Activation Mapping; PD, progressive disease; PR, partial response; SD, stable disease

Full size image

Association between the DL model and potential biological mechanisms

Based on the differential gene expression analysis (predictive response vs. predictive no-response) of the DL model, 96 genes were significantly up-regulated (1.5-fold change, P < 0.05), and 44 genes were down-regulated (1.5-fold change, P < 0.05) (Supplementary Fig. 5, Additional file 9). Additionally, the up-regulated genes were linked to the cell adhesion molecules (CAMs), P53 signaling pathway, and natural killer (NK) cell-mediated cytotoxicity, according to Gene Ontology (GO) enrichment analysis (Fig. 6A). The down-regulated genes were linked to terpenoid backbone biosynthesis, the spliceosome, and renal cell carcinoma (Fig. 6B). We then analyzed the tumor immune microenvironment using single-cell analysis and found that 78,829 immune cells from 6 patients were clustered into 10 sub-types (Fig. 6C). Three important top genes (CD8A, SIGLEC10, and MFSD12) positively related to the predictive response of the DL model were highly expressed in different immune cells (Fig. 6D). The CD8A gene was enriched in NK and CD8 T cells, whereas SIGLEC10 and MFSD12 were enriched in monocytes/macrophages and dendritic cells (DCs).

Fig. 6
figure 6

Investigating the potential biological mechanism of the DL model. a KEGG pathways identified using GSEA for the set of up-regulated expressed genes. b KEGG pathways identified using GSEA for the set of down-regulated expressed genes. c UMAP plot of 78,829 immune cells from 6 patients with NSCLC; these immune cells were clustered into 10 sub-types. d Distribution of expression of three top correlated genes (CD8A, SIGLEC10, and MFSD12) with the DL model. DL, deep learning; KEGG, Kyoto Encyclopedia of Genes and Genomes. GSEA, Gene Set Enrichment Analysis; NSCLC, Non-small-cell lung carcinoma; UMAP, Uniform Manifold Approximation and Projection

Full size image

Discussion

In this study, a DL model was employed to predict response to CCRT in NSCLC. Our results demonstrated that the DL model outperformed the conventional radiomics model and T staging in predicting CCRT response. Additionally, we observed that patients with a high score on the DL model exhibited longer PFS and OS than those with a low score in both the training and validation cohorts. Among the patients with high DLS, the combination group (CCRT + ICI) showed improved PFS and OS compared with the CCRT-alone group. CAMs and NK cell-mediated cytotoxicity were markedly linked to a high DLS, whereas the top genes (CD8A, SIGLEC10, and MFSD12) related to the predictive response of the DL model were significantly associated with different tumor immune cells.

The application of DL for predicting response to CCRT in NSCLC is rarely reported. In our study, our CT image-based DL model demonstrated good performance in both the training and validation cohorts. Radiomics, as a quantifiable method, plays a pivotal role in predicting the sensitivity of radiotherapy and radiation pneumonitis for lung cancer [22,23,24,25]. However, despite the DL model exhibiting a superior AUC compared with the radiomics model, the difference between the two models was not statistically significant. This indicates that the image-based DL model is a promising method for predicting the efficacy of CCRT in patients with NSCLC. Tumor response to treatment often correlates with a favorable prognosis in many cancers [26,27,28,29,30]. Further analyzing the relationship between efficacy and prognosis in these two cohorts, we found that in both the training and validation cohorts, the DL model was strongly linked to PFS and OS, with a markedly better prognosis for patients with a high DLS than those with a low DLS. These findings suggest that the clinical outcomes predicted by the DL model are closely linked to the prognosis of the patients. In addition, immunotherapy has gained significant importance after CCRT for lung cancer (PACIFIC); however, selecting the dominant population that would most benefit from maintenance immunotherapy remains challenging [31, 32]. In this study, we found that patients with a high DLS significantly benefit from immunotherapy, as opposed to those with low scores. This suggests that the DL model can be used to screen for suitable patients who would most benefit from receiving immunotherapy after CCRT and plays an important guiding role in reducing unnecessary subsequent immunotherapy.

As DL often exhibits a black box effect, interpreting the model becomes challenging, especially when applied in the field of medicine [33,34,35]. Our observations revealed that the high activation regions associated with response prediction were predominantly concentrated in tumor regions. Conversely, patients with stable disease and progressive disease exhibited no high activation regions within the tumors. This visual mapping underscores the significance of tumor tissues, particularly those within the lung, as pivotal factors for the ultimate prediction. This integrated approach not only enhances the interpretability of the DL model but also establishes a comprehensive link between ResNet50’s architectural strengths and Grad-CAM’s interpretative power, providing valuable insights into the decision-making process employing the DL model in the context of response prediction. On the other hand, groundbreaking research utilizing bulk-RNAseq and scRNAseq to develop correlations between imaging DL models and genomic data has yielded novel perspectives that enhance our comprehension of the fundamental immunobiological mechanisms involved in CCRT for lung cancer. The activation of various molecular signaling pathways in the tumor microenvironment may play a pivotal role in the treatment of malignant tumors, as per previous studies [36,37,38]. Specifically, GSEA showed that the DL model was strongly associated with the genomic pathways related to CAMs, the P53 signaling pathway, and NK cell-mediated cytotoxicity. As expected, this association arises because the corresponding tumor elicits a sustained immune response through up-regulated immune activation [39]. In this regard, we have confirmed that NK cells, CD8 T cells, monocytes/macrophages, and DCs play a central role in this process. This result aligns with the knowledge that radiotherapy plays an important role in eliciting tumor responses through activated immune states.

This study has a few limitations. First, it was a multicenter retrospective study, and the validation cohort was small and lacked prospective validation. Larger datasets may further enhance the stability of the model. Second, we did not apply a combined analysis of histopathology and genomics. Our future research aims to explore multi-omics models and prospectively validate the proposed models. Third, we did not optimize the model based on the dynamic changes of tumor images during radiotherapy. In the future, we aim to validate the established model according to the DL model we developed.

Conclusions

In this study, we introduced an innovative prediction model for CCRT response prediction in NSCLC with high accuracy, wherein the DL model showed significant associations with PFS and OS. Additionally, the DL model could identify patients who would benefit from sequential immunotherapy after undergoing CCRT, such as those with a high score on the DL model. Furthermore, based on bulk-RNA and single-cell sequencing, we uncovered associations between CAMs, NK cell-mediated cytotoxicity, and prediction models for CCRT in patients with NSCLC. This study offers a new perspective on predicting the efficacy of CCRT for lung cancer and provides new insights into understanding the potential biological mechanisms.

Data availability

The data that support the findings of this study are available from the corresponding author on reasonable request.

Abbreviations

AUC:

Area under the receiver operating characteristic curve

CAMs:

Cell adhesion molecules

CT:

Computed tomography

CCRT:

Concurrent chemoradiotherapy

CI:

Confidence interval

COREQ:

Consolidated criteria for Reporting Qualitative Research

CE-CT:

Contrast-enhanced

DL:

Deep learning

DCs:

Dendritic cells

GO:

Gene Ontology

CAM-Grad:

Gradient-weighted class activation mapping

HR:

Hazard ratio

LASSO:

Least absolute shrinkage and selection operator

NK:

Natural killer

NSCLC:

Non-small cell lung carcinoma

PFS:

Progression-free survival

ROC:

Receiver operating characteristic curve

OS:

Survival

TPM:

Transcripts per kilobase million

References

  1. Liang Y, Wakelee HA. Adjuvant chemotherapy of completely resected early-stage non-small cell lung cancer (NSCLC). Transl Lung Cancer Res. 2013;2:403–10.

    CAS  PubMed  PubMed Central  Google Scholar 

  2. Uramoto H, Tanaka F. Recurrence after surgery in patients with NSCLC. Transl Lung Cancer Res. 2014;3:242–9.

    PubMed  PubMed Central  Google Scholar 

  3. Taylor MD, Nagji AS, Bhamidipati CM, Theodosakis N, Kozower BD, Lau CL, et al. Tumor recurrence after complete resection for non-small cell lung cancer. Ann Thorac Surg. 2013;93:1813–20.

    Article  Google Scholar 

  4. NSCLC Meta-analysis Collaborative Group. Preoperative chemotherapy for non-small-cell lung cancer: a systematic review and meta-analysis of individual participant data. Lancet. 2014;383:1561–71.

    Article  PubMed Central  Google Scholar 

  5. Peng J, Huang J, Huang G, Zhang J. Predicting the initial treatment response to transarterial chemoembolization in intermediate-stage hepatocellular carcinoma by the integration of radiomics and deep learning. Front Oncol. 2021;11:730282.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Saad MB, Hong L, Aminu M, Vokes NI, Chen P, Salehjahromi M, et al. Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study. Lancet Digit Health. 2023;5:e404–20.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Peng J, Kang S, Ning Z, Deng H, Shen J, Xu Y, et al. Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging. Eur Radiol. 2020;30:413–24.

    Article  PubMed  Google Scholar 

  8. Mu W, Jiang L, Shi Y, Tunali I, Gray JE, Katsoulakis E, et al. Non-invasive measurement of PD-L1 status and prediction of immunotherapy response using deep learning of PET/CT images. J Immunother Cancer. 2021;9:e002118.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Peng J, Zhang J, Zou D, Xiao L, Ma H, Zhang X, et al. Deep learning to estimate durable clinical benefit and prognosis from patients with non-small cell lung cancer treated with PD-1/PD-L1 blockade. Front Immunol. 2022;13:960459.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Chen J, Wang X, Ma A, Wang QE, Liu B, Li L, et al. Deep transfer learning of cancer drug responses by integrating bulk and single-cell RNA-seq data. Nat Commun. 2022;13(1):6494.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Dong D, Fang MJ, Tang L, Shan XH, Gao JB, Giganti F, et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol. 2020;31:912–20.

    Article  CAS  PubMed  Google Scholar 

  12. Vanguri RS, Luo J, Aukerman AT, Egger JV, Fong CJ, Horvat N, et al. Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer. Nat Cancer. 2022;3(10):1151–64.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Hosny A, Parmar C, Coroller TP, Grossmann P, Zeleznik R, Kumar A, et al. Deep learning for lung cancer prognostication: a retrospective multi-cohort radiomics study. PLoS Med. 2018;15:e1002711.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Sun R, Limkin EJ, Vakalopoulou M, Dercle L, et al. A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol. 2018;19:1180–91.

    Article  CAS  PubMed  Google Scholar 

  15. She Y, He B, Wang F, Zhong Y, Wang T, Liu Z, et al. Deep learning for predicting major pathological response to neoadjuvant chemoimmunotherapy in non-small cell lung cancer: a multicentre study. EBioMedicine. 2022;86:104364.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Kermany DS, Goldbaum M, Cai W, Valentim CC, Liang H, Baxter SL, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018;172:1122–31.

    Article  CAS  PubMed  Google Scholar 

  17. Wu TD, Madireddi S, de Almeida PE, Banchereau R, Chen YJ, Chitre AS, et al. Peripheral T cell expansion predicts tumour infiltration and clinical response. Nature. 2020;579(7798):274–8.

    Article  CAS  PubMed  Google Scholar 

  18. He K, Zhang X, Ren R, Sun J. Computed tomography and radiation dose images-based deep-learning model for predicting radiation pneumonitis in lung cancer patients after radiation therapy. Radiother Oncol. 2023;182:109581.

    Article  Google Scholar 

  19. Zhang Z, Wang Z, Luo T, Yan M, Dekker A, De Ruysscher D, Deep residual learning for image recognition., Recognition P et al. (CVPR), Las Vegas, NV, USA 2016;770-8.

  20. Patro R, Duggal G, Love MI, Irizarry RA, Kingsford C, et al. Salmon provides fast and bias-aware quantification of transcript expression. Nat Methods. 2017;14(4):417–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Peng J, Lu F, Huang J, Zhang J, Gong W, Hu Y, et al. Development and validation of a pyradiomics signature to predict initial treatment response and prognosis during transarterial chemoembolization in hepatocellular carcinoma. Front Oncol. 2022;12:853254.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Chen NB, Xiong M, Zhou R, Zhou Y, Qiu B, Luo YF, et al. CT radiomics-based long-term survival prediction for locally advanced non-small cell lung cancer patients treated with concurrent chemoradiotherapy using features from tumor and tumor organismal environment. Radiat Oncol. 2022;17(1):184.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Liu Y, Qi H, Wang C, Deng J, Tan Y, Lin L, et al. Predicting chemoradiotherapy sensitivity with concordant survival benefit in non-small cell lung cancer via computed tomography derived radiomic features. Front Oncol. 2022;12:832343.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Coroller TP, Agrawal V, Huynh E, Narayan V, Lee SW, Mak RH, et al. Radiomic-based pathological response prediction from primary tumors and Lymph nodes in NSCLC. J Thorac Oncol. 2017;12(3):467–76.

    Article  PubMed  Google Scholar 

  25. Jiang W, Song Y, Sun Z, Qiu J, Shi L. Dosimetric factors and Radiomics features within different regions of interest in planning CT images for improving the prediction of Radiation Pneumonitis. Int J Radiat Oncol Biol Phys. 2021;110(4):1161–70.

    Article  PubMed  Google Scholar 

  26. Chen M, Lu H, Copley SJ, Han Y, Logan A, Viola P, et al. A novel radiogenomics biomarker for predicting treatment response and pneumotoxicity from programmed cell death protein or Ligand-1 inhibition immunotherapy in NSCLC. J Thorac Oncol. 2023;18(6):718–30.

    Article  CAS  PubMed  Google Scholar 

  27. Long J, Wang D, Yang X, Wang A, Lin Y, Zheng M, et al. Identification of NOTCH4 mutation as a response biomarker for immune checkpoint inhibitor therapy. BMC Med. 2021;19(1):154.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Peng J, Xiao L, Zou D, Han L. A somatic mutation signature predicts the best overall response to anti-programmed cell death Protein-1 treatment in Epidermal Growth Factor Receptor/Anaplastic Lymphoma kinase-negative non-squamous non-small cell Lung Cancer. Front Med (Lausanne). 2022;9:808378.

    Article  PubMed  Google Scholar 

  29. Khorrami M, Prasanna P, Gupta A, Patil P, Velu PD, Thawani R, et al. Changes in CT radiomic features associated with lymphocyte distribution predict overall survival and response to immunotherapy in non-small cell lung cancer. Cancer Immunol Res. 2020;8:108–19.

    Article  CAS  PubMed  Google Scholar 

  30. Zhang Y, Zhang M, Chen M, Mei J, Xu L, Guo R, et al. Association of sustained response duration with survival after conventional transarterial chemoembolization in patients with hepatocellular carcinoma. JAMA Netw Open. 2018;1:e183213.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Antonia SJ, Villegas A, Daniel D, Vicente D, Murakami S, Hui R, et al. Overall survival with Durvalumab after chemoradiotherapy in stage III NSCLC. N Engl J Med. 2018;379:2342–50.

    Article  CAS  PubMed  Google Scholar 

  32. Jazieh K, Khorrami M, Saad A, Gad M, Gupta A, Patil P, et al. Novel imaging biomarkers predict outcomes in stage III unresectable non-small cell lung cancer treated with chemoradiation and durvalumab. J Immunother Cancer. 2022;10(3):e003778.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Mahesh TR, Vinoth KV, Guluwadi S. Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50. BMC Med Imaging. 2024;24(1):107.

    Article  Google Scholar 

  34. Qayyum F, Khan MA, Kim DH, Ko H, Ryu GA. Explainable AI for material property prediction based on energy cloud: a shapley-driven approach. Mater (Basel). 2023;16(23):7322.

    Article  CAS  Google Scholar 

  35. Amado-Caballero P, Casaseca-de-la-Higuera P, Alberola-López S, Andrés-de-Llano JM, López-Villalobos JA, Alberola-López C, et al. Insight into ADHD diagnosis with deep learning on actimetry: quantitative interpretation of occlusion maps in age and gender subgroups. Artif Intell Med. 2023;143:102630.

    Article  PubMed  Google Scholar 

  36. Han Z, Zhang Z, Yang X, Li Z, Sang S, Islam MT, et al. Development and interpretation of a pathomics-driven ensemble model for predicting the response to immunotherapy in gastric cancer. J Immunother Cancer. 2024;12(5):e008927.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Huang W, Xiong W, Tang L, Chen C, Yuan Q, Zhang C, et al. Non-invasive CT imaging biomarker to predict immunotherapy response in gastric cancer: a multicenter study. J Immunother Cancer. 2023;11(11):e007807.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Fan X, Li J, Huang B, Lu H, Lu C, Pan M, et al. Noninvasive radiomics model reveals macrophage infiltration in glioma. Cancer Lett. 2023;573:216380.

    Article  CAS  PubMed  Google Scholar 

  39. Sun R, Sundahl N, Hecht M, Putz F, Lancia A, Rouyar A, et al. Radiomics to predict outcomes and abscopal response of patients with cancer treated with immunotherapy combined with radiotherapy using a validated signature of CD8 cells. J Immunother Cancer. 2020;8:e001429.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work was supported by the Science and Technology Foundation of Guizhou Province (grant numbers: Qian ke he ji chu-ZK 2021 and yi ban 454) and the Qian Dong Nan Science and Technology Program (Grant Number: qdnkhJz [2023] 14).

Author information

Author notes

  1. Jie Peng and Xudong Zhang contributed equally to this work.

Authors and Affiliations

  1. Department of Oncology, The Second Affiliated Hospital, Guizhou Medical University, Kaili, China

    Jie Peng

  2. Department of Radiation Oncology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China

    Xudong Zhang

  3. Department of Oncology, Guiyang Public Health Clinical Center, Guiyang, China

    Yong Hu

  4. Department of Oncology, Qiandongnan Prefecture People’s Hospital, Kaili, China

    Tianchu He

  5. Department of Oncology, Qiannan Prefecture Hospital of Traditional Chinese Medicine, Duyun, China

    Jun Huang & Mingdan Zhao

  6. Department of Oncology, Qiannan Prefecture People’s Hospital, Duyun, China

    Jimei Meng

Authors
  1. Jie Peng
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xudong Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yong Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Tianchu He
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jun Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Mingdan Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Jimei Meng
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Conception and design: JP; (II) Administrative support: JP; (III) Provision of study materials or patients: JP and XDZ; (IV) Collection and assembly of data: JP, XDZ, and YH; (V) Data analysis and interpretation: JP; (VI) Manuscript writing: All authors; (VII). All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jie Peng.

Ethics declarations

Ethics approval and consent to participate

The institutional review board of the aforementioned five institutions (2023-LUNSHEN-02) and the Second Affiliated Hospital of Guizhou Medical University authorized this study. The study followed the Declaration of Helsinki. Because the patient data was collected retrospectively, informed consent was not required.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, J., Zhang, X., Hu, Y. et al. Deep learning to estimate response of concurrent chemoradiotherapy in non-small-cell lung carcinoma. J Transl Med 22, 896 (2024). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12967-024-05708-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12967-024-05708-4

Keywords

Journal of Translational Medicine

ISSN: 1479-5876

Contact us