A radiomics model predicts the response of patients with advanced gastric cancer to PD-1 inhibitor treatment

Study design, patient selection and clinical data collection

This clinical trial is a retrospective observational study that was approved by the Ethics Committee of Huazhong University of Science and Technology, and the requirement for written consent was waived in this study (number UHCT-IEC-SOP-016-02-03).

Patients with advanced gastric cancer receiving PD-1 inhibitors from December 2018 to February 2021 were enrolled. The inclusion criteria were as follows: 1) Patient age was > 18 years and < 75 years old. 2) Relapsed or metastatic inoperable gastric adenocarcinoma was cytologically or pathologically confirmed. 3) Gastric cancer progressed after the first line of standard chemotherapy or a targeted molecular therapy and a subsequent anti-PD-1 therapy for at least 4 cycles. 4) The cancer had at least one evaluable lesion that met the Response Evaluation Criteria in Solid Tumors (RECIST 1.1). 5) CT imaging data before and after anti-PD-1 treatments were complete. 6) Follow-up data were complete. Patients whose gastric lesions were not clearly delineated under CT images or whose metastatic lesions could not be measured or evaluated were excluded from this study. Patients whose survival analyses could not be performed were also excluded from the study. All enrolled cases were further distinguished into a training set and validation set according to the enrolled time stage [15]. The cases enrolled before September 2020 were divided into a training set, while enrolled cases from October 2020 to February 2021 were divided into a validation set.

Clinical features such as age, sex, physical status, tumor differentiation, primary tumor site, and treatment outcomes were retrieved from medical records. Molecular biomarkers including PD-L1, MMR, and HER-2 expression levels were detected by immunohistochemistry and in situ hybridization and collected. Per the combined positive score (CPS) of PD-L1, the positive expression of PD-L1 in cancer cells was defined as CPS > 1, and the expression of other molecular markers was interpreted as previously recommended [3].

Treatment protocol and study endpoints

All patients enrolled in this study had been treated with anti-PD-1 treatment. The outcomes were classified as complete remission (CR), partial remission (PR), stable disease (SD), or progressive disease (PD) per the immune-related RECIST (irRECIST) [16]. Based on the immunotherapy evaluation, SD was also considered to indicate lack of an effect. Therefore, patients with CR, PR, or SD were considered non-PD responders. All patients, including PD and non-PD patients, were followed up for progression-free survival (PFS) and overall survival (OS). PFS was defined as the time from study entry to disease progression or death, and OS was defined as the time from study entry to death or the last follow-up. The primary endpoints were the radiomics prediction accuracy and effectiveness of cancer response to PD-1 inhibitor treatment. The secondary endpoint was survival prediction, including PFS and OS, by the radiomics model.

CT imaging data collection

All patients in this study were subjected to contrast-enhanced CT scans (16-MDCT, Brilliance Big bore Philips Health care, Cleveland, OH, USA) at 120 kV. The slice intervals were 5 mm. Portal venous phases of enhanced abdominal CT images were reviewed to determine the tumor location, size, shape, internal echoes of lymph nodes, and the spatial correlation between the tumor and surrounding normal tissues.

Tumor area segmentation and radiomic feature extraction

The primary tumor area on each slice of transverse CT images was manually segmented and set as the region of interest (ROI) by two independent experienced radiologists who were blinded to this study design though the commercial software AccuContuor (Manteia Medical Technologies Co, XiaMen, China). Both radiologists repeated the segmentation procedure twice. The ROI segmented images were resampled with an isotropic voxel size (2×2×2 mm³) and interpolated with the Bspline algorithm. Filters including wavelet and Laplacian of Gaussian (LOG) with different sigma values (0.5–5 with steps 0.5) were applied to all the segmented images. Subsequently, the images were discretized into fixed bin widths. A total of 103 radiomics features were extracted from the original segmented images including 13 shape-based features,17 first-order based features, 22 texture features consisting of gray level cooccurrence matrix (GLCM) features, 16 gray level run length matrix (GLRLM) features, 16 gray level size zone matrix (GLSZM) features, 14 gray level dependence matrix (GLDM) features, and 5 neighboring gray-tone difference matrix (NGTDM) features, through the ontology-guided radiomics analysis workflow (O-RAW) package [17]. For filtered images, the same radiomics features were extracted except the shape-based features. Therefore, a total of 1723 radiomics features were extracted and analyzed in each patient.

Feature selection and radiomics score calculation

All extracted radiomic features were normalized with z scores. Intra- and interobserver reproducibility were evaluated by intraclass correlation coefficients (ICCs), and features with ICCs less than 0.75 were excluded. To avoid collinearity, redundant features (correlation coefficients > 0.9) were excluded by Pearson’s correlation analysis, and zero importance features were removed by a gradient boosting method (LightGBM) using the feature-selector python package (https://github.com/WillKoehrsen/feature-selector). An all-relevant feature selection method was applied to determine the final significant features [18]. A support vector machine (SVM) model and logistic regression model were constructed and compared using 5-fold cross validation and 10 repeats [19]. A radiomics score (rad-score) was built based on the linear combination of the features weighted by the selected model.

Nomogram model building and verification

In order to determine the contributing factors to immunotherapy response, Multivariate logistic regression was used to analyze the relationship of the clinical risk factors including age, sex, primary tumor site, and molecular biomarkers as PD-L1, MMR, HER-2 expression levels, etc., as well as the radiomics score with the immunotherapy response. Radiomics and clinical models were established based on the high-risk clinical factors combined with or without the rad-score, respectively. The discrimination performance of the two models was evaluated by Harrell’s c-index and receiver operator characteristic (ROC) curves in both the training and validation sets. The ROC curves between the two models were compared by the Delong test. The calibration curve and Hosmer–Lemeshow test were used to identify the predictive accuracy of the radiomics model in both sets.

Clinical application evaluation

A decision curve analysis was performed to compare the net benefit difference of the two models and determine the corresponding clinical application value. Based on the total points of the radiomics nomogram, a high or a low-risk probability of disease progression after PD-1 inhibitor treatment was determined. Finally, the impact on survival, such as PFS and OS, was analyzed through survival analysis.

Statistical analysis

Statistical analyses were performed with SPSS (version 26, Chicago, IL, USA) and R software (version 3.6.2, http://www.Rproject.org). Categorical variables were analyzed using the chi-square test or Fisher’s exact test, and continuous variables were analyzed using Student’s t test or Mann–Whitney U test. Correlation analysis was assessed by the Pearson correlation test. Nomograms and calibration curves were plotted by the R package ‘RMS’. The decision curves were plotted by the R package ‘RMDA’. All of the R packages can be installed directly in the R console by commands. The ROC curves were plotted by the python package ‘Scikit-learn’ (https://github.com/scikit-learn/scikit-learn). The Kaplan–Meier method was used for survival analysis, and the log-rank test was used to analyze survival data. Cox regression analysis was performed to identify survival-related factors. For all tests, two-sided p< 0.05 was considered statistically significant.

Baseline features

A total of 87 patients with advanced gastric cancer were included in this study. All enrolled patients were treated with PD-1 inhibitors (toripalimab) after first-line standard chemotherapy, and enrolled patients were divided into a training set (n = 58) and a validation set (n = 29) (Figure 1). Every patient in the cohort was classified as a PD responder or non-PD responder according to the treatment efficacy. The baseline features are listed in Table 1. The median age of the participants was 55 (range: 28–76) years old, and 59.7% of the patients were males. No significant difference was observed in any of these factors, including age, sex, primary tumor site, tumor differentiation, and genetic status (MMR, PD-L1), between the two responders in either the training or validation cohort. However, a significant difference was observed in the level of serum carcinoembryonic antigen (CEA) and the metastatic tumor site between the responder and non-responder groups. Patients with elevated CEA levels exhibited a significantly higher response rate to PD-1 inhibitors than those with normal CEA levels before treatment. In addition, the response rate to PD-1 inhibitors was significantly higher in patients without peritoneal metastasis (Table 1).

Radiomics feature selection

The radiomics feature selection and relevant workflow are illustrated in Figure 2. The primary tumor area was manually segmented and set as the ROI. After extracting important radiomics features and excluding the features with ICCs less than 0.75, 1244 radiomics features were used for further analyses. Colinear feature analysis resulted in 329 features being relevant, and final zero-importance feature analysis revealed that 18 features were important. Of them, 7 features showed significant differences between the response and nonresponse groups. Among these 7 radiomics features, 3 were identified as most relevant at the end of selections. Logistic regression and SVM models were built based on the 3 selected radiomic features to classify the PD responders and non-PD responders. The AUCs of the logistic regression and SVM models were 0.702 (95% CI: 0.694-0.711) and 0.695 (95% CI: 0.685-0.706) in the training set, respectively. More comparison results can be found in Supplementary Table 1. The logistic regression model was chosen for calculating rad-scores for each patient, and the rad-score formulas are listed in the Supplementary Text. Our analyses found that the patients with PD after immunotherapy had obviously higher rad-scores than those without PD in the training set (0.11 ± 1.10 vs. - 0.90 ± 0.68, p < 0.001) and in the validation set (0.64 ± 1.18 vs.-0.88 ± 1.62, p = 0.001) (Figure 3A, 3B).

The radiomics model predicts immunotherapy efficacy better than the clinical model

To identify the clinical risk factors attributing to the therapeutic effect, we performed multi-factor regression analysis. The results manifested that the serum CEA level and the sites of metastasis were two major clinical risk factors contributing to the immunotherapy response (Table 2). Thus, we developed a clinical model consists of two major clinical risk factors and a radiomics model that integrated two clinical risk factors and rad-scores (Figure 4). Our study revealed that the AUCs of the radiomics model were 0.865 (95% CI: 0.812-0.828) and 0.778 (95% CI: 0.732–0.776) in the training and validation datasets. While the AUCs of the clinical model were 0.750 (95% CI: 0.718–0.734) and 0.667 (95% CI: 0.665–0.724), respectively (Table 3 and Figure 5A, 5B). The nomogram calibration curve showed good agreement between the predicted and observed outcomes. Compared with the clinical model, the radiomics model not only performed a more accurate diagnosis than the clinical model (p < 0.01) (Figure 5A, 5B) but also better predicted probabilities of anti-PD-1 efficacy that were consistent with the actual probability in both the training and validation cohorts (Figure 6A, 6B). The decision curve showed that the radiomics model has a better net benefit rate than the clinical model in further (Figure 7).

The radiomics model accurately predicts the progression-free survival

All enrolled patients were followed up to monitor PFS and OS. Two patients’ follow up data were lost, and 85 patients had completed survival data. Among them, 61 patients had PD, and 35 patients died. The median follow-up time was 10 (range: 3–23) months. Based on the radiomics nomogram, we identified the cutoff value for total points as 54 for classifying patients into high risk of disease progression and low-risk disease progression groups after PD-1 inhibitor treatment. Patients with more than 54 total points were grouped in the high-risk group, and the others were grouped in the low-risk group. We found that there was no difference in OS between the high- and low-risk groups (12.0 vs. 10.8 months, p = 0.94). However, the median PFS in the low-risk group was 6.5 months, which was significantly longer than the 3.2 months in the high-risk group (hazard ratio (HR): 1.99, 95% CI: 1.19-3.31, p = 0.009) (Figure 8A, 8B). These data indicated that patients with low-risk values discriminated by the radiomics model had better survival outcomes after immunotherapy. Our univariate and multivariate regression analyses confirmed that the risk value determined by the radiomics model well reflected the prognosis of PFS (Table 4).