Establishment of a machine learning-based prognostic model for corneal ulcers

Study participants

In the first part of modeling, we retrospectively collected 4973 slit lamp images of patients with corneal ulcers from the First Affiliated Hospital of Guangxi Medical University, Ophthalmology Research Center of Xiamen University, and Ophthalmology Research Center of Sun Yat-sen University (SUSTEC-SYSU database).¹⁷ From 2017 to 2019, four ocular surface physicians divided the images into five datasets according to the lesion features: corneal ulcer (1960 images), corneal scar (1734 images), corneal neovascularization (947 images), anterior corneal abscess (234 images), and corneal abrasion (98 images). Each dataset was divided into training, test, and validation sets according to different central sources. For each image, regions of interest (ROIs) were manually annotated using LabelMe (4.5.6) software. Pathological ROIs included corneal scar, neovascularization, anterior corneal abscess, and abrasion on diffuse white-light slit-lamp images. Corneal ulcer lesions were annotated using diffuse blue-light slit-lamp images. Lesions were labeled by a keratopathy expert and three ophthalmologists. Non-pathological ROIs included limbal and pupil markings and were identified by three ophthalmologists. Each slit lamp image contained at least one lesion and corneal limbus and pupil markings. In the second part of modeling, clinical data and 1010 slit lamp images were retrospectively collected from 240 corneal ulcer patients admitted between December 2019 and May 2022. The data were randomly split into training and validation sets in an 8:2 ratio. The main inclusion criterion was patients with etiologically confirmed infectious corneal ulcers. The main exclusion criterion was patients who underwent surgical treatment during diagnosis and treatment. (Supplementary Figure 1) For eligible patients, demographic and clinical data were collected, including age at onset, corrected visual acuity in the affected eye, type of corneal ulcer, and slit lamp images from two light sources. No written or verbal informed consent was obtained from participants, as this study was a non-interventional retrospective study design and all data were analyzed anonymously (Figure 1).

Data Preprocessing and Augmentation

In the segmentation model, the shortest side of all data was randomly resized from 260 to 1040, and the long side was changed proportionally. Then, random clipping to 480×480 was performed. In training, vertical flipping was not applied considering the dataset was generally positive, and horizontal flipping was introduced with a 50% probability to increase the size of the training dataset considering the symmetry between the left and right eyes.

For the classification model, the data was randomly cropped to 224 × 224 pixels and fitted with the ResNSet50 model and transfer learning. Similar to the segmentation model, only horizontal flipping was introduced as a data augmentation technique.

Segmentation Model

Automated ROI segmentation models of slit lamp images were performed for five different tasks: corneal abrasion, corneal scar, corneal neovascularization, blue light corneal ulcer, and anterior corneal abscess. Popular deep learning segmentation models such as FCN, U-net, and DeepLabV3 were investigated. Combining unique application scenarios and model performance on benchmarks, the DeeplabV3 model was selected as the basis for all subsequent models.

In the Deeplab series segmentation models, the dilated convolution technique was introduced, which adds an “atlas” to the convolution operation to expand the receptive field.¹⁸ The network can effectively capture multi-scale information at different rates. (Supplementary) DeeplabV3 also adds a branch of ASPP to improve the overall view of the image. Specifically, we first compress the resolution of the feature maps to 1 × 1 using GAP, and then utilize 1 × 1 convolution to adjust the number of channels to 256. Finally, we adjusted the image resolution to the target resolution by batch normalization and bilinear interpolation upsampling. For each region of interest (ROI) type, we trained a separate classification model. All models were fine-tuned using a transfer learning algorithm based on model parameters pre-trained on the MS COCO dataset (Figure 2A).

Classification Models

ResNet has been widely used in various feature extraction applications. The deeper the number of layers in a deep learning network, the stronger the representation ability. However, when the CNN network reaches a certain depth and goes deeper, the classification performance no longer improves, and the network converges slowly and the accuracy decreases, so the classification performance and accuracy did not improve. The connection of residual blocks can effectively solve the gradient variance in the training process. Similarly, in the classification of corneal scars, we utilized transfer learning and initialized the model with parameters pre-trained on ImageNet. We trained the ResNet-50 model with the following hyperparameters: batch size: 32, initial learning rate (init_lr): 0.01, cosine learning decay is the same as the segmentation model, epochs: 50, optimizer: SGD (Fig. 2B and Supplementary).

Feature building

Using classification and segmentation algorithms, automated segmentation and classification of ROIs in slit lamp images was performed. To obtain better prognostic models, the data features identified by the algorithm need to be further quantified and predictive models developed.

[Corneal scarring]

1. 1.

   We automatically classify the cornea through a classification network and obtain the label and corresponding probability for each category.

2. 2.

   The percentage of corneal scar area relative to the corneal area is graded.

   $$class_{CSs} = \left\{ {\begin{array}{*{20}l} 1 \hfill & {{\text{ if 0}}{.00} \le \frac{{area_{CSs} }}{{area_{corneal} }} \le 0.25} \hfill \\ 2 \hfill & {{\text{ if 0}}{.25 < }\frac{{area_{CSs} }}{{area_{corneal} }} \le 0.50} \hfill \\ 3 \hfill & {{\text{ if 0}}{.50 < }\frac{{area_{CSs} }}{{area_{corneal} }} \le 0.75} \hfill \\ 4 \hfill & {{\text{ if 0}}{.75 < }\frac{{area_{CSs} }}{{area_{corneal} }} \le 1.00} \hfill \\ \end{array} } \right.$$

   area_CS The variables in the formula are corneal scar area and_cornea The variable in the formula is the corneal area.

3. 3.

   Whether corneal scarring is blocking the pupil.

   $$corneal\_{overlap}_{CSs}=\left\{\begin{array}{ll}0& \text{ if }are{a}_{CSs}\cap are{a}_{corneal}=\phi \\ 1& \text{ else}\end{array}\right.$$

4. Four.

   Number of corneal quadrants occupied by corneal scar

[Corneal abrasion]

1. 1.

   Classification of the rate of corneal abrasion

   $$class_{CD} = \left\{ {\begin{array}{*{20}l} 1 \hfill & {{\text{ if 0}}{.00} \le \frac{{area_{CD} }}{{area_{corneal} }} \le 0.25} \hfill \\ 2 \hfill & {{\text{ if 0}}{.25 < }\frac{{area_{CD} }}{{area_{corneal} }} \le 0.50} \hfill \\ 3 \hfill & {{\text{ if 0}}{.50 < }\frac{{area_{CD} }}{{area_{corneal} }} \le 0.75} \hfill \\ 4 \hfill & {{\text{ if 0}}{.75 < }\frac{{area_{CD} }}{{area_{corneal} }} \le 1.00} \hfill \\ \end{array} } \right.$$

   area_CD The variable in the formula is the corneal ablation area, and the area is_cornea The variable in the formula is the corneal area.

[Anterior corneal abscess]

1. 1.

   The depth of the abscess in the anterior chamber is assessed as a percentage of the vertical diameter of the cornea.

   $$class_{ACA} = \left\{ {\begin{array}{*{20}l} 1 \hfill & {{\text{ if 0}}{.00} \le \frac{{area_{ACA} }}{{area_{corneal} }} \le 0.25} \hfill \\ 2 \hfill & {{\text{ if 0}}{.25 < }\frac{{area_{ACA} }}{{area_{corneal} }} \le 0.50} \hfill \\ 3 \hfill & {{\text{ if 0}}{.50 < }\frac{{area_{ACA} }}{{area_{corneal} }} \le 0.75} \hfill \\ 4 \hfill & {{\text{ if 0}}{.75 < }\frac{{area_{ACA} }}{{area_{corneal} }} \le 1.00} \hfill \\ \end{array} } \right.$$

   area_{Australian Consumers Association} The variables in the formula are anterior corneal abscess area and area_cornea The variable in the formula is the corneal area.

[Blue light induced corneal ulcer]

1. 1.

   The corneal ulcer area is graded by the percentage of blue light corneal ulcer.

   $$class_{CU} = \left\{ {\begin{array}{*{20}l} 1 \hfill & {{\text{ if 0}}{.00} \le \frac{{area_{CU} }}{{area_{corneal} }} \le 0.25} \hfill \\ 2 \hfill & {{\text{ if 0}}{.25 < }\frac{{area_{CU} }}{{area_{corneal} }} \le 0.50} \hfill \\ 3 \hfill & {{\text{ if 0}}{.50 < }\frac{{area_{CU} }}{{area_{corneal} }} \le 0.75} \hfill \\ 4 \hfill & {{\text{ if 0}}{.75 < }\frac{{area_{CU} }}{{area_{corneal} }} \le 1.00} \hfill \\ \end{array} } \right.$$

   area_CU The variables in the formula are the blue light corneal ulcer area and the area_cornea The variable in the formula is the corneal area.

2. 2.

   Whether the corneal ulcer blocks the pupil under blue light.

   $$corneal\_{overlap}_{CU}=\left\{\begin{array}{ll}0& \text{ if }are{a}_{CU}\cap are{a}_{corneal}=\phi \\ 1& \text{ else}\end{array}\right.$$

3. 3.

   Number of corneal quadrants occupied by blue light corneal ulcers.

[Corneal neovascularization]

1. 1.

   Grading of the percentage of corneal neovascular area relative to the corneal area.

   $$class_{CN} = \left\{ {\begin{array}{*{20}l} 1 \hfill & {{\text{ if 0}}{.00} \le \frac{{area_{CN} }}{{area_{corneal} }} \le 0.25} \hfill \\ 2 \hfill & {{\text{ if 0}}{.25 < }\frac{{area_{CN} }}{{area_{corneal} }} \le 0.50} \hfill \\ 3 \hfill & {{\text{ if 0}}{.50 < }\frac{{area_{CN} }}{{area_{corneal} }} \le 0.75} \hfill \\ 4 \hfill & {{\text{ if 0}}{.75 < }\frac{{area_{CN} }}{{area_{corneal} }} \le 1.00} \hfill \\ \end{array} } \right.$$

   area_CN The variable in the formula is the corneal neovascular area, and the area is_cornea The variable in the formula is the corneal area.

2. 2.

   Whether corneal neovascularization obstructs the pupil.

   $$corneal\_{overlap}_{CN}=\left\{\begin{array}{ll}0& \text{ if }are{a}_{CN}\cap are{a}_{corneal}=\phi \\ 1& \text{ else}\end{array}\right.$$

3. 3.

   Number of corneal neovascularizations in the corneal quadrant.

Model Building

The results automatically recognized by the artificial intelligence algorithm were converted into features and combined with the patients' corresponding clinical features to perform the feature pre-fusion method to construct the final prognostic model.

Before building the model, we used the LASSO model to screen these features for different tasks and reduce the feature dimensionality to some extent. After filtering the features, we used common machine learning algorithms such as Gradient Boosting (XGB).¹⁹ For algorithm validation, we use models such as LightGBM.