Creation of endoscopic images
NBI images of neoplastic lesions in patients who underwent endoscopic or surgical resection at Sendai Municipal Medical Center Sendai Public Hospital from April 2017 to December 2019 were used in this single-center retrospective study. I was. The collected NBI image features of him are summarized in Table 1. A total of 1390 NBI images of him from a total of 210 lesions were sampled and a definite histological diagnosis was made.13: 53 low-grade dysplasia (LGD); 120 high-grade dysplasia or mucosal carcinoma (HGD); < 1000 μm) 粘膜下癌 (SM) および 17 の深層浸潤性 (浸潤前部の深さ > 1000 μm) submucosal carcinoma (SMd). Pathological diagnosis was performed blindly by a pathologist blinded to the study design. A diagnosis of mucosal lesion, LGD or HGD was assigned the most severe grade, regardless of component size. The number of images per lesion was 5.5-7, and the average imaging conditions were no magnification 41.0%, no magnification 41.0%, low magnification 37.9%, and high magnification 21.1%. Images of isolated lesions at various magnifications were carefully selected to minimize potential bias in the selection process. Video endoscope CF-HQ290ZI, PCF-H290ZI, PCF-H290TI and video endoscope system EVIS LUCERA ELITE CV-290/CLV-290SL (Olympus Medical Systems, Co., Ltd., Tokyo, Japan) It was used.
Preparing the dataset
The NBI image (Figure 1a) was manually segmented into lesion (Figure 1b) and background (Figure 1c), from which a patch image (128 × 128 pixels) was cropped from the upper left corner (white dotted patch). Right (solid white patch) then down (solid red patch) every 32-pixel stride (white and red arrows) across the active area of interest. Patches with blackouts greater than 10% of the effective area were automatically excluded from analysis. Blackouts were defined as areas where the intensity of the red component was less than 50. Similarly, patches with halation greater than 5% of the active area were also excluded. Halation was defined as the region where the intensity of the green component was above 250. In this study, the patches were further classified into focused and unfocused patches according to the amount of spatial high-frequency region extracted by the high-pass filter. It has a 6.25% Nyquist frequency cutoff. In-focus patches were classified as (0) Background (BG), (1) LGD, (2) HGD, (3) SM, (4) SMd, and out-of-focus patches were classified as (5) Background ground (BG-oof) and 6) lesion (L-oof). A total of 598,801 patches were grouped into seven categories (Table 2). The study had no inclusion or exclusion criteria for patch image quality by endoscopists. As mentioned above, patches with excessive blackouts or halation were automatically filtered out before entry. This study aimed to establish an effective histological classifier that can be used in common imaging conditions for NBI.
Preparing the dataset. The original NBI image (a) manually segmented into lesions (b) and the background (c). Patch images (128 × 128 pixels) stride every 32 pixels (white and red arrows) across the region of interest.
Evaluation method
Employed cross-validation for less biased and more accurate results in machine learning research14In this study, the dataset is randomly split into three equal-sized folds. One of the folds is for validation and the other is for training. The percentage of labels was equal in each fold. The training and validation process was repeated three times using a different fold each time. You can then average the three validation results to produce a single estimate.
CNN architecture
ResNet50 (CNN) proposed by He et al.15 and Pytorch were utilized. ResNet50 without pretraining was imported from the Pytorch library (torchvision.models). The original patch of 128×128 pixels was converted to an image of 224×224 pixels. We adjusted the human-set hyperparameters as follows: loss function, cross-entropy loss; number of training epochs, 50; batch size, 256; trial-and-error learning rate, 0.00005. and the number of outer layers, 7 classes.
Image level classification
Examples of SMd and annotation masks without blackout or halation (indicated by X) are shown in Fig. 2a,i, respectively. Patches classified into BG, LGD, HGD, SM, SMd, BG-oof, and L-oof by the trained CNN are shown in white (Figure 2b), green (Figure 2c), and yellow (Figure 2d). increase. , magenta (Fig. 2e), red (Fig. 2f), dark gray (Fig. 2g), and cyan (Fig. 2h) open squares, respectively, and the corresponding union masks in Fig. 2j–p. Classification algorithms should be developed utilizing focused patches without sacrificing image information. Here, the union masks of patches classified into labels BG, LGD, HGD, SMs, and SMd are M0, M1, M2, M3, and M4, respectively. The union of X and Mi (IoUi) is given by X ∩ Mi/ X.∪Mi (i = 0, 1, 2, 3, 4). Lesions were classified with an argmax of IoUi (i = 0, 1, 2, 3, 4) with IoUi values of 0.12, 0.05, 0.21, 0.04 and 0.57, respectively, leading to label 4 or histologic classification SMd.
Patch-level histological mapping predicted by the trained model, along with annotation masks. A photo of an example SMd and annotation mask without blackout or halation can be found in (a) and (I), Each. White (b), green (c), yellow (d), reddish purple (e), red (fart), dark gray (g), and cyan (time(day –p), Each. Background, background. HGD, high-grade dysplasia; LGD, low-grade dysplasia; SM, surface-invasive submucosal carcinoma. SMd, deep invasive submucosal carcinoma. BG-oof, out-of-focus background. L-oof, out-of-focus lesion.
Ethical Acknowledgment and Consent to Participate
This study was approved by the medical ethics committees of the Hirosaki University Graduate School of Medicine (Aomori Prefecture, document number 2019-1099) and Sendai City Medical Center (Sendai, Japan: document number 2019-0029). Obtain informed consent in the form of opt-out on this site (https://www.https://www.med.hirosaki-u.ac.jp/hospital/outline/resarch.html), Hirosaki University Graduate School Graduate School of Medicine Medical Ethics Committee This study was designed and conducted in accordance with the Declaration of Helsinki.
