subject
The Institutional Review Board (IRB) of Yongin Severance Hospital approved this prospective study (IRB number 9-2021-0106) and all participants provided written informed consent to participate in this study. Did. Informed consent was given by radiologists who voluntarily agreed to participate in this study. An attending radiologist who agreed to collect reading times of her CXR interpretation daily from September 2021 until he December was prospectively recruited in August 2021 (Fig. 1). All radiologists wishing to participate in the study, regardless of their experience in the field of radiology, are board-certified radiologists, employed at the hospital for the duration of the study, and agree to the terms. was eligible for inclusion. Her two authors of this study were excluded from the participants to minimize bias. At our hospital, his radiographs, including CXR, are read by all radiologists, regardless of subspecialty, and we recommend over 500 of her radiographs each month. Therefore, the radiologist was requested to read her CXR as she would normally do in her daily routine, with a minimum requirement of 300 her CXR per month for the duration of the study. They freely read her CXR independently, blinded to the reading time, while referring to the electronic medical record or previous available images.
Applying AI to CXR
At our hospital, a commercially available AI-based lesion detection software (Lunit Insight CXR, version 3, Lunit, South Korea) has been integrated into all CXRs since March 2020. Physicians can simply scroll down the photo image to see the analyzed AI results. Archival Communication System (PACS) as the analysis results were attached to her second image of her original CXR when the patient was examined. The software detects a total of 8 lesions (atelectasis, cardiomegaly, sclerosis, fibrosis, nodule, pleural effusion, pneumoperitoneum, pneumothorax) and provides a contour map showing the location of the lesion when the operating point exceeds 15%. (Fig. 3). Detected lesions, abbreviations, and anomaly scores are displayed separately on PACS. Abnormality score is determined by AI and represents the probability of presence of lesion in CXR ranging from 0 to 100%. Of the detected lesions’ abnormality scores, the highest score was used as the total abnormality score, which was listed as a separate column in his PACS. Therefore, the physician could refer to the AI results at any time and the radiologist could use the total abnormality score column in her PACS to prioritize her CXR during the reading session, if desired. . A more detailed description of the AI integration process into all her CXRs was given in a recent study.20,27Therefore, participating radiologists had used AI software for >1 year during the relevant study period.
a The AI results attached to the second image of the original CXR include contour plots, abbreviations, and anomaly scores for detected lesions. Physicians can simply scroll down the original image in PACS to see the AI results. b The highest anomaly score was used as the total anomaly score for each CXR, which was listed as a separate column (red square) in the PACS.
Measurement of reading time during non-AI support/AI support period
Read time was defined as the number of seconds from opening the CXR to transferring the image at the PACS by the same radiologist. The read time for each CXR can be extracted from the PACS log record. For participating radiologists, preconfigure the PACS to not display AI results for September and November 2021 (AI non-assisted period) and AI results for October and December 2021 (AI-assisted period) is automatically displayed (Fig. 1). During the AI-free period, AI results, including the secondary captured images attached to his original CXR and the Worklist Abnormal Score column, were not automatically displayed in PACS, and participating radiologists could was not informed. However, during the AI-assisted period, the results were made available and freely available to radiologists. Her CXR of patients aged 18 years and older was included in the analysis, as the software is approved for her CXR in adults. Based on the outlier detection method, outliers with read times greater than 51 s were excluded. These outliers in read times can result from a variety of conditions, including delays in interpreting the corresponding CXRs after opening due to unexpected interruptions from other work.12.
For included CXRs, information on patient age, sex, and whether the CXR was obtained at an inpatient or outpatient clinic was reviewed using electronic medical records. The patient’s location during CXR, including ER, general ward, and intensive care unit, was also identified. The presence of a previous equivalent CXR was analyzed as a possible factor affecting read time. For AI results, the anomaly score was analyzed both as a continuous variable using the numbers themselves and as a categorical variable by applying a cutoff value of 15%. This cut-off value was chosen because he adopted an operating point of 15% when our institution determined the presence of lesions according to vendor guidelines.12When the operating point exceeded 15%, the AI software marked the lesion location using contour plots, anomaly scores, and abbreviations for each lesion on the image.20Therefore, the presence or absence of lesions such as atelectasis, cardiomegaly, sclerosis, fibrosis, nodules, pleural effusion, pneumoperitoneum, and pneumothorax were evaluated using each abnormality score itself as a continuous variable and applying the action point. Additionally, the highest score was used as the total anomaly score for each CXR and used to determine whether the CXR contained anomalies.
statistical analysis
An R program (4.1.3, Foundation for Statistical Computing, Vienna, Austria, packages lme4, lmerTest) was used for statistical analysis. We used the 1.5 IQR method to exclude CXRs with read time outliers. This method is the traditional method of defining outliers using the 1st quartile (6 seconds in this study) and the 3rd quartile (24 seconds). The formula for determining the cutoff value for outliers is: 24 + (24–6) × 1.5 = 51 seconds.Chi-square test and 2 samples t– The test was used to compare the total number of CXRs included and the patient’s age between AI-free and AI-using periods. A linear mixed model was used to compare read times taking into account random effects of radiologist and patient. We compared read times in seconds between AI-free and AI-using periods according to patient characteristics (gender, age, location, and the presence of her previous comparable CXR). Read times were detected by AI (one of the following eight abnormalities: atelectasis, cardiomegaly, sclerosis, fibrosis, nodule, pleural effusion, pneumoperitoneum, pneumothorax) using a 15% operating point Comparisons were made according to the presence of lesions. When the anomaly score was considered a continuous variable, we compared read times between conditions without and with AI. Variables, AI availability, and their interactions were considered fixed effects in a linear mixed model. p– Values less than 0.05 were considered statistically significant.
Report overview
For more information on the study design, see the Nature Research Reporting Summary linked to this article.
