Scientists train deep learning models to scrutinize biopsies like human pathologists

In the age of AI, many healthcare providers dream of being a digital assistant that is not disturbed by fatigue, workload, burnout and hunger, and can provide a simple second opinion on medical decisions, including diagnosis, treatment plans, prescriptions and more.

Today, computing power and AI know-how are available to develop such assistants. However, replicating the expertise of a specially trained, experienced pathologist, radiologist, or other professional is not easy or easy. In particular, AI algorithms require a huge amount of data to create highly accurate models. And the higher quality of data, the better.

Especially for pathologists, a method called pixel-by-pixel manual annotation can be used to train AI models with great success and accurate diagnosis of specific diseases from tissue biopsy images. However, this method requires a trained pathologist to annotate every pixel in the tissue biopsy image and outline the area of ​​interest in machine learning model training. The burden of pathologist annotation in this case is clear, limiting the amount of quality data that can be produced for model training, thereby limiting the diagnostic accuracy of the final model.

To address this challenge, a team of researchers led by scientists at Medsight AI Research Lab, the first hospital at China's Medical University, China's National Joint Engineering Research Center for Immunological Skin Diseases, developed a method to annotate biopsy image data in China.

Researchers published their research Natural Communication July 1st.

“To minimize pathologist workload and gain pathologist expertise… we collect[ed] Pathologist image review patterns [using] Eye tracking device. At the same time, we will design[ed] Deep learning systems, Pathology Expertise Acquisition Network (PEAN) based on collected visual patterns that can decipher the expertise of pathologists [and] diagnosis [whole slide images]”Something from the Medsite AI Research Lab at Northeastern University's School of Medsite and Biological Information Engineering, said Xiaoyu Cui, senior author of the research paper.

Specifically, the team hypothesized that visual data obtained with the eye-opening device could teach AI models of particular interest in biopsy images while reviewing tissue biopsy images, providing a less burdensome area to replace pixel-by-pixel annotations. In this way, the team wanted to extract the pathologist's expertise in a less labor-intensive way, generate more data, and develop and train more accurate, deep learning-assisted diagnostic models.

To achieve this, the team collected slide review data from pathologists using custom-developed software and eye-catching devices reporting pathologist eye movements. A total of 5,881 tissue samples, including five types of skin lesions, were reviewed.

The Pean System calculates “specialty knowledge values” for all areas of a tissue sample by simulating the pathologist's area of ​​interest by comparing eye tracking data with manual pixel annotation data from the same tissue biopsy image. Using this training data, the PEAN model can predict the suspicious area of ​​each biopsy image and classify tissue sample diagnosis (PEAN-C) by mimicking the pathologist's expertise (PEAN-I) or train model.

Surprisingly, the PEAN-C achieved an accuracy of 96.3% and an area under the curve (AUC) of 0.992. This classifies the samples the model was trained to, classify the accuracy of the AUC with 93.0% and AUC, and classify the accurate samples when classifying the AUC of 0.984 in tissue samples. Pean-C was able to outperform the accuracy of the second best AI classification by 5.5% using the same set of external tests.

By mimicking the expertise of pathologists, the Pean-I system allows further selection of areas of interest that will help other learning models to more accurately diagnose tissue images. When the other three learning models, crumb, abmil, and transmit mill, were trained on tissue sample images generated by Pean-I, the accuracy and AUC increased significantly with p values ​​of 0.0053 and 0.0161, respectively.

“Peen is not just a new, deep learning-based diagnostic system, but a pioneering paradigm with the potential to revolutionize the current state of intelligent medical research. It can extract and quantify human diagnostic expertise, thereby overcoming the general shortcomings of mainstream models such as high human resource consumption and low trust from physicists,” CUI said.

The researchers acknowledge that they have only developed a small fraction of the peen potential to help healthcare providers with disease classification and lesion detection. In the future, the authors hope to apply peen to a variety of downstream tasks, including personalized diagnosis, bionic humans, and large-scale predictive models in multimodal.

“As for the ultimate goal, we aim to develop each unique “replica human” of experienced pathologists using Pean and leading language models. It is facilitated by two major benefits of Pean.

Provided by Medsight AI Research Lab