AI for Microscopy and Biological Imaging

What AI has changed in biological imaging

Biological imaging has moved from a discipline where a trained eye reviewed each field of view to one where deep learning models process thousands of images before a researcher looks at any of them. This shift did not happen because manual review became impossible; it happened because high-content screening, longitudinal live-cell imaging, and whole-tissue scanning began generating image volumes that manual review could never keep pace with in the first place.

The specific tasks that changed first were the most repetitive ones: identifying individual cells or nuclei, tracking objects across time-lapse frames, and classifying visible phenotypes. Each of these tasks maps naturally onto a computer vision problem, which is why convolutional neural network architectures developed for general image analysis transferred so effectively into microscopy once training data became available. Segmentation, detection, and restoration models now handle the majority of routine image processing in a typical imaging core facility.

Imaging is one piece of a broader shift already reshaping life science research more generally, alongside genomics, research data management, and computational methods across the field. The same pattern recurs across each: generalist, well-trained models increasingly outperform narrow, hand-tuned tools built for a single dataset.

What has not changed is the need for biological judgment. AI models identify structures and patterns; they do not decide what those patterns mean in the context of a specific experiment. Automated cell segmentation reveals nothing about cell health unless a researcher defines the downstream features that matter, and a restored image is only useful if the restoration has not introduced artifacts that could be mistaken for biology. The tools have compressed the mechanical part of image analysis, not the interpretive part.

How deep learning powers cell segmentation

Cell segmentation, the task of delineating individual cell bodies, membranes, or nuclei within an image, has historically been the single most time-consuming step in quantitative microscopy. Image segmentation of this kind is a core computer vision problem, and deep learning has largely solved the generalization issue that limited earlier automated approaches, which typically required extensive parameter tuning for each new imaging modality or cell type.

Cellpose, a generalist segmentation algorithm developed at Janelia Research Campus, demonstrated that a single deep learning model trained on a highly varied dataset of more than 70,000 segmented objects could precisely segment cells across a wide range of image types without retraining or manual parameter adjustment. That generalist capability, rather than any single performance benchmark, is what distinguished Cellpose from earlier specialist tools and drove its rapid adoption across imaging labs.

StarDist takes a different approach to the same problem, representing cell nuclei as star-convex polygons rather than per-pixel masks. This shape representation, introduced by researchers working with star-convex object detection, performs particularly well on densely packed nuclei where bounding-box or pixel-grouping methods are prone to merging adjacent objects or missing partially overlapping ones. The method was later extended to three-dimensional star-convex polyhedra for volumetric microscopy data, adapting to the anisotropic voxel sizes common in fluorescence datasets.

Both tools illustrate a broader pattern in segmentation research: architectural choices matter less than the diversity and scale of training data. Newer approaches, including successive Cellpose releases and segmentation models adapted from general-purpose foundation models, continue to push generalization further, and the shift from specialist, single-modality segmentation tools toward broadly capable deep learning models is likely to continue defining the field.

The same per-cell segmentation step is also the computational foundation for a related but distinct problem: mapping gene expression back to spatial coordinates within tissue sections. Assigning transcript or protein signal to the correct individual cell in a spatial dataset depends on the same accurate boundary detection that drives standard microscopy segmentation, just applied within a tissue-scale imaging context rather than a dissociated cell culture.

AI for object detection and cell tracking

Object detection and tracking extend segmentation into the time dimension, following individual cells, organelles, or particles across sequential frames to reconstruct trajectories, lineages, and dynamic behavior. This is a distinct computational problem from single-frame segmentation because it requires linking detected objects correctly across frames, even when cells divide, merge with neighbors, or temporarily leave the field of view.

TrackMate, an open-source Fiji plugin developed at Institut Pasteur, has become one of the most widely used platforms for this task in the bioimage analysis community. Its most significant recent evolution came with the integration of deep learning detectors: rather than relying solely on blob-like particle detection, current versions incorporate segmentation algorithms including StarDist, Cellpose, and ilastik directly into the tracking pipeline, allowing accurate tracking of irregularly shaped or densely packed objects that earlier tracking methods struggled to follow.

The practical value of combining strong per-frame segmentation with robust linking algorithms shows up most clearly in developmental biology and cell migration studies, where researchers need to reconstruct full lineage trees or quantify migration dynamics across hundreds of cells simultaneously. Manual tracking at that scale is not a realistic option, and the reliability of the underlying segmentation step directly determines how much manual correction a tracking result requires afterward.

How AI enables super-resolution microscopy

Fluorescence microscopy has always required trade-offs between imaging speed, spatial resolution, and the light exposure a sample can tolerate before phototoxicity or photobleaching compromises the experiment. Deep learning has not eliminated these physical trade-offs, but it has shifted where the balance can sit.

Content-aware restoration techniques train neural networks on paired low- and high-quality images so the network learns to recover signal that would otherwise require more light or slower acquisition to capture directly. A related but distinct approach, DeepCAD-RT, applies self-supervised denoising directly to fluorescence time-lapse imaging, learning from low-signal data alone rather than requiring paired high-quality training examples. In the study introducing the method, real-time denoising enabled high-sensitivity imaging with 10-fold fewer photons than standard acquisition, while also cutting processing time 20-fold and memory use by a factor of 27, fast enough to run live on a two-photon microscope.

The practical consequence for a live-cell imaging experiment is that a researcher can acquire data faster or with less illumination and rely on trained restoration models to recover image quality computationally rather than optically. This matters most for long time-lapse experiments in light-sensitive organisms or cell types, where cumulative phototoxicity, not signal-to-noise at any single time point, is the limiting factor. Related deep learning denoising approaches extend the same principle to other imaging modalities, including some electron microscopy applications, and the broader family of AI-enhanced restoration techniques continues to expand alongside advances in generative image models.

AI in high-content screening analysis

High-content screening generates some of the largest image datasets in biological research, often running into millions of individual cell images across a single screening campaign. At that scale, AI-driven analysis is not a convenience; it is the only practical way to convert raw images into interpretable, comparable data.

Phenotypic profiling approaches, most prominently the Cell Painting assay developed at the Broad Institute’s Imaging Platform, use multiplexed fluorescent dyes to label multiple cellular compartments simultaneously and then extract large panels of morphological features from every cell in an image. The current version of the protocol describes labeling eight broadly relevant cellular components using six stains imaged across five channels, producing thousands of quantitative features per cell that can be used to compare the phenotypic effects of chemical or genetic perturbations without predefining which readout matters most.

This feature-rich, unbiased approach is what distinguishes modern phenotypic profiling from earlier high-content assays that measured only one or two predefined endpoints, such as nuclear translocation or a single morphology score. A 2021 review of the field described image-based profiling for drug discovery as a maturing strategy that machine learning is actively renewing, particularly for identifying disease-associated phenotypes and predicting a compound’s mechanism of action. Translating that volume of image data into biological insight, rather than simply generating it, is the harder half of the problem, and it depends on the same segmentation and feature-extraction pipelines that underpin cell segmentation more broadly. High-content screening applications overlap substantially with cytometry-based phenotyping approaches, including automated gating for flow and mass cytometry data, since both fields face the common challenge of extracting meaningful population structure from high-dimensional, per-cell measurements.

Key AI tools for microscopy: Cellpose, StarDist, ilastik and CellProfiler

Four open-source tools account for a large share of the deep learning and machine learning workflows currently in use across biological imaging labs, and each occupies a distinct niche rather than directly competing with the others.

Figure 1: A four-stage schematic of the AI-driven biological imaging pipeline, from raw image acquisition to phenotypic classification. Credit: AI-generated image created using Google Gemini (2026).

• Cellpose provides generalist, largely hands-off cell and nucleus segmentation across a wide range of imaging modalities without requiring model retraining for new experiments.
• StarDist specializes in segmenting densely packed, roughly round nuclei using a star-convex shape representation, and extends naturally to three-dimensional volumetric data.
• ilastik, developed at the European Molecular Biology Laboratory, offers an interactive, no-code interface where users train pixel classification and object tracking models by directly labeling example regions of their own images.
• CellProfiler, developed at the Broad Institute’s Imaging Platform, provides a modular pipeline builder that chains segmentation, feature extraction, and phenotypic classification steps for high-throughput image analysis without requiring programming expertise. A 2021 update to the software, CellProfiler 4, reported a 10-fold reduction in the CPU time needed to run a typical Cell Painting analysis pipeline compared with the prior version.

Table 1: Open-source AI and machine learning tools commonly used in biological image analysis, grouped by primary function.

These tools are frequently used together rather than as alternatives. A typical modern pipeline might use Cellpose or StarDist for segmentation, feed those results into CellProfiler for feature extraction, and use ilastik for tasks that benefit from interactive, user-guided classification, such as distinguishing subtle phenotypic categories that are hard to specify with fixed rules alone. Understanding how these components fit together matters more for a working imaging scientist than mastering any single tool in isolation, and the same underlying segmentation approaches increasingly extend into related domains such as preclinical tissue analysis, where whole-slide histopathology images present many of the same detection and quantification challenges at a different scale.

Choosing the right AI approach for microscopy image analysis

The practical starting point for most imaging scientists is not choosing between competing AI architectures but identifying which task in an existing pipeline is the actual bottleneck. Segmentation quality determines the ceiling for nearly everything downstream, which makes it the most valuable place to invest evaluation time before adopting a full analysis pipeline.

Open-source, generalist tools have made the barrier to entry for AI-assisted image analysis lower than it has ever been, and that trend shows no sign of reversing. The researchers who benefit most from these tools are the ones who understand what each model was trained to do, where its assumptions might not hold for a new imaging modality, and how to validate its output against ground truth before relying on it for a publication-quality result.

This content includes text that has been created with the assistance of generative AI and has undergone editorial review before publishing. Technology Networks’ AI policy can be found here.