SmartEM: machine learning-guided electron microscopy

Machine Learning


  • White, J. G., Southgate, E., Thomson, J. N. & Brenner, S. The structure of the nervous system of the nematode Caenorhabditis elegans. Philos. Trans. R. Soc. Lond. B Biol. Sci. 314, 1–340 (1986).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Hildebrand, D. G. C. et al. Whole-brain serial-section electron microscopy in larval zebrafish. Nature 545, 345–349 (2017).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Witvliet, D. et al. Connectomes across development reveal principles of brain maturation. Nature 596, 257–261 (2021).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Winding, M. et al. The connectome of an insect brain. Science 379, eadd9330 (2023).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Dorkenwald, S. et al. Neuronal wiring diagram of an adult brain. Nature 634, 124–138 (2024).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Bock, D. D. et al. Network anatomy and in vivo physiology of visual cortical neurons. Nature 471, 177–182 (2011).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Kasthuri, N. et al. Saturated reconstruction of a volume of neocortex. Cell 162, 648–661 (2015).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Morgan, J. L., Berger, D. R., Wetzel, A. W. & Lichtman, J. W. The fuzzy logic of network connectivity in mouse visual thalamus. Cell 165, 192–206 (2016).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Abbott, L. F. et al. The mind of a mouse. Cell 182, 1372–1376 (2020).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Lu, X. et al. A scalable staining strategy for whole-brain connectomics. Preprint at bioRxiv https://doi.org/10.1101/2023.09.26.558265 (2023).

  • Song, K., Feng, Z. & Helmstaedter, M. High-contrast en bloc staining of mouse whole-brain and human brain samples for EM-based connectomics. Nat. Methods 20, 836–840 (2023).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Rother, A., Januszewski, M., Jain, V., Fee, M. S. & Kornfeld, J. The songbird basal ganglia connectome Preprint at bioRxiv https://doi.org/10.1101/2025.10.25.684569 (2020).

  • Loomba, S. et al. Connectomic comparison of mouse and human cortex. Science 377, eabo0924 (2022).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Karlupia, N. et al. Immersion fixation and staining of multicubic millimeter volumes for electron microscopy-based connectomics of human brain biopsies. Biol. Psychiatry 94, 352–360 (2023).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Bidel, F. et al. Connectomics of the Octopus vulgaris vertical lobe provides insight into conserved and novel principles of a memory acquisition network. eLife 12, e84257 (2023).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Shapson-Coe, A. et al. A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution. Science 384, eadk4858 (2024).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Swanson, L. W. & Lichtman, J. W. From Cajal to connectome and beyond. Annu. Rev. Neurosci. 39, 197–216 (2016).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Zheng, Z. et al. A complete electron microscopy volume of the brain of adult drosophila melanogaster. Cell 174, 730–743 (2018).

    Article 

    Google Scholar 

  • Phelps, J. S. et al. Reconstruction of motor control circuits in adult drosophila using automated transmission electron microscopy. Cell 184, 759–774 (2021).

    Article 

    Google Scholar 

  • Lichtman, J. W., Pfister, H. & Shavit, N. The big data challenges of connectomics. Nat. Neurosci. 17, 1448–1454 (2014).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Beier, T. et al. Multicut brings automated neurite segmentation closer to human performance. Nat. Methods 14, 101–102 (2017).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Januszewski, M. et al. High-precision automated reconstruction of neurons with flood-filling networks. Nat. Methods 15, 605–610 (2018).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Lee, K. et al. Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy. Curr. Opin. Neurobiol. 55, 188–198 (2019).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Meirovitch, Y. et al. Cross-classification clustering: an efficient multi-object tracking technique for 3-d instance segmentation in connectomics. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 8425–8435 (IEEE, 2019).

  • Sheridan, A. et al. Local shape descriptors for neuron segmentation. Nat. Methods 20, 295–303 (2023).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Popovych, S. et al. Petascale pipeline for precise alignment of images from serial section electron microscopy. Nat. Commun. 15, 289 (2024).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Celii, B. et al. Neurd offers automated proofreading and feature extraction for connectomics. Nature 640, 487–496 (2025).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Januszewski, M., Templier, T., Hayworth, K., Peale, D. & Hess, H. Accelerating neuron reconstruction with PATHFINDER. Preprint at bioRxiv https://doi.org/10.1101/2025.05.16.654254 (2025).

  • Anderson, H.S., Ilic-Helms, J., Rohrer, B., Wheeler, J. & Larson, K. Sparse imaging for fast electron microscopy. In Proc. IS&T/SPIE Electronic Imaging (eds Bouman, C. A. et al.) 86570C-1–86570C-12 (SPIE, 2013).

  • Mohammed, A. & Abdullah, A. Scanning electron microscopy (SEM): a review. In Proc. 2018 International Conference on Hydraulics and Pneumatics—HERVEX (eds Matache, G. et al.) 77–85 (Hydraulics and Pneumatics Research Institute, 2018).

  • Minnen, D. et al. Denoising-based image compression for connectomics. Preprint at bioRxiv https://doi.org/10.1101/2021.05.29.445828 (2021).

  • Fang, L. et al. Deep learning-based point-scanning super-resolution imaging. Nat. Methods 18, 406–416 (2021).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Meila, M. Comparing clusterings by the variation of information. In Learning Theory and Kernel Machines, Proc. 16th Annual Conference on Computational Learning Theory and 7th Kernel Workshop, COLT/Kernel (eds Schölkopf, B. & Warmuth, M. K.) 173–187 (Springer, 2003).

  • Lee, K., Zung, J., Li, P., Jain, V. & Seung, H.S. Superhuman accuracy on the snemi3d connectomics challenge. Preprint at https://arxiv.org/abs/1706.00120(2017).

  • Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (eds Navab, N. et al.) 234–241 (Springer, 2015).

  • Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. 32nd International Conference on Machine Learning (eds Bach, F. & Blei, D.) 448–456 (PMLR, 2015); https://proceedings.mlr.press/v37/ioffe15.html

  • He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016); https://doi.org/10.1109/CVPR.2016.90

  • Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. International Conference on Learning Representations (ICLR) (ICLR, 2015).

  • Vincent, L. & Soille, P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13, 583–598 (1991).

    Article 

    Google Scholar 

  • Maitin-Shepard, J. et al. google/neuroglancer. Zenodo https://doi.org/10.5281/zenodo.5573294 (2021).

  • Turner, N. L. et al. Synaptic partner assignment using attentional voxel association networks. In Proc. 17th IEEE International Symposium on Biomedical Imaging (ISBI) 1–5 (IEEE, 2020); https://doi.org/10.1109/ISBI45749.2020.9098489

  • Bae, J. A. et al. Functional connectomics spanning multiple areas of mouse visual cortex. Nature 640, 435–447 (2025).

    Article 

    Google Scholar 

  • Thorpe, S., Fize, D. & Marlot, C. Speed of processing in the human visual system. Nature 381, 520–522 (1996).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Knott, G., Marchman, H., Wall, D. & Lich, B. Serial section scanning electron microscopy of adult brain tissue using focused ion beam milling. J. Neurosci. 28, 2959–2964 (2008).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Denk, W. & Horstmann, H. Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol. 2, e329 (2004).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Rolnick, D. et al. Morphological error detection in 3D segmentations. Preprint at https://arxiv.org/abs/1705.10882 (2017).

  • Zung, J., Tartavull, I., Lee, K. & Seung, H. S. An error detection and correction framework for connectomics. In Proc. 31st Annual Conference on Neural Information Processing Systems (eds Guyon, I. et al.) 6818–6829 (NeurIPS, 2017).

  • Athey, T. et al. Analysis of smart imaging runtime. Appl. Microsc. 55, 10 (2025).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Hayworth, K. J. et al. Imaging atum ultrathin section libraries with wafermapper: a multi-scale approach to EM reconstruction of neural circuits. Front. Neural Circuits 8, 68 (2014).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Potocek, P. Adaptive specimen image acquisition using an artificial neural network. US patent 10,928,335 (2021).

  • Pizer, S. M., Johnston, R. E., Ericksen, J. P., Yankaskas, B. C. & Muller, K. E. Contrast-limited adaptive histogram equalization: speed and effectiveness. In Proc. 1st Conference on Visualization in Biomedical Computing (eds Höhne, K. H. & Pizer, S. M.) 337–345 (IEEE, 1990).

  • Mirza, M. & Osindero, S. Conditional generative adversarial nets. Preprint at https://arxiv.org/abs/1411.1784 (2014).

  • Wang, H. et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods 16, 103–110 (2019).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Mi, L. et al. Learning guided electron microscopy with active acquisition. In Proc. Medical Image Computing and Computer Assisted Intervention (eds Martel, A. L. et al.) 77–87 (Springer, 2020).

  • Li, Y. et al. X-ray2em: uncertainty-aware cross-modality image reconstruction from X-ray to electron microscopy in connectomics. In Proc. 20th IEEE International Symposium on Biomedical Imaging (ISBI) 1–5 (IEEE, 2023); https://doi.org/10.1109/ISBI53787.2023.10230759 (2023).

  • Isola, P., Zhu, J. Y., Zhou, T. & Efros, A. A. Image-to-image translation with conditional adversarial networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 5967–5976 (IEEE, 2017).

  • Pavarino, E. C. et al. mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops. Front. Neural Circuits 17, 952921 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Bailoni, A. et al. Gasp, a generalized framework for agglomerative clustering of signed graphs and its application to instance segmentation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 11645–11655 (IEEE, 2022).

  • Berger, D. R., Seung, H. S. & Lichtman, J. W. Vast (volume annotation and segmentation tool): efficient manual and semi-automatic labeling of large 3D image stacks. Front. Neural Circuits 12, 88 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Lin, Z., Wei, D., Lichtman, J. & Pfister, H. Pytorch connectomics: a scalable and flexible segmentation framework for EM connectomics. Preprint at https://arxiv.org/abs/2112.05754 (2021).

  • Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. In Proc. Neural Information Processing Systems (NeurIPS) 8024–8035 (Curran Associates, Inc., 2019).

  • Silversmith, W. cc3d: connected components on multilabel 3D & 2D images. Zenodo https://doi.org/10.5281/zenodo.5719536 (2021).

  • Bishop, C. et al. CONFIRMS: a toolkit for scalable, black box connectome assessment and investigation. In Proc. 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) 2444–2450 (IEEE, 2021); https://doi.org/10.1109/EMBC46164.2021.9630109

  • Xenes, D. et al. Neuvue: a framework and workflows for high-throughput electron microscopy connectomics proofreading. Preprint at bioRxiv https://doi.org/10.1101/2022.07.18.500521 (2022).

  • Mulcahy, B. et al. A pipeline for volume electron microscopy of the Caenorhabditis elegans nervous system. Front. Neural Circuits 12, 94 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Weir, K., Dupre, C., van Giesen, L., Lee, A. S.-Y. & Bellono, N. W. A molecular filter for the cnidarian stinging response. eLife 9, e57578 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Baena, V., Schalek, R. L., Lichtman, J. W. & Terasaki, M. in Three-Dimensional Electron Microscopy, vol. 152 of Methods in Cell Biology (eds Müller-Reichert, T. & Pigino, G.) Ch. 3 (Academic, 2019).

  • Hider, R. et al. The brain observatory storage service and database (bossdb): a cloud-native approach for petascale neuroscience discovery. Front. Neuroinform. 16, 828787 (2022).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 



  • Source link