Beyond relighting: RTI for clustering fragmented heritage textiles using deep learning

Machine Learning


  • Margariti, C., Lukesova, H. & Gomes, F. B. Advanced analytical techniques for heritage textiles. Herit. Sci. 12, 388 (2024).

    Article 

    Google Scholar 

  • Yílmaz, S. & Nabiyev, V. V. Comprehensive survey of the solving puzzle problems. Comp. Sci. Rev. 50, 100586 (2023).

    Article 

    Google Scholar 

  • Nedumpillil, N. N., Sankaran, A., Jose, S., George, S. & Thomas, S.Destructive Analytical Techniques for the Analysis of Historic Textiles, vol. 2, chap. 11, 215–234 (Wiley, 2022).

  • Margariti, C., Sava, G., Sava, T., Boudin, M. & Nosch, M.-L. Radiocarbon dating of archaeological textiles at different states of preservation. Herit. Sci. 11, 44 (2023).

    Article 
    CAS 

    Google Scholar 

  • Magdy, M. Analytical techniques for the preservation of cultural heritage: Frontiers in knowledge and application. Crit. Rev. Anal. Chem. 52, 1171–1196 (2022).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Pozzi, F. & Stephens, C. H. Advances in analytical methods for cultural heritage. Appl. Sci. 14, 7587 (2024).

    Article 
    CAS 

    Google Scholar 

  • Gigilashvili, D., Lukesova, H. & Hardeberg, J. Y. Criteria for matching fragmented archaeological textiles: a survey. Archaeol. Text. Rev. 66, 64–75 (2024).

    Google Scholar 

  • Gigilashvili, D. et al. Toward solving a puzzle of fragmented archeological textiles. J. Imaging Sci. Technol. 68, 1–16 (2024).

    Article 

    Google Scholar 

  • Gorla, G., Nielsen, F., Montague, P. B., Kösegi, N. & Amigo, J. M. Challenges and spectra interpretability in textile sorting: NIR hyperspectral images and chemometrics. Spectrochimica Acta Part A: Mol. Biomolecular Spectrosc. 344, 126665 (2025).

    Article 

    Google Scholar 

  • Dian, R., Li, S., Sun, B. & Guo, A. Recent advances and new guidelines on hyperspectral and multispectral image fusion. Inf. Fusion 69, 40–51 (2021).

    Article 

    Google Scholar 

  • Frank, E. Lights, camera, archaeology: documenting archaeological textile impressions with reflectance transformation imaging (RTI). Text. Specialty Group Postprints 25, 11–42 (2015).

    Google Scholar 

  • Mytum, H. & Peterson, J. R. The application of reflectance transformation imaging (RTI) in historical archaeology. Historical Archaeol. 52, 489–503 (2018).

    Article 

    Google Scholar 

  • Ciortan, I. et al. A practical reflectance transformation imaging pipeline for surface characterization in cultural heritage. In Proceedings of the 14th Eurographics Workshop on Graphics and Cultural Heritage, GCH ’16, 127-136 (Eurographics Association, 2016).

  • Miles, J., Pitts, M., Pagi, H. & Earl, G. New applications of photogrammetry and reflectance transformation imaging to an Easter Island statue. Antiquity 88, 596–605 (2014).

    Article 

    Google Scholar 

  • Siatou, A. et al. New methodological approaches in Reflectance Transformation Imaging applications for conservation documentation of cultural heritage metal objects. J. Cult. Herit. 58, 274–283 (2022).

    Article 

    Google Scholar 

  • Ono, S., Matsuda, Y. & Mizuochi, T. Development of a multispectral RTI system to evaluate varnish cleaning. In Bridgeland, J. (ed.) ICOM-CC 18th Triennial Conference Preprints, Copenhagen, 4–8 September 2017, art. 0205 (International Council of Museums, 2017).

  • Min, J. et al. Reflectance transformation imaging for documenting changes through treatment of Joseon dynasty coins. Herit. Sci. 9, 105 (2021).

    Article 
    CAS 

    Google Scholar 

  • Boute, R. et al. Revisiting Reflectance Transformation Imaging (RTI): a tool for monitoring and evaluating conservation treatments. In IOP conference series: materials science and engineering, vol. 364 (IOP, 2018).

  • Yahaghi, E., García, J. A. M., Movafeghi, A. & Mirzapour, M. Improving X-ray images of historically significant textiles. J. Cult. Herit. 66, 415–425 (2024).

    Article 

    Google Scholar 

  • Morigi, M. & d’Errico, V. Application of X-ray computed tomography to cultural heritage diagnostics. Appl. Phys. A 100, 653–661 (2010).

    Article 
    CAS 

    Google Scholar 

  • Lanteri, L. & Pelosi, C. 2D and 3D ultraviolet fluorescence applications on cultural heritage paintings and objects through a low-cost approach for diagnostics and documentation. In Optics for Arts, Architecture, and Archaeology VIII, vol. 11784, 156–165 (SPIE, 2021).

  • Picollo, M., Cucci, C., Casini, A. & Stefani, L. Hyper-spectral imaging technique in the cultural heritage field: new possible scenarios. Sensors 20, 2843 (2020).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Fischer, C. & Kakoulli, I. Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications. Stud. Conserv. 51, 3–16 (2006).

    Article 

    Google Scholar 

  • Daffara, C., Pampaloni, E., Pezzati, L., Barucci, M. & Fontana, R. Scanning multispectral IR Reflectography SMIRR: an advanced tool for art diagnostics. Acc. Chem. Res. 43, 847–856 (2010).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Schreiner, M., Melcher, M. & Uhlir, K. Scanning electron microscopy and energy dispersive analysis: applications in the field of cultural heritage. Anal. Bioanal. Chem. 387, 737–747 (2007).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Romani, A., Clementi, C., Miliani, C. & Favaro, G. Fluorescence spectroscopy: a powerful technique for the noninvasive characterization of artwork. Acc. Chem. Res. 43, 837–846 (2010).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Jackson, J. B. et al. A Survey of terahertz applications in cultural heritage conservation science. IEEE Trans. Terahertz Sci. Technol. 1, 220–231 (2011).

    Article 
    CAS 

    Google Scholar 

  • Malzbender, T., Gelb, D. & Wolters, H. Polynomial texture maps. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 519–528 (CGIT, 2001).

  • Pitard, G. et al. Discrete modal decomposition: a new approach for the reflectance modeling and rendering of real surfaces. Mach. Vis. Appl. 28, 607–621 (2017).

    Article 

    Google Scholar 

  • Gautron, P., Krivanek, J., Pattanaik, S. & Bouatouch, K. A novel hemispherical basis for accurate and efficient rendering. In Proceedings of the Fifteenth Eurographics Conference on Rendering Techniques, EGSR’04, 321-330 (Eurographics Association, 2004).

  • Dulecha, T. G., Fanni, F. A., Ponchio, F., Pellacini, F. & Giachetti, A. Neural reflectance transformation imaging. Vis. Comp. 36, 2161–2174 (2020).

    Article 

    Google Scholar 

  • Hess, M., MacDonald, L. W. & Valach, J. Application of multi-modal 2D and 3D imaging and analytical techniques to document and examine coins on the example of two Roman silver denarii. Herit. Sci. 6, 5 (2018).

    Article 

    Google Scholar 

  • Saha, S., Siatou, A., Mansouri, A. & Sitnik, R. Supervised segmentation of RTI appearance attributes for change detection on cultural heritage surfaces. Herit. Sci. 10, 173 (2022).

    Article 
    CAS 

    Google Scholar 

  • Siatou, A. et al. A Methodological Approach for Multi-Temporal Tracking of Silver Tarnishing. In Proceedings of the 4th ACM International workshop on Structuring and Understanding of Multimedia heritAge Contents, 5–13 (ACM, 2022).

  • Siatou, A. Detection and characterization of appearance change by imaging and artificial vision for the conservation documentation of cultural heritage metal objects. Ph.D. thesis, Université Bourgogne Franche-Comté https://theses.hal.science/tel-04623231 (2023).

  • Cantó, A., Lerma, J. L., Martínez Valle, R. & Villaverde, V. Multi-light photogrammetric survey applied to the complex documentation of engravings in Palaeolithic rock art: the Cova de les Meravelles (Gandia, Valencia, Spain). Herit. Sci. 10, 169 (2022).

    Article 

    Google Scholar 

  • Le Goïc, G. et al. Reflectance Transformation Imaging for the quantitative characterization of experimental fracture surfaces of bonded assemblies. Eng. Fail. Anal. 140, 106582 (2022).

    Article 

    Google Scholar 

  • Nurit, M. et al. HD-RTI: an adaptive multi-light imaging approach for the quality assessment of manufactured surfaces. Comp. Ind. 132, 103500 (2021).

    Article 

    Google Scholar 

  • Zendagui, A. et al. Quality assessment of dynamic virtual relighting from rti data: application to the inspection of engineering surfaces. In Fifteenth International Conference on Quality Control by Artificial Vision, vol. 11794 Article 117940E. (SPIE, 2021).

  • Righetto, L. et al. Ancient coins’ surface inspection with web-based neural RTI visualization. In Optics for Arts, Architecture, and Archaeology (O3A) IX, vol. 12620 Article 126200D (SPIE, 2023).

  • Righetto, L. et al. Effective Interactive Visualization of Neural Relightable Images in a Web-based Multi-layered Framework. In GCH 2023-Eurographics Workshop on Graphics and Cultural Heritage, 57–66 (GCH, 2023).

  • Coules, H., Orrock, P. & Seow, C. E. Reflectance Transformation Imaging as a tool for engineering failure analysis. Eng. Fail. Anal. 105, 1006–1017 (2019).

    Article 

    Google Scholar 

  • Ding, Q.-K. & Liang, H.-E. Digital restoration and reconstruction of heritage clothing: a review. Herit. Sci. 12, 225 (2024).

    Article 

    Google Scholar 

  • Jahnke, C. & Ling Huang, A.Textiles and the Medieval Economy: Production, Trade, and Consumption of Textiles, 8th-16th Centuries (Oxbow Books, 2014).

  • Schoeser, M. World Textiles (Thames & Hudson, 2022).

  • Eriksen, Å. The techniques of samitum. Based on a reconstruction of a silk from the Oseberg burial. In The Social Fabric: Deep Local to Pan Global; Proceedings of the Textile Society of America 16th Biennial Symposium. https://digitalcommons.unl.edu/tsaconf/1083 (Vancouver, BC, Canada, 2018).

  • Bonde, N. & Stylegar, F.-A. Between Sutton Hoo and Oseberg–dendrochronology and the origins of the ship burial tradition. Dan. J. Archaeol. 5, 19–33 (2016).

    Article 

    Google Scholar 

  • Vedeler, M.The Oseberg Tapestries (Scandinavian Academic Press, 2019).

  • Righetto, L. et al. Efficient and user-friendly visualization of neural relightable images for cultural heritage applications. ACM J. Comput. Cult. Herit. 17, 54 (2024).

    Google Scholar 

  • Gigilashvili, D., Lukesova, H., Gulbrandsen, C. F., Harijan, A. & Hardeberg, J. Y. Computational techniques for virtual reconstruction of fragmented archaeological textiles. Herit. Sci. 11, 259 (2023).

    Article 

    Google Scholar 

  • Lester-Makin, A. & Owen-Crocker, G. R. (eds.) Textiles of the Viking North Atlantic: Analysis, Interpretation, Re-creation. No. 7 in Medieval and Renaissance Clothing and Textiles (Boydell Press, 2024).

  • Ngo, V. M., Helmer, S., Le-Khac, N.-A. & Kechadi, M.-T. Structural textile pattern recognition and processing based on hypergraphs. Inf. Retr. J. 24, 137–173 (2021).

    Article 

    Google Scholar 

  • Abitbol, R., Shimshoni, I. & Ben-Dov, J. Machine Learning Based Assembly of Fragments of Ancient Papyrus. J. Comput. Cultural Herit. 14, 33 (2021).

    Article 

    Google Scholar 

  • Villegas-Suarez, A. M., Lopez, C. & Sipiran, I. MatchMakerNet: Enabling Fragment Matching for Cultural Heritage Analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 1632–1641 (Paris, France, 2023).

  • Ren, P. et al. A survey of deep active learning. ACM Comput. Surv. 54, 180 (2021).

    Google Scholar 

  • Amin, S. U., Hussain, A., Kim, B. & Seo, S. Deep learning based active learning technique for data annotation and improve the overall performance of classification models. Expert Syst. Appl. 228, 120391 (2023).

    Article 

    Google Scholar 

  • Fiorucci, M. et al. Machine learning for cultural heritage: a survey. Pattern Recognit. Lett. 133, 102–108 (2020).

    Article 

    Google Scholar 

  • Tasci, B. et al. InCR: Inception and concatenation residual block-based deep learning network for damaged building detection using remote sensing images. Int. J. Appl. Earth Observ. Geoinf. 123, 103483 (2023).

    Google Scholar 

  • Wen, L., Li, X. & Gao, L. A transfer convolutional neural network for fault diagnosis based on ResNet-50. Neural Comput. Appl. 32, 6111–6124 (2020).

    Article 

    Google Scholar 

  • Cendre, R., Mansouri, A., Perrot, J.-L., Cinotti, E. & Marzani, F. Classification of lentigo maligna at patient-level by means of reflectance confocal microscopy data. Appl. Sci. 10, 2830 (2020).

    Article 
    CAS 

    Google Scholar 

  • Ul Amin, S., Sibtain Abbas, M., Kim, B., Jung, Y. & Seo, S. Enhanced anomaly detection in pandemic surveillance videos: an attention approach with efficientNet-B0 and CBAM Integration. IEEE Access 12, 162697–162712 (2024).

    Article 

    Google Scholar 

  • Sultani, W., Chen, C. & Shah, M. Real-World Anomaly Detection in Surveillance Videos. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6479–6488 (IEEE, 2018).

  • Ul Amin, S. et al. EADN: an efficient deep learning model for anomaly detection in videos. Mathematics 10, 1555 (2022).

    Article 

    Google Scholar 

  • Ul Amin, S., Kim, B., Jung, Y., Seo, S. & Park, S. Video anomaly detection utilizing efficient spatiotemporal feature fusion with 3d convolutions and long short-term memory modules. Adv. Intell. Syst. 6, 2300706 (2024).

    Article 

    Google Scholar 

  • Weinmann, M. Visual Features – From Early Concepts to Modern Computer Vision. In Farinella, G. M., Battiato, S. & Cipolla, R. (eds.) Advanced Topics in Computer Vision, Advances in Computer Vision and Pattern Recognition, 1–34 (Springer, 2013).

  • Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. How transferable are features in deep neural networks? In Proceedings of the 28th International Conference on Neural Information Processing Systems – Volume 2, NIPS’14, 3320-3328 https://doi.org/10.5555/2969033.2969197 (MIT Press, 2014).

  • He, K., Zhang, X., Ren, S. & Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 770–778 (IEEE, 2016).

  • Dosovitskiy, A. et al. An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=YicbFdNTTy (2021).

  • Alrahhal, M. & Supreethi, K. Integrating machine learning algorithms for robust content-based image retrieval. Int. J. Inf. Technol. 16, 5005–5021 (2024).

    Google Scholar 

  • Zheng, Q., Tian, X., Yang, M. & Wang, H. Differential learning: A powerful tool for interactive content-based image retrieval. Eng. Lett. 27, 202–215 (2019).

    Google Scholar 

  • Simonyan, K. & Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations, ICLR (ICLR, 2015).

  • Sezavar, A., Farsi, H. & Mohamadzadeh, S. Content-based image retrieval by combining convolutional neural networks and sparse representation. Multimed. Tools Appl. 78, 20895–20912 (2019).

    Article 

    Google Scholar 

  • Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems25 (NIPS, 2012).

  • Huang, H., Wang, C., Wei, X. & Zhou, Y. Deep image clustering: a survey. Neurocomputing 599, 128101 (2024).

    Article 

    Google Scholar 

  • Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C. & Dosovitskiy, A. Do vision transformers see like convolutional neural networks? Adv. neural Inf. Process. Syst. 34, 12116–12128 (2021).

    Google Scholar 

  • Khawaja, M. A., George, S., Marzani, F., Hardeberg, J. Y. & Mansouri, A. An interactive method for adaptive acquisition in reflectance transformation imaging for cultural heritage. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 1698–1706 (IEEE, 2023).

  • Zhang, M. & S Drew, M. Efficient robust image interpolation and surface properties using polynomial texture mapping. EURASIP J. Image Video Process. 2014, 25 (2014).

    Article 

    Google Scholar 

  • McGuigan, M. & Christmas, J. Automating RTI: Automatic light direction detection and correcting non-uniform lighting for more accurate surface normals. Computer Vis. Image Underst. 192, 102880 (2020).

    Article 

    Google Scholar 

  • Manfredi, M. et al. A new quantitative method for the non-invasive documentation of morphological damage in paintings using RTI surface normals. Sensors 14, 12271–12284 (2014).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Ponchio, F., Corsini, M. & Scopigno, R. A compact representation of relightable images for the web. In Proceedings of the 23rd International ACM Conference on 3D Web Technology, 1 (ACM, 2018).

  • Khan, A., Sohail, A., Zahoora, U. & Qureshi, A. S. A survey of the recent architectures of deep convolutional neural networks. Artif. Intell. Rev. 53, 5455–5516 (2020).

    Article 

    Google Scholar 

  • Tan, C. et al. A survey on deep transfer learning. In Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Part III, vol. 11141, 270–279 (Springer, 2018).

  • Deng, J. et al. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 248–255 (IEEE, 2009).

  • Greenacre, M. et al. Principal component analysis. Nat. Rev. Methods Prim. 2, 100 (2022).

    Article 
    CAS 

    Google Scholar 

  • Saeed, N., Nam, H., Haq, M. I. U. & Muhammad Saqib, D. B. A survey on multidimensional scaling. ACM Comput. Surv. 51, 47 (2018).

    Google Scholar 

  • Borg, I. & Groenen, P. J.Modern multidimensional scaling: Theory and applications (Springer Science & Business Media, 2007).

  • Maaten, L. vd & Hinton, G. Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605 (2008).

    Google Scholar 

  • Belkina, A. C. et al. Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets. Nat. Commun. 10, 5415 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • McInnes, L., Healy, J. & Melville, J. UMAP: uniform manifold approximation and projection for dimension reduction. J Open Source Soft 3, 861 (2018).

    Article 

    Google Scholar 

  • Mittal, M. et al. Dimensionality Reduction Using UMAP and TSNE Technique. In Second International Conference on Advances in Information Technology (ICAIT) (IEEE, 2024).

  • Saxena, A. et al. A review of clustering techniques and developments. Neurocomputing 267, 664–681 (2017).

    Article 

    Google Scholar 

  • Aggarwal, C. C., Hinneburg, A. & Keim, D. A. On the Surprising Behavior of Distance Metrics in High Dimensional Space. In Database Theory — ICDT 2001, vol. 1973 of Lecture Notes in Computer Science, 420–434 (Springer, 2001).

  • Ding, L., Li, C., Jin, D. & Ding, S. Survey of spectral clustering based on graph theory. Pattern Recognit. 151, 110366 (2024).

    Article 

    Google Scholar 

  • Kruskal, J. B. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika 29, 1–27 (1964).

    Article 

    Google Scholar 

  • Venna, J. & Kaski, S. Neighborhood Preservation in Nonlinear Projection Methods: An Experimental Study. In Dorffner, G., Bischof, H. & Hornik, K. (eds.) International Conference on Artificial Neural Networks (ICANN), Lecture Notes in Computer Science, 485–491 (Springer, 2001).

  • Kobak, D. & Berens, P. The art of using t-sne for single-cell transcriptomics. Nat. Commun. 10, 5416 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 



  • Source link