The Hand of the Artist: a research project on attribution

Factum Foundation has initiated a collaboration with a team from the Physics, Materials Science and Engineering, and Art History and Art Departments at Case Western Reserve University in Cleveland, Ohio. The goal is to apply machine learning methods to surface profilometry images of paintings in order to attribute stylistic components of brushwork in some of the later paintings of El Greco, with the aim of exploring differences between the hand of El Greco, those of his son Jorge Manuel and his workshop, and later conservation interventions.

The analysis involves AI / machine learning (ML) techniques using neural-net methods reacting to the three-dimensional characteristics of the surface relief. The aim is to identify different hands at work during the production (and restoration) of El Greco’s paintings. The individual technique of each artist leaves its trace on the painted surface. These traces can be clearly read if they are recorded with enough detail. With El Greco, there are often several hands working together to produce a single coherent and recognisable style.

A graphic from an article published in November in the journal Heritage Science was part of a paper in which multidisciplinary scholars from Case Western Reserve University used artificial intelligence algorithms to correctly attribute four nearly identical flower paintings by students at the Cleveland Institute of Art. The technique combined detailed 3-D mapping of the surface of the paintings with machine-learning analysis.Courtesy Case Western Reserve University

The team at Case Western Reserve have developed image analysis and machine learning techniques via a series of controlled experiments using images of lilies painted by artists from the Cleveland Institute of Art. In order to simulate workshop methods and produce sufficient training data, the team created virtual patches in each of the twelve paintings, three by each of four artists. After training and validation on two of the three paintings, testing on the remaining four paintings was carried out using machine learning methods. They were able to correctly attribute 98% of the patches, a significant result given that the patch size was a fraction
of a complete brush stroke. The methods made it possible to confidently attribute brushwork even when the virtual patches were as small as a few bristle widths.

The implications of this project for the art world are widereaching and game-changing. Connoisseurs will soon have a new tool to assist with thorny attribution questions for many paintings. This project is one of several under the umbrella of art.lab.cle, all of which are transdisciplinary collaborations using 21st-century technology which will have a major impact on our understanding of art.

The project comprises a close collaboration with the Fundación Casa Ducal de Medinaceli and the Hospital de Tavera in Toledo, and with The Auckland Project in Bishop Auckland.

The next steps

As of 2023, the team at Cleveland have been analysing the 3D data of El Greco’s paintings recorded by Factum Foundation in Toledo: The Portrait of Cardinal Tavera and The Baptism of Christ. In both cases, the application of pattern recognition software is starting to give promising results: a series of maps are being generated from the surface of paintings, differentiating possible “hands” based on the gesture that gave shape to the brushstrokes.

The work in progress was shared in a scientific paper written by the Cleveland team published in November 2021, titled “Discerning the painter’s hand: machine learning on surface topography” (click here to read on Springer Open).

On the same month, a team from Factum Foundation / ARCHiVe travelled to Cleveland to record another painting by El Greco, Christ on the Cross, at the Cleveland Museum of Art. The relief data was acquired with the Lucida 3D Scanner, while the colour was captured with panoramic composite photography. The obtained information has been provided to the working group at Case Western Reserve University to be analysed. It is also planned to record another version of the Crucifixion in the Spanish Gallery at Bishop Auckland, UK.

As a continuation to the work in Cleveland, the Lucida 3D Scanner was used to record the surface of a painting, located in Toledo’s Museo de Santa Cruz, which is a copy of Caravaggio’s Crucifixion of St. Andrew. The original Caravaggio, located in the Cleveland Museum of Art, was recently the subject of an in-depth technical study and restoration process by conservator Dean Yoder. An eventual comparison between the copy in Toledo and the original in Cleveland could provide a perfect case study for the AI study in progress.