AI reveals the hidden layers of great paintings

Scientists say they have “spectacularly” improved the clarity of X-ray images of what lies beneath old paintings, helping art historians and restorers to understand and protect great works.

What’s more, they didn’t really expect the results of their work to be quite so good.

X-ray images are already a valuable tool in the examination and restoration of paintings, as they can reveal the underlying condition of the work and provide insights into artists’ techniques. They can also help experts to authenticate works.

But there is a problem. Interpreting X-rays can be difficult because the images capture everything – the visible top layer, what is underneath, the materials and support structures such as struts, and anything that is on the back. The experts have to pick apart this visual jumble to find what they need. 

So a team from University College in the UK and Duke University, US, set about using artificial intelligence to study one of the world’s great masterpieces, The Adoration of the Mystic Lamb, painted in 1432 by the brothers Hubert and Jan Van Eyck.

More commonly known as the Ghent Altarpiece, it’s a very large artwork made up of several panels, some of which are painted with images on both sides. 

The researchers, led by Zahra Sabetsarvestani, chose two of the two-sided panels and got to work.

They set up what is called a convolutional neural network – a computer program in which neurons connect in a way that resembles the functioning of an animal’s visual cortex – and for each of the two panels fed into it high resolution images of the front, the back, and a typical blended X-ray with both sides mixed up.

The computer gave them back two separated X-ray images for each panel, one of the front and one of the back. And that’s when the team got excited.

“The final results produced by this approach appear to present a near-perfect separation of the mixed X-ray images in both cases,” they write in Science Advances, describing the images as a spectacular improvement on the original X-ray photo.

The researchers had not expected such clarity and liken their efforts to a physicist conducting an experiment producing exceptional results not yet explained by theory.

“This approach works and seems to give remarkable results. Such an unexpected outcome calls for further investigation, possibly into the nature itself of the neural network application – an appealing prospect in its own right, since the singular effectiveness of deep neural networks is not well understood, and any peculiar behaviour, in any of its successful implementations, can possibly be used to unlock new insights.”

Next the team hopes to try their new approach on some other famous masterpieces where an artist is known to have reused a canvas, including Rembrandt’s Portrait of Frederick Rihel on Horseback, and where the artist has altered a composition, such as Titian’s The Death of Actaeon. Both paintings await Sabetsarvestani’s team in the British National Gallery.