AI-system promises better art reproductions – but not yet

A team from the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology in the US is developing a new, deep learning-assisted system to reproduce art with a 3D printer to make more accurate, convincing reproductions. 

The system combines a process known as halftoning, which uses little dots of ink, and a layering technique that has 10 different colours, rather than the usual cyan, magenta, yellow, and black of 2D printers. This keeps the ink from blotting, which happens when too much is deposited on the printing surface, and it allows the printer to produce a wider range of tones. 

The technique, combined with a “deep learning model to predict the optimal stack of different inks”, results in “unprecedented spectral accuracy”, the team writes in a new paper, being presented this month at a computer graphics conference in Tokyo. 

“If you just reproduce the colour of a painting as it looks in the gallery, it might look different in your home,” says Changil Kim, one of the paper’s authors. “Our system works under any lighting condition, which shows a far greater colour reproduction capability than almost any other previous work.”{%recommended 6743%}

The researchers they hope the project will eventually make art more available, since “our reliance on museums to exhibit original paintings and sculpture inherently limits access and leaves those precious originals vulnerable to deterioration and damage”.

“The value of fine art has rapidly increased in recent years, so there’s an increased tendency for it to be locked up in warehouses away from the public eye,” notes mechanical engineer Mike Foshey. 

“We’re building the technology to reverse this trend, and to create inexpensive and accurate reproductions that can be enjoyed by all.”

The developers concede that there is still work to be done on the system, which they named RePaint, to truly render a van Gogh simulacrum. For starters, images like Starry Night use a cobalt blue that the ink library isn’t able to “faithfully reproduce”. 

But paintings – particularly oil paintings – are three-dimensional works. The brush strokes leave ridges and bumps that can reflect light, throwing off the rendering. Right now, the printer reads glossy reflections as white highlights, but the team has plans to incorporate recognition of “the rich spatially-varying gloss and translucency found in many paintings”. The system will learn to use surface reflection, rather than less colour, to reproduce the gloss. 

One other issue? Those glorious Monet water lilies look more like postage stamps, since the system’s reproductions are only a few centimetres across. The engineers are hoping to bring down the costs and time printing to accommodate larger reproductions.