Imagine: you are the victim of a vicious assault and the trauma of the event leaves you unable to describe the face of the thug who hit you.
At the hospital, doctors connect you to an electrode cap, honing in on two small areas of your inferior temporal cortex (ITC) – just 205 neurons in total.
Five minutes later, a connected printer spits out a near photo-perfect image of your assailant. Evidence in hand, the police head off on the hunt …
This might seem like something taken from a William Gibson novel, but new research into the way in which brains recognise and record faces suggests that this type of forensic retrieval might be just around the corner.
Using macaque monkeys, researchers from the California Institute of Technology have convincingly demonstrated how brains respond to faces, and have also shown that it is possible to extract a two-dimensional image of a recalled face with a startling degree of accuracy.
What’s more, they have also revealed that the entire process, from memory creation to image readout, is surprisingly simple.
In a report published in the journal Cell, Doris Tsao and colleague Steve Le Chang report that by using functional magnetic resonance imaging (fMRI) they have successfully identified the areas of the brain that respond primarily to faces. They turn out to be six small zones of the ITC dubbed, prosaically enough, face patches.
The patches are rich in neurons that fire much more strongly when their owner – a macaque, in this case – is looking at a face than at anything else.
Early thinking suggested that each neuron in the face patch complex might be responsible for encoding the memory of a specific face, but this idea threw up obvious problems.
“You could potentially recognize six billion people, but you don’t have six billion face cells in the IT cortex,” says Tsao. “There had to be some other solution.”
Tsao and Le Chang worked out instead that each neuron encodes a specific axis within a multidimensional space.
To deduce this, they created a 50-dimensional matrix capable of representing any possible face. Half of the dimensions were assigned to represent shape – lip thickness, for instance, or the distance between the eyes – while the other half were given over to non-shape facial components, such as skin colour and complexion.
Wiring up the monkeys and showing them photographic portraits, the researchers found that each face patch neuron fired in relation to a given axis within the 50-dimensional matrix.
To confirm this, the pair worked out an algorithm that decoded the neural responses – in effect, translating each firing into a particular piece of visual data.
The macaques were shown photos and their face patch neural responses fed through the algorithm and then into a printer. The resulting shots were almost identical to the originals.
Tsao and Le Chang found that it wasn’t necessary to monitor all six patches to produce a near-perfect likeness: it could be done by reading just two, a total of only 205 neurons.
“People always say a picture is worth a thousand words,” Tsao says. “But I like to say that a picture of a face is worth about 200 neurons.”
To ensure that they were reading the data correctly, the scientists followed up by showing the macaques a wide range of highly differing faces and monitoring how specific neurons behaved each time. It turned out they worked exactly the same, every time, regardless of the visage in front of them.
“This was completely shocking to us,” notes Tsao.
“We had always thought face cells were more complex. But it turns out each one is just measuring distance along a single axis of face space, and is blind to other features.”
She adds that the findings had potential uses in neuroscience, artificial intelligence – and police investigations: “One can imagine applications in forensics where one could reconstruct the face of a criminal by analysing a witness’s brain activity.”