Is the search for dark energy a dead end?

In 2006, I ventured to the Carnegie Institution in Washington DC. There, I had a long conversation with the astronomer Vera Rubin. Thirty-six years earlier, she was one of the first modern cosmologists to suggest that a huge part of the universe was missing. At the time, she had suggested it might take a decade to find this missing stuff, now best known as “dark matter”.

By 1990, two decades later, when dark matter was still missing, the English Astronomer Royal Martin Rees said it would turn up within a decade. In 1999 dark matter hadn’t made an appearance, but Rees was unbowed: he declared himself “optimistic” that, in five years’ time, he would be able to report what dark matter is. 

But by the time Rubin and I met in Washington, astronomers were all still empty-handed. What’s more, things had gotten worse: in 1997 astronomers had discovered “dark energy”, another missing component of the cosmos. Now a full 96% of the universe involved a form of matter and energy unknown to science. 

Has there been progress since then? Not really. In 2018, more than 20 years after we had to acknowledge our ignorance of the vast majority of the universe, we still haven’t identified what dark matter or dark energy might be. “I’m certainly ready for the great leap forward,” says Rocky Kolb, an astronomer based at the University of Chicago.

And there is not much hope of making such a leap either. In fact, some researchers are proposing that we might be living through our generation’s “ether moment”. For centuries, mainstream science believed that light propagated through a space filled with a mysterious stuff – the ether. But by the turn of the 20th century, the ether’s existence had been refuted. Could both dark matter and dark energy be similarly seductive illusions? 

The first hint of a dark side to the universe came in 1933, when the Swiss astronomer Fritz Zwicky noticed that the Coma galaxy cluster was spinning so fast that it should be falling apart due to centripetal forces. Zwicky suggested that they might be holding together because of the gravitational action of embedded massive particles that didn’t betray their presence by reflecting light. He called this hypothetical stuff “Dunkle Materie”: dark matter. 

The search didn’t really get off the ground though until the 1970s. It was the heyday of particle physics, so when Rubin noticed an anomaly in the Andromeda galaxy “that suggested the presence of a novel form of matter particle”, physicists were all ears. Rubin had measured the galaxy’s “rotation curve”, a graph of the speed at which its stars are orbiting the galaxy’s centre, plotted against their distance from the centre. The problem Rubin noticed was that, far out from the centre, the graph was flat.

Just as Pluto’s motion through space is slower than Earth’s, the outer stars should have a lower velocity than the inner stars. If their velocity is equally high, what’s to stop them flying off into space? Certainly not the gravitational pull of the galaxy’s visible matter, which is nowhere near strong enough. There must be a gravitational pull from dark matter holding these fast-spinning stars in place, Rubin said.

So what is this stuff? Physicists have come up with various candidates. The basic qualification is that it must have mass but no interaction with electromagnetic radiation: it must, in other words, have a gravitational pull without being detectable in any other way. 

Contenders have included mini black holes, neutrinos, hypothesised particles called axions, other things called Massive Compact Halo Objects (MACHOs) – and many more. There have been scores of experiments direct and indirect, looking for such candidates, and none have succeeded. 

Two elaborately designed snares for dark matter caused particularly bitter disappointment when they failed to capture anything. One involved smashing dense particles of matter like protons together in the Large Hadron Collider at CERN, the European particle physics lab in Geneva, and looking for dark matter candidates in the debris. None have been found. The other was the Large Underground Xenon or LUX experiment in the former Homestake gold mine in South Dakota, which hoped to detect “Weakly Interacting Massive Particles” or WIMPs, that are believed to gently rain down on our planet. 

LUX’s detector, buried deep underground to avoid noise, comprises a cylinder filled with cooled liquid xenon. The idea is that if a WIMP collides with a xenon atom, the atom emits a tiny flash of light that gets picked up by a bank of detectors encircling the tank. However, since we don’t know how often a WIMP would nudge a xenon atom, or how hard, the detector has had to search through a range of possible energy levels. In 2016, after a two-year, $US10 million-dollar search, its operators gave up the hunt, their snares conspicuously empty.

“I thought this would be the decade of the WIMPs,” Kolb says. “But we are 70% through the decade, with experiments and observations that in principle finally have the sensitivity and range to have discovered a WIMP. And we are empty-handed. All of the parameter space hasn’t been closed yet but it’s getting less likely that a WIMP is the answer.”

As a result, physicists are starting to look for new ideas about dark matter particles, or resurrecting ideas that were previously discarded. As Martin Sloth of the University of Southern Denmark has put it: “Everybody is signing up, thinking that they now have a chance”. 

But if all comers are now welcome in the new search for dark matter particles, alternative explanations for the anomalous observations in galaxy rotation curves and galaxy cluster spins are not. Take, for example, the work of Stacy McGaugh, a professor at Case Western Reserve University in Ohio and a former colleague of Vera Rubin. McGaugh has gone back to the drawing board. Rather than a problem of insufficient matter, perhaps gravity simply obeys a different rule over the huge, intergalactic distances involved? 

Has there been progress since the 1990s? Not really. Could dark matter and dark energy be seductive illusions?

The idea that gravity might not obey Newtonian (or Einsteinian) laws, so called Modified Newtonian Dynamics (MOND), was first put forward in the 1980s by Israeli physicist Mordehai Milgrom. “There are already observations explained by modified gravity that can’t be explained by dark matter,” McGaugh points out. One, he suggests, is the distribution of mass in dwarf spheroidal galaxies, which are small, dim galaxies dotted around the edges of the Milky Way and Andromeda. Because they contain comparatively little dust, their contents are relatively easy to scrutinise. 

But modifying gravity certainly doesn’t solve all the cosmological conundrums that astronomers want to resolve. Even McGaugh admits that for some observations, such as gravitational lensing where light from distant galaxies is bent by the pull of invisible matter, dark matter is a better explanation than modified gravity. 

But, rather than point scoring for the different theories, McGaugh says it’s time for a reckoning. If the dark matter search has turned up nothing since the 1930s, who gets to decide how long we keep looking? “Every five years for the past 25 years I’ve heard a talk by some impressive person in which it was confidently asserted that in five years we would know what the dark matter was,” McGaugh says. “It was always an ‘odds on slam dunk’ – and always an overly optimistic assessment.” 

The problem is, we can’t ever rule out dark matter’s existence just because we haven’t found it. “If we get tired of looking for WIMPs, maybe it is axions. When we tire of those, we’re free to make something else up, ad infinitum,” McGaugh says. He believes this transgresses the very idea of a scientific endeavour. “Is that science? Popper would say ‘no’.” 

The iconic 20th century science philosopher, Karl Popper, held that if there is no piece of evidence that – if found – could show unequivocally that your theory is false, you’re not doing science. The theory of dark matter is unfalsifiable, McGaugh says.

But according to Michela Massimi, a philosopher of science based at the University of Edinburgh, that doesn’t disqualify its merit. Invoking Popper’s falsifiability is inadequate for capturing cosmology’s issues, she says. 

Indeed, while most cosmologists today still hope to find evidence for dark matter through Popperian experiments, the evidence actually accrues through a variety of channels: from the cosmic microwave background radiation (an echo of the big bang) to the motion of galaxy clusters, among others. “Until and unless a rival dark-matter-free model can be found that proves as successful at explaining all these phenomena, the hypothesis of cold dark matter is bound to remain live, even in the absence of direct detection evidence,” Massimi says. 

Massimi is sympathetic to those trying to work on rival ideas, though. She describes McGaugh’s work as important and regrets that his ideas, and those of others, don’t receive the attention they should. She remains optimistic, however. “I think things are slowly changing,” she says.

Colin Rourke, a mathematician at the University of Warwick, doesn’t share Massimi’s optimism. Like McGaugh he has also developed a mathematical model of galaxies that does away with the need for dark matter. Instead, he suggests that a rotating, superheavy black hole at the centre of galaxies is enough to create the flat rotation curve. 

In his mathematical scheme, which builds on the early 20th century ideas of Austrian Ernst Mach (of sound speed fame), the rotating mass creates a distortion in space-time that would alter the apparent velocity of the stars around it. Because of the distortion (an effect known as frame-dragging), they look from the outside like they are being pulled around more quickly, which creates the illusion of dark matter’s existence. “It’s just something in the geometry, something warping in space-time,” he says. 

Though he has many mathematical admirers and collaborators, Rourke has had no success trying to get his idea taken seriously by cosmologists, or published in any of the mainstream cosmology journals. “It’s been like dropping it down a deep well. I’m still waiting to hear something,” he says.

For Donald Saari, a mathematics professor from the University of California, Irvine, the answer to missing matter lies in the mathematics of many-body problems or how forces interact between multiple objects. He says he has created simulations that show the theoretical rotation curves of galaxies – the root of Rubin’s observation of dark matter – are the wrong shape. 

Simulations show the theoretical rotation curves of galaxies – the source of theories of dark matter – are the wrong shape.

That is because they rely on solving the two-body problem to give the theoretical rotation curve, approximating the motion of any particular star by assuming it is pulled by the galaxy as a whole, rather than each of the other objects at once. 

Saari has worked out the effects of having billions of massive objects simultaneously moving and pulling on one another. The result, he claims, gives precisely the rotation curve that’s observed. “I’ve had it reviewed by astronomers, and they have not found any errors in what I’ve done. They just don’t like the conclusion.” He published his analysis in the Astronomical Journal three years ago, after a four-year review process, but it hasn’t changed anything. “It’s had no impact that I’m aware of,” he says. 

Saari is relatively sanguine about being ignored. McGaugh is less happy, and expends a significant amount of energy engaging with the cosmology community, offering new tests for modified gravity, exploring where and how astronomers might test whether it, or dark matter, is the more accurate idea. 

Despite all the effort, it is unlikely to make any difference. Modified gravity researchers have long battled mainstream cosmologists over the interpretations of observations, drawing conflicting conclusions from the same evidence. 

Even if we were to solve the dark matter problem, we’re still left with another huge hole in our picture of the cosmos – though this too might be an illusion induced by a faulty theory. This hole is occupied by dark energy, and it accounts for 70% of the total mass and energy in the universe. That’s almost three times the size of the dark matter hole, which accounts for about 27%. 

As with dark matter, dark energy’s existence was initially inferred from astronomical observations – this time from light emanating from exploding stars known as supernovae. Analysing how the wavelengths of the light had stretched as it travelled through space to our telescopes suggested not only that space was expanding, but also that the expansion was speeding up. 

This Nobel prize-winning discovery by the teams of Saul Perlmutter at Lawrence Berkeley National Laboratory, Brian P. Schmidt of the Australian National University and Adam Riess of Johns Hopkins University was a complete surprise in 1997. Our understanding was that in the aftermath of the big bang, gravity should have put a brake on the exploding universe. 

We assume that there must be an energy source for this acceleration: hence the hypothesis of dark energy. But so far all attempts to work out what it is, and where it comes from, have failed. There are also those who think it might be a mathematically induced illusion.

One possibility is that we may have made some false assumptions: essentially, the universe is more complex than we might have hoped. Carl Gibson of the University of California San Diego, for instance, reckons we can’t do reliable cosmology without taking into account turbulence and other complexities of fluid dynamics that might have arisen in the high-energy environment of the big bang. 

It’s not just about turbulence, though. In order to have a manageable theory, we assume that the universe is isotropic – the same in every direction – and homogeneous, with no areas of the cosmos that have special, peculiar characteristics. Those assumptions make the equations easier to solve, but they may be oversimplifying things. Kolb has been suggesting for more than a decade that we need a more complex, nuanced theory that can work without these assumptions. The trouble is: the maths is prohibitively difficult and, according to some, the effort might be a waste of time. Martin Kunz of the University of Geneva, for instance, has published work suggesting the inhomogeneities would have to be unrealistically huge to account for the dark energy. Kolb isn’t convinced. Rumours of the idea’s death are “exaggerated”, he reckons. 

Alternatively, might there be problems with our supernova observations? The conclusions about dark energy rely on all supernovae of the same type emitting their light in exactly the same way. That’s why the ones used for the dark energy calculation are known in the community as “standard candles”.

The conclusions about dark energy rely on supernovae all emitting their light in the same way. Maybe that’s a dangerous assumption?

But maybe that’s another dangerous assumption. We’re currently puzzled by a set of observations of supernova iPTF14hls, for example. Instead of dimming continuously after its initial explosion, it has brightened on occasion, maintaining this variable luminosity for years. Though this is not the same type of supernova as used in dark energy measurements, it does raise the question of whether we understand supernovae as well as we think. 

But this avenue of inquiry is still a long shot – like all the others, it seems. Take the idea that the solution to dark energy might come with a re-examination of Einstein’s cosmological constant. Einstein introduced the term as a fudge factor: while his equations showed the universe was expanding, he “knew” the universe to be static. He later referred to this as his biggest blunder and removed it from the equations. But physicists have essentially re-inserted it because since the discovery of dark energy, we need a term that will push hard on space and time, causing the accelerating expansion we observe. 

Not everyone is convinced, though, that this simple re-insertion is the right way to account for the observations. After all, the cosmological constant term makes the equations work but doesn’t actually give us any clue about the source of the dark energy. Maybe there are better fudges? “To me, looking for flaws in the cosmological constant is the thing to do,” Kolb says. 

It’s worth noting that the universe’s rate of expansion is already the subject of controversy. The value obtained by using stellar measurements such as supernova standard candles is different to the value obtained using the record of the universe’s first moments preserved in the cosmic microwave background (CMB). (Here the expansion rate of the universe is inferred from gravitational lensing effects on the photons of the CMB.) 

Some of the researchers involved think we might be able to explain the discrepancy with a hitherto unknown particle called a “sterile neutrino” – which could also be the source of dark matter. “That remains one of the stronger possibilities,” says Riess of Johns Hopkins University.

His group’s latest analysis of the tension, which has been accepted into the Astrophysical Journal, suggests the discrepancy between supernovae and CMB data is not going away with better measurements.

So, as things stand, there is no resolution in sight to the dark energy problem. There are myriad further data-gathering plans, such as the Australian Dark Energy Survey (OzDES), led by Chris Lidman of the Australian Astronomical Observatory in North Ryde, New South Wales, which is measuring the output of more than 3,000 new supernovae to give us more information about the universe’s expansion. 

But this will all take years, maybe several decades, to give us a firm conclusion. In the meantime, there’s a familiar non-committal refrain floating through the ether. “It’s just a hard problem,” Riess says. “I am optimistic we will learn more about dark energy in the coming decade.” 

Likewise for dark matter. It seems we will just to have to wait for something to change. At the edge of what’s known, Kolb points out, science is not a slow steady march of progress: it’s leaps, bounds and occasional missteps. “I’m willing to be patient for a while longer,” he says.