Can you tell whether the title of a scientific article was written by a human or an AI? Because half the time, not even doctors can spot the fake science headline, according to a paper published in BMJ.
Go on, give it a try!
In a study worthy of the silly season, researchers used an AI to generate research paper titles and tested whether readers could tell if they were fake.
They took the top titles from 10 years of BMJs Christmas edition – which are often quirkier than normal – to teach an AI to write its own titles. These were then rated by a random sample of doctors from multiple disciplines and countries.
They found that AI-generated titles were rated at least as enjoyable (69%) compared to real titles (64%), although the real titles were rated as more plausible (73%) than AI titles (48%).
They also found that the AI titles were deemed less scientific if generated at random, but this became less apparent when the titles were then curated by a human.
The authors say that this shows how the best results come from an AI and a human working together, where the AI can compensate for human oversights but humans can make the final call.
The two AI-generated titles deemed the most plausible were: “The clinical effectiveness of lollipops as a treatment for sore throats” and “The effects of free gourmet coffee on emergency department waiting times: an observational study.”
The silliest title generated by the AI was: “Superglue your nipples together and see if it helps you to stop agonising about erectile dysfunction at work.” The authors note that this demonstrates the AI doesn’t know how to be polite, which limits its real-world application without human help.
Deborah Devis is a science journalist at Cosmos. She has a Bachelor of Liberal Arts and Science (Honours) in biology and philosophy from the University of Sydney, and a PhD in plant molecular genetics from the University of Adelaide.
Read science facts, not fiction...
There’s never been a more important time to explain the facts, cherish evidence-based knowledge and to showcase the latest scientific, technological and engineering breakthroughs. Cosmos is published by The Royal Institution of Australia, a charity dedicated to connecting people with the world of science. Financial contributions, however big or small, help us provide access to trusted science information at a time when the world needs it most. Please support us by making a donation or purchasing a subscription today.