Much of the activity about academic journal articles posted on Twitter is “mechanical and devoid of original thought”, according to a new study that calls into question the value of some alternative metrics used to evaluate research.
The authors of the study, which analysed 8,000 tweets about 4,000 research papers in the field of dentistry, say that “simplistic and na?ve” use of social media data risks damaging science. The paper, titled “The unbearable emptiness of tweeting – about journal articles”, is published in?PlosOne.
An altmetrics provider said in response that tweet numbers can add valuable insights, but cannot tell the “whole story” if used in isolation.
In recent years, academics have been encouraged to use social media sites to disseminate their research findings and keep abreast of developments in their field. Counting the number of accounts that tweet a paper is just one of a series of alternative metrics that some companies, such as Altmetric, use to offer insights on scholarly literature.
But so far little research has looked at whether tweet counts can act as a measure of engagement with scientific literature. So a group of researchers led by Nicolas Robinson-Garcia, a postdoctoral researcher in the School of Public Policy at the Georgia Institute of Technology, looked at the content of 8,200 tweets from 2,200 US-based Twitter accounts on the subject of 4,350 dental research papers.
They found that the most tweeted paper, about acetaminophen (paracetamol), accumulated 264 tweets, putting it in the top 5 per cent of research outputs scored by Altmetric. The researchers found that almost 75 per cent of the tweets came from the same account, which linked to the paper 65 times in a repeated tweet that said just “paracetamol research” and a further 33 times in another tweet that offered just repetitions of the same wording referring to the safety of acetaminophen in pregnancy.
A second account tweeted the paper 58 times and both accounts retweeted each other on occasion. If the tweet count ignored all but one of the tweets from these two accounts it would sit at only 15, write the researchers. A similar thing was seen with the second most tweeted paper, they add.
Across the board, Dr Robinson-Garcia and colleagues found “obsessive single issue tweeting, duplicate tweeting from many accounts presumably under centralised professional management, bots, and much presumably human tweeting duplicative, almost entirely mechanical and devoid of original thought”.
Less than 10 per cent of the tweets about the papers curated or informed others of the literature. “Finding these accounts or seeing their influence on Twitter data about dental papers would be like looking for a needle in a haystack,” the authors say.
“Simplistic and na?ve use of social media data risks damaging the scientific enterprise, misleading both authors and consumers of scientific literature,” they add.
Catherine Williams, chief marketing officer at Altmetric, said: “We would completely agree that relying on the numbers alone could potentially be damaging, which is why it is so important to look beyond them into the actual mentions to understand the context of who is talking about a piece of research, and why.”
“When it comes to evaluating a research output, altmetrics are complementary to other quantitative and qualitative measures and, although they often add valuable insights, of course they alone cannot tell the whole story,” she added.
