Whose fault is shoddy science reporting?
Ours, of course! The BBC has this to say about inaccurate stories:
"...media cynics may be surprised to learn that - as in the case of the hypnosis study - the stories generally have their origins in the scientific community." (Emphasis mine)
Well, you'd hope so, wouldn't you? That has to be true, unless journalists just fabricate science stories without reference to any actual research. Actually, I could believe that of those "food X causes/cures condition Y" snippets in Cosmo.
The article linked above deals with a recent Royal Socety report. Having made the startling realisation that scientific papers are sometimes wrong, the author rebukes Nature and The Lancet for publishing them, and also chides the organisers of a research conference for allowing research to be presented at a conference. From the tone, you'd think the Royal Society was on his side in railing against the dangerous, potentially misleading practice of publication.
However, the Royal Society report is about publicity, discussing whether, when and how a scientist should talk directly to the public or press. It incudes a few helpful and commonsensical tips - be clear about your statistical limitations, make it obvious when you are presenting preliminary data, that kind of thing - and also mentions a previous report which gave similar hints for media types, noting that they "largely did not welcome offers of such help."
Of course, science journalists who know enough about science can read the primary literature and report on it drawing on their own knowledge of statistics and experimental design, but they are hampered in this task by the fact that, with rare exceptions, they don't exist. So, in reality, almost all science news stories come from press releases put out by Nature, Science and a few other big-name journals to coincide with a paper's publication. The researchers involved have no say in what's written in the press release, but at a pinch we can consider the story in the hands of the "scientific community" until this point. Now, the big journals could probably do a lot better in many ways, e.g. by giving replies and rebuttals as much publicity as the papers they criticise. However, I can hardly fault them for including in their press releases descriptions of papers contained in the journal.
So I'm rather confused as to how we can prevent the egregious miscarriages of truth for which we are entirely responsible. It appears that our options are:
a. Not make any anomalous or controversial findings, working only on obscure stuff about which nobody gives a metric toss. I have so far been employing this tactic with great success.
b. Not publish any anomalous or controversial findings, discussing them only in darkened lecture theatres through the use of code words and interpretive dance. I like the sound of this one too.
c. Conceal anomalous or controversial findings by not including them in press releases. This would be easy, but conflicts with previous complaints about our making insufficient effort to communicate.
d. Add a disclaimer to all press releases:
"WARNING: MIGHT NOT BE RIGHT.
This is the entire point of science. If you don't like it, write about celebrities' haircuts."
"...media cynics may be surprised to learn that - as in the case of the hypnosis study - the stories generally have their origins in the scientific community." (Emphasis mine)
Well, you'd hope so, wouldn't you? That has to be true, unless journalists just fabricate science stories without reference to any actual research. Actually, I could believe that of those "food X causes/cures condition Y" snippets in Cosmo.
The article linked above deals with a recent Royal Socety report. Having made the startling realisation that scientific papers are sometimes wrong, the author rebukes Nature and The Lancet for publishing them, and also chides the organisers of a research conference for allowing research to be presented at a conference. From the tone, you'd think the Royal Society was on his side in railing against the dangerous, potentially misleading practice of publication.
However, the Royal Society report is about publicity, discussing whether, when and how a scientist should talk directly to the public or press. It incudes a few helpful and commonsensical tips - be clear about your statistical limitations, make it obvious when you are presenting preliminary data, that kind of thing - and also mentions a previous report which gave similar hints for media types, noting that they "largely did not welcome offers of such help."
Of course, science journalists who know enough about science can read the primary literature and report on it drawing on their own knowledge of statistics and experimental design, but they are hampered in this task by the fact that, with rare exceptions, they don't exist. So, in reality, almost all science news stories come from press releases put out by Nature, Science and a few other big-name journals to coincide with a paper's publication. The researchers involved have no say in what's written in the press release, but at a pinch we can consider the story in the hands of the "scientific community" until this point. Now, the big journals could probably do a lot better in many ways, e.g. by giving replies and rebuttals as much publicity as the papers they criticise. However, I can hardly fault them for including in their press releases descriptions of papers contained in the journal.
So I'm rather confused as to how we can prevent the egregious miscarriages of truth for which we are entirely responsible. It appears that our options are:
a. Not make any anomalous or controversial findings, working only on obscure stuff about which nobody gives a metric toss. I have so far been employing this tactic with great success.
b. Not publish any anomalous or controversial findings, discussing them only in darkened lecture theatres through the use of code words and interpretive dance. I like the sound of this one too.
c. Conceal anomalous or controversial findings by not including them in press releases. This would be easy, but conflicts with previous complaints about our making insufficient effort to communicate.
d. Add a disclaimer to all press releases:
"WARNING: MIGHT NOT BE RIGHT.
This is the entire point of science. If you don't like it, write about celebrities' haircuts."
Labels: Boffinry
posted by MissPrism at 2:01 PM
6 Comments:
Fabulous suggestions. But then journalists might only write about celebrity haircuts and the public might be even more clueless about how science works. While I agree that scientists and journal publishers could do a better job of communicating science, shouldn't the reporters and the public be more willing to read more than 150 words on a subject before flipping to the comics? Oh, and if they could grasp uncertainty - then we'd really be in business.
Sulk. You beat me to the blog! I saw this article on the BBC this morning and was calming down before blogging about it. I don't think we, as scientists, need do anything about it. My hard line view is that it is the media that portrays the results in an overly 'hyped up' mannger, it is journalists that fuel the fire of miscarriages of truth and damage science reputation by writing about stuff they don't understand.
Hint: University media studies degrees do not incorporate courses on understanding scientific publications.
Maybe they should.
Kinder solution: editiors PLEASE speak to the corresponding authors before publishing on the publications, and let them see the article before it goes to press.
OK, getting in a knot here. Rant over.
Good job blogging it by the way, and I like your blog name!
“So, in reality, almost all science news stories come from press releases put out by Nature, Science and a few other big-name journals to coincide with a paper's publication. The researchers involved have no say in what's written in the press release, but at a pinch we can consider the story in the hands of the "scientific community" until this point.”
This, I think, is a core issue. I can’t think of any other activity in society where the common consensus is that the people doing it should be allowed to keep to themselves and do whatever they want for as long as it takes, and the rest of society should be happy to wait until they’re finished and then we can get a peek at the results. While politicians, artists, athletes and business people have to perform their processes and make their mistakes in public, scientists usually do it among themselves.
I don’t mean to blame the scientists, and I’m not sure if I’m being very clear, but this old post from Pharyngula sums up my thoughts very nicely.
I think this will change with time, though. We just have to keep working at it. To be able to report from a research project from the start, investigate the reasons for the project and the questions raised, to ask what the different contributors are hoping to achieve and why, to report on the setbacks as well as the progress – THAT would be a story I’d love to write!
I'm sorry for the long comment!
Kjerstin - yes, I agree it would be great to have a more continuous communication between scientists and the public. The BBC article was arguing the opposite: that potentially newsworthy results should be kept quiet until they are "established fact". The problem, which Sciencewoman puts her finger on, is uncertainty: peer reviewed publication is not the end of the assessment process, only the first stage of it, and the earlier a result is communicated to the public the more uncertain it will be. It'd take a lot of (worthwhile) work to prevent this translating to "scientists change their minds from week to week, can't believe a word they say" in the public mind.
Propter, you are so right about media studies degrees. Some quantitative content would do a world of good (and help the courses shed their "Mickey Mouse" image, which would be useful for all concerned). John Allen Paolos runs a "Numbers in the News" course (link to syllabus) which sounds like a good start.
Yes, that's what I mean: If there was less focus on results and more focus on process, uncertainty would inevitably be a bigger part of the picture. And we'd all be better off.
hi, only just spotted this. amazing how other journalists can critically appraise the assertions in a press release but with science stories they have to blindly and innocently follow.. surely thats the point, that they should be able to critically appraise the material.
and that's before i get onto the errors in this bbc piece:
"The GM crop scare started after a professor at a respected institute said his experiments showed that genetically modified potatoes had stunted the growth of lab rats."
so, arpad pusztai has been promoted to professor now? and of course, there was no paper during the frankenfoods saga, there were just some chats with some journalists. and then when the paper came out so people could actually read it, it had numerous interesting flaws. etc.
Post a Comment
<< Home