Science, LIVE: A made-for-TV experiment

[also see my related Guardian post]

A few months ago, Channel 4 attracted considerable attention for their sensationally titled – “Drugs Live: the Ecstasy Trial”. The subject matter was clearly designed to court controversy, with Prof Nutt at the centre of the storm. The show "hooked almost 2 million" viewers on the first night, and triggered a lively debate around highly charged questions, such as: Do we need to focus more on the medical/chemical nature of particular drugs, and less on the moral/legal status? Is it right to film volunteers taking a Class A drug, even for a medial experiment? Is Channel 4 glorifying illegal drugs, or contributing to rational discussion?

Much was written and said on either side of this debate (e.g. this conversation between Nutt and Manning). However, as an empirical scientist, I would like to draw attention to another more general issue that was  perhaps neglected in the mêlée of moral and ethical arguments. I would like to know why TV science is conducting experiments in the first place?

The made-for-TV experiment

Drugs Live was structured around an ethically approved double-blinded experiment to test the effects of MDMA. Data were collected from 25 participants under the influence of MDMA and a placebo control (sugar pill). Tests included questionnaire and computer-based tasks to measure changes in mood and cognition, as well as the ever-TV-friendly fMRI to measure changes in brain activity. This sounds like a reasonable set of experiments, but according to Prof Nutt, the Medical Research Council (MRC) declined to fund the research because it did not "fit in with the MRC's portfolio of addiction". Instead, Channel 4 agreed to pick up the tab, presumably for more financial motives compared to the MRC's commitment to "improve human health through world-class medical research" (from mission statement). Perhaps we should celebrate this innovative collaboration between academia and the private sector. In these times of austerity, perhaps TV-funded research is the future big-society answer to maintain Britain’s place as a leading powerhouse of innovation, science and technology.

And indeed, if the research is well-conducted, the results could provide valuable insights into the effects of MDMA, of genuine scientific interest with important political/social relevance. After many weeks and months of painstaking data analysis, the results could be submitted to a reputable scientific journal for rigorous peer-review. If the submitted findings are accepted by the academy as sufficiently trustworthy, then the scientific report would be published for consideration by a wider scientific audience. Journal press-releases might then alert the popular news outlets, who may then report these novel findings to their more general readership. 

This is how scientific findings are normally disseminated to the wider audience. Slowly, but surely, complex data yield their secrets to careful systematic analysis. It is not gripping TV, but this systematic process is the foundation of modern scientific research. Made-for-TV science, on the other hand, can by-pass the process completely and stream their own results directly into living rooms across the country. 

Science to a production schedule: lights, camera, action!

Maybe science needs a bit more of a can-do attitude. Like Jon Snow, who promises on Drugs Live that “tonight we will get to the bottom of it”. Not in another month, six months, or couple of years - but this very evening! And true to his word, by the end of the first episode Snow can already announce that we have all witnessed “two scientific breakthroughs”. He was not very specific, but we may assume that he was referring to the two brain scans that were rotated on a large plasma screen.


The first scan, from actor Keith Allen, appeared to show a relative decrease in communication between two areas previously associated with the so-called default mode network. Firstly, I should leave aside any academic debate about the true nature of this brain network, as tempting as it is to question Prof Nutt’s proclamation that this area is no less than “you, your personality, sense of self”. My purpose here is just to make the point that data from one brain in one volunteer, hastily analyzed and not peer-reviewed, does not constitute a “scientific breakthrough”. Perhaps an interesting hint. A potential clue, maybe. Promising lead, why not? But certainly not a: “scientific breakthrough”.

The second scan was even more ambiguous. The rotating image appeared to show a number of brain regions we are told were more active when the volunteer closed her eyes after taking MDMA. We are told that this reflects the heightened perceptual experience caused by the drug. But to be blunt, these data look like a mess, like random variation in the MRI signal. This is not really all that surprising, considering it is only one scan from one person, analyzed under the unrealistic time pressure of the TV production schedule. I would be amazed, or even suspicious if the result was any clearer. 

Of course science needs to be simplified for a TV audience. Matt Wall, neuroscientist involved in the Drugs Live programme, says of his experience:

"TV needs everything to be black-and-white, and unambiguous... They don’t care that you haven’t run the necessary control experiments, or that the study was only correlational and therefore can’t be used to imply direct causation – they want a neat, clear story above all else"

Oversimplifying the deeper complexities "can very often lead to distortions, or ‘Lies-to-children’". This is a perennial issue for TV science, whether following the classic science reporting formula or made-for-TV experiments. To be able to articulate complex theoretical concepts and technical details to general audience without misrepresentation is a great but rare skill.

Astonishing Science

Jon Snow promised his live audience “astonishing science”. Quite right, TV should bring astonishing science to the wider audience. But this mission is seriously compromised by the production demands associated with made-for-TV experiments.

Advising the Discovery Channel on a recent made-for-TV experiment, I was told by a production assistant: “you don’t have to always be so cynical, you know.” And I absolutely agree with the sentiment. Science TV should convey the excitement of science, not just the limitations. Just like this production assistant, I am also frustrated by too many caveats. The reason I came to science was to discover something about the world, not just to point out flaws in putative findings. But like any empirical scientist, I have learned many times over not to get too excited over half-baked results. Only solid reliable results are really exciting.

But TV science does not have to be boring. TV has many tricks up its sleeve, such as dramatic music, frenetically paced scene cuts, angled screen shots in darkened laboratories, and expensive props like MRI. All these can be used to covey the excitement of science, without resorting to made-for-TV experiments.

Reality-science TV

The enormous success of reality TV tells us that viewers like to experience the activities on screen through people they can relate to. Extrapolating to science TV, I guess viewers like to feel the science experience through personalities that they can relate to. The personal touch can make it seem more real.

Recently, I was asked to help conduct another made-for-TV brain imaging experiment with the show's presenter as the experimental subject. To fit the production schedule, we had to analyse complex brain imaging data within a matter of hours. We did manage to produce some very rudimentary results within this science-improbable time frame - we had to! Production costs are clocked by the hour.

Of course the actual result was of limited scientific value, and we were naturally so circumspect about what we said on camera that it is hard to see how this data could have been of any great interest to the audience. It was essentially just a brain on a screen: demonstration science - what it looks like to do science.
There is of course no harm in such demonstrations - eye-catching demonstrations are bread and butter tools for conveying the excitement of science. We don't need to pretend that they are also conducting novel scientific research. It is important enough to help convey the process of science without pretending to add to the content of scientific knowledge.There are plenty of good and proper TV production devices for engaging public interest in science. And we expect a little bit of TV gimmickry, it is show biz after all. But why not be content with reporting on science, rather than making science as well?

Investigative Journalism

The Drugs Live formula is a hybrid of traditional science programme and the exposé. Borrowing from the rich history of the investigative journalism, TV is not just reporting news, but making news as well. The production company can herald exclusive access to a breaking news story:

“Now, in a UK television first, two live programmes will follow volunteers as they take MDMA, the pure form of ecstasy, as part of a ground-breaking scientific study” [from the Channel 4 series synopsis]

But investigative journalism is also tricky business. The precise outcome of any investigation is impossible to predict, and therefore hard to plan for. Out of the many possible leads, only a minority will reveal something worth reporting. Producers are presumably familiar with the frustration of stories that lead nowhere, and presumably they are reasonably careful about committing to a production schedule until after the results of the investigation are relatively clear. Jumping to premature conclusions can lead to serious false claims, as dramatically highlighted recently by the Newsnight debacle that cost the BBC general director his job. In a recent post-mortem of this botched investigation, David Leigh writes: “to be faithful to the evidence" is essential for successful investigative journalism. And although journalism may not be "rocket science" [in Leigh's words], investigative journalism also demands a genuine commitment to follow the evidence, wherever it leads. 

Conflict of interest

In the shadow of recent revelations of fraud and serious malpractice across a range of scientific disciplines, from psychology to anaesthesiology, many scientists have been asking how to improve the scientific process (e.g., see here and here). Televising the process is unlikely to be the answer. 

Although industry-funding can be a valuable source of revenue, we must always seriously consider potential conflicts of interest. Increasingly, concerns have been raised regarding dubious practices in clinical research funded by pharmaceutical companies. Ben Goldacre’s book, Bad Pharma, is a must read on this serious public health issue. Conflicts of interest can distort many stages of the experimental process to increase the likelihood of finding a particular result. Clinical research is becoming increasingly alert to these problems, and serious steps are being made to avoid the malevolent influence of funding agencies with vested interests.

But even without a commercial interested in a particular result, the pressure to "find something" noteworthy can also motivate bad scientific practice. In academic circles, the pressure to publish is typically considered a major driving factor in scientific malpractice and fraud.The bottom line for Channel 4, of course, is to maximise audience numbers to boost the value of their commercial time. This does not automatically rule out potential scientific merit of TV-funded experiments, but it certainly is worth bearing in mind, especially when thinking about how the particular demands of TV could compromise scientific method. As discussed above, these include short-cuts and rushed analyses to fit a tight production schedule, as well as the pressure to find "something in the data" by the end of the shoot, however unreliable it might turn out to be later. The experimental approach in Drugs Live was also apparently compromised by recruiting celebrities (and other TV-friendly personalities) as experimental subjects, tapping into the proven success of the reality-TV format but skewing the sample of experimental subjects. It is also hard to imagine that omnipresent TV cameras did not influence the results of the experiment.

It will be interesting to follow up on this research to see how the results are received within the academic community. There is no reason not to expect some interesting and important findings, but I wonder if the more detailed and scientifically meaningful results will be heralded with as much fanfare as the actual pill-popping on camera. I fear Channel 4 might be more interested in the controversy surrounding MDMA than the science motivating the research.