Research Meeting: Visual Search and Selective Attention

Just returned from a really great meeting at the scenic lakeside (“Ammersee”) location near Munich, Germany. The third Visual Search and Selective Attention symposium was hosted and organised by Hermann Müller and Thomas Geyer (Munich), and supported by the Munich Center for Neurosciences (MCN) and the German Science Foundation (DFG). The stated aim of the meeting was:

"to foster an interdisciplinary dialogue in order to identify important shared issues in visual search and selective attention and discuss ways of how these can be resolved using convergent methodologies: Psychophysics, mental chronometry, eyetracking, ERPs, source reconstruction, fMRI, investigation of (neuropsychological) impairments, TMS and computational modeling."

The meeting was held over three days, and organised by four general themes:

- Pre-attentive and post-selective processing in visual search (Keynote: Hermann Müller)
- The role of (working) memory guidance in visual search (Keynote: Chris Olivers, Martin Eimer)
- Brain mechanisms of visual search (Keynote: Glyn Humphreys)
- Modelling visual search (Keynote: Jeremy Wolfe).

Poster sessions gave grad students (including George Wallis and Nick Myers) a great chance to chat about their research with the invited speakers as well as other students tackling similar issues. 

Of course, a major highlight was the Bavarian beer. Soeren Kyllingsbaek was still to give his talk, presumably explaining the small beer in hand!

More photos of the meeting can be found here.

***New***

All presentations can be downloaded from here

Journal Club: Brains Resonating to the Dream Machine

By George Wallis

On: VanRullen and Macdonald (2012). PerceptualEchoes at 10Hz in the Human Brain

One day in 1958 the artist Brion Gysin was sleeping on a bus in the south of France. The bus passed a row of trees, through which the sun was shining. As the flickering light illuminated Gysin, he awoke and with his eyes closed, began to hallucinate, seeing:

“an overwhelming flood of intensely bright patterns in supernatural colours… Was that a vision?”.  

By the turn of the decade Gysin was living with William S Burroughs in the flophouse in Paris that became known as the Beat Hotel. Gysin told Burroughs of his experience, and they decided to build a device to recreate the flickering stimulation. The ‘Dream Machine’ is a cylinder of cardboard, cut at regular intervals with windows, which can be spun on a 78rpm record player, a light bulb inside to throw off a flickering light.   The light flickers around ten times per second (10Hz). Some, like the poet Ginsberg (“it sets up optical fields as religious and mandalic as the hallucinogenic drugs”), claim to have experienced vivid hallucinations when seated eyes closed before a spinning Dream Machine (although, most devotees admitted that the effect was much stronger in combination with psychedelic drugs).

Gysin and Burroughs had rediscovered a phenomenon that had been known to scientists for some time. The great neurophysiologist Purkinje documented the hallucinatory effect of flickering light by waving an open-fingered hand in front of a gaslight. Another neuro-luminary, Hermann von Helmholtz, investigated the same phenomenon in Physiological Optics, calling the resulting hallucinations ‘shadow patterns’. In the 1930s Adrian and Matthews, investigating the rhythmic EEG signal recently discovered by Hans Berger, shone a car headlamp through a bicycle wheel and found that they could ‘entrain’ the EEG recording of their subject to the stimulation, in ‘a coordinated beat’. And from there investigation of the magical 10Hz flicker continued, on and off, until the present day (for a very readable review, see the paper by ter Meulen, Tavy and Jacobs referenced at the bottom of this post – from which the above quotations from Gysin and Ginsberg are taken; see also a relate post by Mind Hacks).

This week’s journal club paper is not about flicker-induced hallucinations. However, it does use EEG to address the related idea that there is something rather special to the visual system about the 10Hz rhythm. The paper, by Rufin VanRullen and James Macdonald, and published this month in Current Biology, used a very particular type of flickering stimulation to probe the ‘impulse response’ of the brain. They found – perhaps to their surprise – that the brain seems to ‘echo back’ their stimulation at about 10 echoes per second.

Macdonald and VanRullen’s participants were ‘plugged in’ during the experiment – electroencephalography (EEG) was used to measure the tiny, constantly changing voltages on their scalps that reflect the workings of the millions of neurons in the brain beneath. The stimulus sequence presented (with appropriate controls to ensure the participants paid attention) was a flickering patch on a screen. The flicker was of a very particular kind. It was a flat spectrum sequence, a type of signal used by engineers to probe the ‘impulse response’ of a system. The impulse response is the response of a system to a very short, sharp stimulation. Imagine clicking your fingers in an empty cathedral – that short, sharp click is transformed into a long, echoing sound that slowly dies away. This is the impulse response of the cathedral: VanRullen and MacDonald were trying to measure the impulse response of the brain’s visual system. Because of its property of very low autocorrelation (the value of the signal at one point in time says nothing about what the value of the signal will be at any other time), the kind of signal the authors flashed at their participants can be used to mathematically extract the impulse response of a system (for more details, see the paper by Lalor at al., referenced at the bottom of this post).

To extract the impulse response, you do a ‘cross-correlation’ of the input signal (the flickering patch on the screen) with the output of the system – which, in this case, was the EEG signal from over the visual cortex of the participants (the occipital lobe). Cross-correlation involves lining up the input signal with the output at many different points in time and seeing how similar the signals are. So, you start with the input lined up exactly with the output, and ask how similar the input and output signals look. Then you move the input signal so it’s lined up with the output signal 1ms later – how similar now? And so on… all the way up to around 1s ‘misalignment’, in this paper.  Here, for two example subjects (S1 and S2), is the result:

The grey curves are the cross-correlation functions, stretched out over time. Up until about 0.2 seconds you see the classic ‘visual evoked potential’ response, but after that time a striking 10Hz ‘echo’ emerges. The authors perform various controls, to show, for example, that these ‘echoes’ are not induced only by the brightest or darkest values in their stimulus sequence. They argue that because of the special nature of the stimuli they used, this effect must represent the brain actually ‘echoing back’ the input signal at a later time. In their discussion, they propose that this could be a mechanism for remembering stimuli over short periods of time: replaying them 10 times per second.

This is a bold hypothesis. Are these 10Hz reverberations really ‘echoes’ of the visual input, used for visual short term memory? We weren’t sure. We already know that the EEG resonates by far the most easily to flickering stimuli at 10Hz (see the paper by Hermann, referenced below), so despite the sophisticated stimulus used here, it is easy to suspect that the result of this experiment depends more on this ‘ringing’ quality of the EEG than on mnemonic echoes of stimuli themselves. We felt that in order to really nail this question you would need to show, for example, that our sensitivity to specific stimuli we have just been shown changes with a 10Hz rhythm in the seconds after we encounter it. However, this is the sort of thing that could be achieved with behavioural experiments.

Perhaps a new theory of short term memories will emerge.  

In the meantime, why not build yourself a dream machine and see if you can have your own visionary insights with the help of some 10Hz flickering light?  You’ll need the diagram below (blow up; cut out; fold into a cylinder), an old 78rpm record player, and a light-bulb.

References:

Current Biology, 2012: Perceptual Echoes at 10Hz in the Human Brain, Rufin VanRullen and James S.P. Macdonald

European Neurology, 2009: FromStroboscope to Dream Machine: A History of Flicker-Induced Hallucination,  B.C. ter Meulen, D. Tavy and B.C. Jacobs

NeuroImage, 2006: The VESPA: a method for the rapid estimation of a visual evoked potential.  Edmund C. Lalor, Barak A. Pearlmutter, Richard B. Reilly, Gary McDarby and John J. Foxe

Experimental Brain Research, 2001: Human EEG responses to 1-100 Hzflicker: resonance phenomena in visual cortex and their potential correlationto cognitive phenomena.

In the news: Mind Reading

Mind reading tends to capture the headlines. And these days we don't need charlatan mentalists to perform parlour tricks before a faithful audience - we now have true scientific mind reading. Modern brain imaging tools allow us to read the patterns of brain activity that constitute mind... well, sort of. I thought to write this post in response to a recent Nature News Feature on research into methods for reading the minds of patients without any other means of communication. In this post, I consider what modern brain imaging brings to the art of mind reading.

Mind reading as a tool for neuroscience research

First, it should be noted that almost any application of brain imaging in cognitive neuroscience can be thought of as a form of mind reading. Standard analytic approaches test whether we can predict brain activity from the changes in cognitive state (e.g., in statistical parametric mapping). It is straightforward to turn this equation round to predict mental state from brain activity. With this simple transformation, the huge majority of brain imaging studies are doing mind reading. Moreover, a class of analytic methods known as multivariate (or multivoxel) pattern analysis (or classification) have come even closer to mind reading for research purposes. Essentially, these methods rely on a two-stage procedure. The first step is to learn which patterns of brain activity correspond to which cognitive states. Next, these learned relationships are used to predict the cognitive state associated with brain activity. This train/test procedure is strictly "mind reading", but essentially as a by-product.

In fact, the main advantage of this form of mind reading in research neuroscience is that it provides a powerful method for exploring how complex patterns in brain data vary with the experimental condition. Multivariate analysis can also be performed the other way around (by predicting brain activity from behaviour, see here), and similarly, there is no reason why train-test procedures can't be used for univariate analyses. In this type of research, the purpose is not actually to read the mind of cash-poor undergraduates who tend to volunteer for these experiments, but rather to understand the relationship between mind and brain.

Statistical methods for prediction provide a formal framework for this endeavour, and although they are a form of mind reading, it is unlikely to capture the popular imagination once the finer details are explained. Experiments may sometimes get dressed up like a mentalist's parlour trick (e.g., "using fMRI, scientists could read the contents of consciousness"), but such hype invariably leaves those who actually read the scientific paper a bit disappointed by the more banal reality (e.g., "statistical analysis could predict significantly above chance whether participants were seeing a left or right tilted grating"... hardly the Jedi mind trick, but very cool from a neuroscientific perspective), or contribute to paranoid conspiracy theories in those who didn't read the paper, but have an active imagination.

Mind reading as a tool for clinical neuroscience

So, in neuroscientific research, mind reading is most typically used as a convenient tool for studying mind-brain relationships. However, the ability to infer mental states from brain activity has some very important practical applications. For example, in neural prosthesis, internal thoughts are decoded by "mind reading" algorithms to control external devices (see previous post here). Mind reading may also provide a vital line of communication to patients who are otherwise completely unable to control any voluntary movement.

Imagine you are in an accident. You suffer serious brain damage that leaves you with eye blinking as your only voluntary movement for communicating with the outside world. That's bad, very bad in fact - but in time you might perfect this new form of communication, and eventually you might even write a good novel, with sufficient blinking and heroic patience. But now imagine that your brain damage is just a little bit worse, and now you can't even blink your eyes. You are completely locked in, unable to show the world any sign of your conscious existence. To anyone outside, you appear completely without a mind. But inside, your mind is active. Maybe not as sharp and clear as it used to be, but still alive with thoughts, feelings, emotions, hopes and fears. Now mind reading, at any level, becomes more than just a parlour trick.

"It is difficult to imagine a worse experience than to be a functioning mind trapped in a body over which you have absolutely no control" Prof Chris Frith, UCL [source here]

As a graduate student in Cambridge, I volunteered as a control participant in a study conducted by Adrian Owen to read mental states with fMRI for just this kind of clinical application (since published in Science). While I lay in the scanner, I was instructed to either imagine playing tennis or to spatially navigate around a familiar environment. The order was up to me, but it was up to Adrian and his group to use my brain response to predict which of these two tasks I was doing at any given time. I think I was quite bad at spatially navigating, but whatever I did inside my brain was good enough for the team to decode my mental state with remarkable accuracy.

Once validated in healthy volunteers (who, conveniently enough, can reveal which task they were doing inside their head, thus the accuracy of the predictions can be confirmed), Adrian and his team then applied this neuroscientific knowledge to track the mental state of a patient who appeared to be in a persistent vegetative state. When they asked her to imagine playing tennis, her brain response looked just like mine (and other control participants), and when asked to spatially navigate, her brain looked just like other brains (if not mine) engaged in spatial navigation.

In this kind of study, nothing very exciting is learned about the brain, but something else extremely important has happened: someone has been able to communicate for the first time since being diagnosed as completely non-conscious. Adrian and his team have further provided proof-of-principle that this form of mind reading can be applied in other patients to test their level conscious awareness (see here). By following the instructions, some patients were able to demonstrate for the first time a level of awareness that was previously completely undetected. In one further example, they even show that this brain signal can be used to answer some basic yes/no questions.

This research has generated an enormous amount of scientific, clinical and public interest [see his website for examples]. As quoted in a recent Nature New Feature, Adrian has since been "awarded a 7-year Can$10-million Canada Excellence Research Chair and another $10 million from the University of Western Ontario" and "is pressing forward with the help of three new faculty members and a troop of postdocs and graduate students". Their first goal is to develop cheaper and more effective means of using non-invasive methods like fMRI and EEG to restore communication. However, one could also imagine a future for invasive recording methods. Bob Knight's team in Berkeley have been using electrical recording made directly from the brain surface to decode speech signals (see here for a great summary in the Guardian by Ian Sample). Presumably, this kind of method could be considered for patients identified as partially conscious.

See also an interesting interview with Adrian by Mo Constandi in the Guardian

References:
Monti, al. (2010). Willful modulation of brain activity in disorders of consciousness. New England Journal of Medicine
Owen, et al (2006). Detecting awareness in the vegetative state. Science

Pasley,  et al (2012). Reconstructing Speech from Human Auditory Cortex. PLoS Biology

Book Review: Sum

 Sum: Forty Tales from the Afterlives by David Eagleman

This inaugural book review for the Brain Box does not feature the latest neuroscience book to hit the shelves, nor is it even the latest work by author, David Eagleman. What marks this book out in particular is a recent chamber opera adaptation by composer Max Richter and directed by the choreographer Wayne McGregor, which I saw performed last night at the Royal Opera House Linbury Studio Theatre. So, this is a slightly unconventional start, part book and opera review!

"In the afterlife..."

As the title suggests, the book is comprised of a collection of short stories, more like a series of vignettes, each imagining a different possible afterlife. For example, the opening tale, Sum, invites you to image an afterlife in which you relive all your previous experiences, but reordered and grouped according to common themes/qualities. You spend six days clipping your nails, six weeks waiting for a green light, one year reading books, two week lying, three week realising you are wrong, two weeks counting money, etc. And buried in this inventory of such life experience is fourteen minutes of pure joy, as well as the pain and heartache all tallied and accounted for.

"...fourteen minutes of pure joy..."

The opera also beings with this title piece, with an intense overlay of instrument and voice, interweaving fragments of a categorised life with fourteen minutes of pure joy at the heart of the storm. It is a powerful opening, the emotion intensified by the use of space to trap and magnify the experience of sound.

"The spoken word becomes like a thought flying across space"

The whole performance is contained within a cube, or waiting room, surrounded by large projected walls carrying a constant flow of images. The musical ensemble plays from a central pit and the vocal performers roam about the audience. According to the director, all these elements should "coalesce to have a visceral, personal and profound impact on each individual in the room". Indeed, within the confined space, it is impossible to remained detached. The director certainly succeeds in creating a "living, breathing installation where the audience become intrinsic players"

Max Richter likens Sum to a series of literary variations, a study of the same subject from different angles and  perspectives. Although, strictly, each story is mutually exclusive, the narrative flows from one vignette to the other as Eagleman sketches out the human condition. Like the lone quark in the tale Conservation, a singular common theme is used to sketch out the hopes, dreams, loves and disappointments of the human, a curious creature, who, despite the sophisticated sensory apparatus, simply wants to clump together with other conspecifics, to be stroked and look at one another (from tale Narcissus).

The opera captures the powerful emotion and beauty of Sum, but not so much the humour. It would probably be a mistake to attempt an operatic translation of hilarious tales like the Death Switch, in which life is preserved through an absurd extension of the out-of-office-reply. Also conspicuously absent is the humorous tale Graveyard of the Gods. The opera is almost certainly better for these absences, enabling a more coherent, and deeper exploration of a common theme. But for the full experience, the book is essential reading.

Read and listen to more from Max Richter here
And read more from Wayne McGregor here
And hear an interview with David Eagleman on this Guardian podcast

A simple plan for open access?

Just chatting with Chris Chambers, and we came up with this simple 6-step plan to solve the Open Access problem:

1. Submit your paper to your Journal of Choice

2. With editor approval, go to review

3. With reviewer approval, make suggested changes, tweak figures, add caveats, improve the science, etc.

4. Repeat steps 2-3, or 1-3 as necessary

5. Finally, with editor approval, receive acceptance email (and notification of publication cost, copyright restrictions, etc)

6. Now, here's the sting: take your accepted peer-reviewed paper and publish it yourself, on-line, along with all the reviewer comments, reply to reviewers (more reviewer comments, replies to comments, etc.) and most importantly, the final decision email - i.e., your proof of endorsement from said Journal of Choice

Are you brave enough to follow these 6 simple steps to DIY Open Access? It is fully peer-reviewed, and endorsed by Journal of Choice, with good reputation and respectable impactor factor. I am not, and so instead I just signed this petition to the White House to: Require free access over the Internet to scientific journal articles arising from taxpayer-funded research. If you haven't done so already, get to it! Non-US signatories are welcome...

Also see: http://deevybee.blogspot.co.uk/2012/01/time-for-academics-to-withdraw-free.html

A Tale of Two Evils: Bad statistical inference and just bad inference

Evil 1:  Flawed statistical inference

There has been a recent lively debate on the hazards of functional magnetic resonance imaging (fMRI), and what claims to believe or not in the scientific and/or popular literature [here, and here]. The focus has been on flawed statistical methods for assessing fMRI data, and in particular failure to correct for multiple comparisons [see also here at the Brain Box]. There was quite good consensus within this debate that the field is pretty well attuned to the problem, and has taken sound and serious steps to preserve the validity of statistical inferences in the face of mass data collection. Agreed, there are certainly papers out there that have failed to use appropriate corrections, and therefore the resulting statistical inferences are certainly flawed. But hopefully these can be identified, and reconsidered by the field. A freer and more dynamic system of publication could really help in this kind of situation [e.g., see here]. The same problems, and solutions apply to non-brain imaging field [e.g., see here].

But I feel that it may be worth pointing out that the consequence of such failures is a matter of degree, not kind. Although statistical significance is often presented as a category value (sig vs ns), the threshold is of course arbitrary, as undergraduates are often horrified to learn (why P<.05? yes, why indeed??). When we fail to correct for multiple comparisons, the expected probabilities change, therefore the reported statistical significance is incorrectly represented. Yes, this is bad, this is Evil 1. But perhaps there is a greater, more insidious evil to beware.

Evil 2: Flawed inference, period.

Whatever our statistical test say, or do not say, ultimately it is the scientist, journalist, politician, skeptic, whoever, who interprets the result. One of the most serious and common problems is flawed causal inference: "because brain area X lights up when I think about/do/say/hear/dream/hallucinate Y, area X must cause Y". Again, this is a very well known error, undergraduates typically have it drilled into them, and most should be able to recite like mantra: "fMRI is correlational, not causal". Yet time and again we see this flawed logic hanging around, causing trouble.

There are of course other conceptual errors at play in the literature (e.g., there must be a direct mapping between function and structure; each cognitive concept that we can imagine must have its own dedicated bit of brain, etc), but I would argue perhaps that fMRI is actually doing more to banish than reinforce ideas that we largely inherited from the 19th Century. The mass of brain imaging data, corrected or otherwise, will only further challenge these old ideas, as it becomes increasingly obvious that function is mediated via a distributed network of interrelated brain areas (ironically, ultra-conservative statistical approaches may actually obscure the network approach to brain function). However, brain imaging, even in principle, cannot disentangle correlation from causality. Other methods can, but as Vaughan Bell poetically notes:

Perhaps the most important problem is not that brain scans can be misleading, but that they are beautiful. Like all other neuroscientists, I find them beguiling. They have us enchanted and we are far from breaking their spell. [from here]

In contrast, the handful of methods (natural lesions, TMS, tDCS, animal ablation studies) that allow us to test the causal role of brain function do not readily generate beautiful pictures, and perhaps, therefore suffer a prejudice that keeps them under-represented in peer-review journals, and/or popular press. It would be interesting to assess the role of beauty in publication bias...

Update - For even more related discussion, see:
http://thermaltoy.wordpress.com/2012/05/28/devils-advocate-uncorrected-stats-and-the-trouble-with-fmri/
http://www.danielbor.com/dilemma-weak-neuroimaging/
http://neuroskeptic.blogspot.co.uk/2012/04/fixing-science-systems-and-politics.html

The Science in the Middle

When it comes to controversies, science can find itself stuck between GMO doomsayers on the left and climate change deniers on the right. To the former, science may be the evil arm of big business interests, but to the latter, a bunch of left wing saboteurs intent on halting progress and civilisations. Both sides of the argument can be high-jacked for political point scoring.

Today, a united front of self-proclaimed "Geeks in the Park" staged a protest against anti-GMO group "Take the Flour Back" hoping to halt an experimental trial in Harpenden to test a genetically modified wheat crop. It is a pretty fiery debate. Following it live on Twitter today, quite a few inflammatory things were said on both sides.

From a Tweeting Greens Party politician on the anti-GMO side:

The mouth frother is still here. Being debated with. I must say, very brave of him to mix with us. Credit for that.

And from a Tweeting Labour Party politician, on the anti-anti-GMO side:

Have lots of anti-histamines & am tempted to offer them to any sneezing anti-GM protestors. But animal tested

But jokes aside, this is obviously an important issue. Research into how we (i.e., a very large number of people, who show every sign of becoming an ever-larger number!) are going to feed ourselves into the 21st century is probably one of the most pressing issues facing science today. There will be many routes that need to be explored, including genetic modification (following in the tradition of the great 19th century Augustinian friar Gregor Mendel) as well as other agricultural developments (which will also have potential risks and unforeseen side effects - everything does!). Moreover, pending climate change makes this research even more urgent. But here we find science attacked from the other side of the political spectrum. The list of strongly worded claims and counter claims is pretty long, but for a taste of the controversy see here for one scientist's perspective (and ensuing comments) and here for the latest views from climate-change skeptics.