Episode 89      28 min 20 sec
Moving and seeing again: the promise of neural interface technologies

Prof John Donoghue and Prof Robert Shepherd discuss the development of neural interface technologies, the product of researchers across many disciplines, that could enable the brain to interact with damaged limbs and eyes. With host Shane Huntington.

"But we believe that with larger numbers of electrodes we can provide the ability for patients to walk unassisted and, eventually, have potentially face recognition." - Prof Robert Shepherd




           



Prof John Donoghue
Prof John Donoghue

Professor John Donoghue is Researcher at the Providence Veterans Affairs Medical Center, and Director of the Institute for Brain Science at Brown University.

John's research investigates fundamental and translational aspects of cortical information processing that leads to skilled motor behavior. His lab studies the way ensembles of neurons represent and transform information through their interactions within local regions and between the primary motor cortex and its major cortical input areas. This work helps to explain how networks of neurons compute motor actions and transform sensory and internal plans into specific movements of the limbs.

In Prof Donoghue's applied work, his lab translated their technical and basic neuroscience advances to create a neural interface system, termed “BrainGate”, that is designed to connect the brain to the muscles or to assistive devices for people with paralysis. John is the senior investigator in this multidisciplinary project and has led the transformation from basic laboratory studies to human clinical trials.

Prof Robert Shepherd
Prof Robert Shepherd

Professor Robert Shepherd is the Director of the Bionic Ear Institute and Professor of Medical Bionics in the Department of Otolaryngology at the University of Melbourne.

Robert received his Ph.D. in Otolaryngology from the University of Melbourne. His doctoral thesis examined the safety and efficacy of multichannel cochlear implants and contributed to Cochlear Ltd’s successful FDA approval for the clinical use of a multichannel cochlear implant. Prof Shepherd has held international research appointments throughout his career at institutions such as the Virginia Merrill Bloedel Hearing Research Center, University of Washington, Seattle, U.S.A.; Department of Physiology.

Prof Shepherd has authored 145 peer-reviewed scientific papers and invited book chapters. He has been the Principal Investigator on more than $5.3M of research funding including grants from the National Health and Medical Research Council of Australia and the U.S. National Institutes of Health. He continues to work closely with industry through research links with Cochlear Ltd. and Living Cells Technologies Ltd.

Credits

Host: Dr Shane Huntington
Producers: Kelvin Param, Eric van Bemmel
Associate Producer: Dr Christine Bailey
Series Creators: Eric van Bemmel and Kelvin Param
Audio Engineer: Ben Loveridge
Voiceover: Nerissa Hannink

View Tags  click a tag to find other episodes associated with it.

Download file Download mp3 (27.3 MB)

Moving and seeing again: the promise of neural interface technologies

VOICEOVER
Welcome to Up Close, the research, opinion and analysis podcast from the University of Melbourne, Australia.

SHANE HUNTINGTON
Hello and welcome to Up Close.  I’m Dr Shane Huntington.  Not very long ago it was thought near impossible, and almost science fiction-like, to restore movement in paraplegics and sight to the blind.  Breaking through medical boundaries to make near impossibilities a reality is the very work of two experts joining us in this episode of Up Close.  With the help of what are called neural interfaces our guests, together with colleagues, are giving hope to people who once had only minimal prospects of interacting with the external word.  Our guests on Up Close today are Professor John Donoghue, a researcher at the Providence Veterans Affairs Medical Center, and Director of the Institute for Brain Science at Brown University in the United States.  Professor Donoghue is also cofounder of Cyberkinetics Neurotechnology Systems Inc.  Our second guest is Professor Robert Shepherd, Director of the Bionic Ear Institute, Chief Investigator on the Bionic Vision Australia program and a professor of medical bionics at the Department of Otolaryngology at the University of Melbourne, Australia.  Welcome both of you to Up Close.

ROBERT SHEPHERD
Thanks, Shane.

JOHN DONOGHUE
Thank you very much, Shane.

SHANE HUNTINGTON
John, I'd like to start with you and talk a bit about the types of signals that are actually sent by the brain.  Can you give us an idea of what these signals are and how they would compare electrically to other things we'd experience in everyday life?

JOHN DONOGHUE
Well, they're unusual in many ways but, basically, each neuron, each cell in the brain, emits a tiny train of electrical impulses about a thousandth of a second long.  If you were to listen in on one neuron emitting its message it sounds like a bunch of clicks and those clicks are the rapid communication system of the brain.  One neuron talks to another through these clicks and it talks, say, from the brain to the muscles and makes muscles contract by using these click-like signals which we call spikes; the correct name is action potentials, but in the field everyone says spikes.

SHANE HUNTINGTON
How does the brain differentiate, for example, in the signals it would send for, say, blinking versus moving your arm?

JOHN DONOGHUE
Actually, they're not different at all.  The same kinds of signals are used not only for moving any muscle but also for thinking or for seeing or for hearing.  That same kind of coded message is sent everywhere.  One of the differences, say for blinking and moving the arm, is they're at different spots in the brain.  One's sort of down by your cheekbone - that's a place where facial muscles are controlled - and one a little bit higher up towards the middle of your head is a zone that controls your arm.

SHANE HUNTINGTON
We're talking today about restoring function.  Before we get to that, can you describe for me the whole process of how from a thought, how we go about actually, say, lifting our arm off the table?

JOHN DONOGHUE
Well, that would take an entire course to get the details, but to give you just a general sense there is this region in the brain, as I said, sort of up near the top of your head - think of it at a couple of centimetres square - and all of your brain's activities impinge on that area and it gives rise to a signal that says move.  It's in those spiking signals.  Many millions of neurons send that signal, sort of process is a little bit like a computer, and then send those signals out to the spinal cord.  It basically passes from the brain to the spinal cord in a cable of a million neural wires called axons.  When it reaches the spinal cord it gets transformed into a really detailed muscle message and the nerves carry that message out to the muscles and cause a contraction.  So, for example, you think I want to pick up my cup of water; your hand goes out into space, it grabs the cup and, you know, the cup comes up to your mouth and you have a sip.  Now, one of the things that's interesting is up in the level of the brain, the brain appears to be formulating these commands in a very general way - some details - but all the real details of the complexity of controlling muscles seem to be taken care of down lower in the spinal cord and in another place in between called the brain stem.

SHANE HUNTINGTON
You mentioned earlier that some of the different motions and things that we do control in our brain are done by different parts of the brain.  How do you determine which part of the brain does what?

JOHN DONOGHUE
Well, the earliest studies of what was called brain localisation were done by placing an electrical stimulating electrode on the top of the brain on the surface during surgical procedures, say, to remove a problem of blood clot or something.  And, in as early as 1870 procedures were being done where stimulation was used to electrically excite the brain and it was found that this area I've talked about, which is called the motor cortex, when electrically stimulated the arm would jump or the leg would jump or the face would jump.  So this gave rise to what's called the localisationist theory of a brain organisation and this one strip, the motor cortex, was the place that was the most excitable to electrical stimuli.  Now we can even activate this from outside the head with a magnetic stimulator; if you place it over the right spot in your head you can use the magnetic stimulator and make your arm jump.

SHANE HUNTINGTON
In the case where a person is unable to move a limb - I realise there are probably many answers to this - but can you talk us through what's happening in terms of this communication not getting to that part of the body?  What's the most common sort of flaw there that's occurring?

JOHN DONOGHUE
So, based on the discussion we had, I think the easiest one to explain and probably the most familiar is spinal cord injury.  In spinal cord injury literally the most significant event in terms of thinking and moving is cutting what are called these corticospinal axons going from this motor cortex to the spinal cord, that million axon bundle of wires effectively, get cuts.  So literally the cable of communication between the brain and the spinal cord is cut off. So you can't communicate anymore.  So the commands can be generated in the brain, they just can't reach their target, very much like cutting the microphone line - you can still speak but it's not going to get out to your podcast.

SHANE HUNTINGTON
So the brain, in this case, is still functioning normally, the sort of job it's doing is unchanged?

JOHN DONOGHUE
Well, that was a question that was unknown.  Before we started the trial of our device, the BrainGate technology that we've developed, we didn't know that.  We were quite concerned that if the brain had been cut off from the spinal cord from the body for many years that, in fact, there'd be no message going on or the brain may fall silent or, by a principle called plasticity in which experience changes the brain, that the experience of this individual which would be so unusual and changed would cause the part of the brain that normally controls the arm to do something else.

SHANE HUNTINGTON
Are there a range of more naturally occurring problems that yield a similar result, simple ones like dyslexia or even aphasia, where that message scenario is not getting through correctly?

JOHN DONOGHUE
Well, in dyslexia and other disorders like that we don’t actually know how the message is deranged but we presume it is - that is, just the neurons are not speaking to each other in the way that things usually occur - but we don’t understand how that is occurring.  We've been able to look in humans, because these are human disorders, at a very global level with methods like fMRI, Functional Magnetic Resonance Imaging, which allows us to look from the outside at global changes in brain activity but not at the most detailed level.  Without looking at that most detailed level we really can't understand the mechanisms of those disorders like dyslexia.

SHANE HUNTINGTON
Robert, let me turn to you for a moment now and talk about the inputs to the brain.  There's obviously a vast amount of information being collected by the senses nonstop by the human body.  How does the brain sort through and deal with this incredible amount of data being pumped in constantly?

ROBERT SHEPHERD
Well, each sensory pathway is designed to receive that particular input through the auditory system.  Frequency is coded, in fact, at the level of the inner ear, the basilar membrane - which is the very fine membrane that vibrates in response to acoustic input - vibrates for different frequencies at different points along the basilar membrane.  So we're already coded frequency at the very peripheral level of the auditory brain.  That message then is conveyed through a series of neurons firing up along the auditory pathway in a very coordinated way.

SHANE HUNTINGTON
Is there a distinct difference between the type of information sent for smell, hearing, sight?

ROBERT SHEPHERD
As John pointed out, basically once that neural message is encoded within the neural firing patterns of that particular sensory pathway the process is very, very similar.  You cannot distinguish the neural activity within, say, the auditory nerve and the optic nerve unless you knew that you were presenting either acoustic or visual stimulation.  So the basic mechanisms of action potential propagation and the transmission across synapses are almost identical from one system to the other.  It's the timing and the location within the central auditory pathway or the visual pathway, particularly at the cortex level, that's the key that takes this information and presents it to us as perceptual information.

SHANE HUNTINGTON
One of the most impressive things, I think, about some of this work is just the type of signals we're talking about.  What sort of electrical signals would be comparable, I guess, in normal household environments to the sorts of signals we're talking about?  They're very small.

ROBERT SHEPHERD
These are tiny; these are thousandths of a volt and much smaller and the action potential lasts for a thousandth of a second.  So these are very tiny pieces of timing information that are being relayed up very specific localised populations of neurons.

SHANE HUNTINGTON
You're listening to Up Close coming to you from the University of Melbourne, Australia.  Our guests today are Professor John Donoghue and Professor Robert Shepherd and we're talking about neurotechnology.  John, your work is focused on enabling patients that are paralysed or have other problems to essentially interact again with their surrounding environment.  You explained earlier the process of movement.  What part of this are you trying to bypass or replace?

JOHN DONOGHUE
What we're trying to do is reconnect the brain with the outside world.  Basically, we jump over spots in the nervous system below the motor cortex that have had disruption of these pathways from the brain to the spinal cord.  So that could be a spinal cord injury, it could be a disease where the nerves degenerate ALS, amyotrophic lateral sclerosis - or Lou Gehrigs Disease, as it's often known - is a disease where the connections from the spinal cord out to the muscles degenerate from a really horrific disease, or even limb amputation leaves the brain intact but there's nothing to move any longer.  Those are examples of what we're trying to jump over; we're trying to get around or over that barrier and bridge from the brain to the outside world.  That could be to a computer, going directly from the brain to a computer, but it could also be to a robotic arm that could be an assistant for a paralysed person, to a prosthetic arm, so a replacement for the lost arm.  One of the exciting things that we're working on is potentially connecting the brain's signal back up to the muscles through an electrical stimulating device; so the brain commands the stimulating device, the stimulating device then causes muscles to contract.  So this would then restore movement, in effect, but by a very different route than we ordinarily move.

SHANE HUNTINGTON
How do you go about developing such a system?  I can imagine there are a lot of animal trials and modelling and so forth involved, but there seems to be an incredible level of complexity to overcome here.

JOHN DONOGHUE
Yes, certainly when you look at it from the outside it appears to be, and there are many complexities.  I think the development of these technologies is an example of the extraordinary success of funding basic science by governments, by agencies, by foundations that have provided the platform, the background we need in neuroscience.  So you ask me, where does the brain control the arm?  Well, we know that because we've done the fundamental science to do that.  How do we hear?  We know that because of the fundamental science that's behind that.  So years and years of development in fundamental neuroscience, but also here we've brought engineering in terms of make these very small devices, very complex devices that are compact can go inside the body, which is a very harsh environment.  And we now have, of course, technology has gotten smaller, faster, cheaper, less power hungry so it's really an extraordinary engineering advance.  Finally, computational algorithms that can take very large amounts of information and process them into something that we can use a control signal.  The bringing of those all together is also a big deal that requires multidisciplinary science which is something that has emerged considerably in the past decade or so.  That is also a barrier - different fields tend to think differently about problems.  I think, again, this is a success of basic science and now moving through what's called translational science to get, ultimately, these kinds of products into commercialisation.  And Rob, I'm sure, can tell you a lot about Cochlear, for example, an Australian company.  This is the hallmark of success; we have basic science and ideas that become technology that really help enable people to live better lives.

SHANE HUNTINGTON
What would such a device look like on a patient who formerly was unable to walk, you know, wandering around with this device - presumably, partly inside and partly outside the body, similar to Cochlear?


JOHN DONOGHUE
Well, let's start with the way the device looks today.  So today is a very early stage, what we call a pilot device.  There's a tiny sensor in the brain that's actually quite small, the size of a baby aspirin 4mm x 4mm, sits on top of the brain.  Once it's implanted what's left is a plug on the top of the head that has to go through the system; in order for the system to work we connect a sort of matchbox size connector plug on top of the head.  There's a big cable going to something about the size of an apartment refrigerator that has computers and processors that literally tethers the person to that location.  So this is very big, but the big challenging next steps will be to miniaturise all this and get it inside the body.  These things are underway and thanks to advances in electronics we can, in fact, reduce the size of these technologies and we're hoping to get the implantable device down to the size of your thumbnail with all of its electronics wirelessly transmitting through the skin and then something that's got the processors to take the signals to various devices like computers in package something the size of an iPhone.  All this can be done but we're busy doing these things.

SHANE HUNTINGTON
What sort of function have you managed to restore at this point in time?

JOHN DONOGHUE
We have demonstrated that it's possible for individuals who are paralysed to use signals from their brain to control a cursor, say, to open an email, to select icons on a computer like we do, to do point and click actions, to control a very simple robotic arm, to, say, grab a piece of candy and hand it to someone and to even control a wheelchair, just as a demonstration to show that the signals can be used to, say, move an object - not with a person in it but just to show that it can be done.  We have also done very early stage demonstrations of the possibility of controlling muscles with these brain signals.  We do it in a simulation, a computer simulation, because the actual procedure would require both a BrainGate implant and stimulating electrodes and we're not ready to do that yet.

SHANE HUNTINGTON
I suspect whenever you talk to people about this their minds immediately go to things like movement, but there are many other bodily functions that presumably are affected by the signals not getting through from the brain, be it your ability to control bladder function or a variety of other areas.  Are these of interest as well in terms of the exact same technology?

JOHN DONOGHUE
Exactly, every output of the nervous system is through muscles.  So one muscle is, in our case, similar to another; you issue a signal from the nervous system and then control that muscle.  Bowel and bladder control is actually a very serious problem for individuals who have spinal cord injury.  If the bladder is not emptied it can burst the bladder, it can result in infection and infections can be deadly.  So having voluntary control over these, or having some level of control over these functions is extremely useful.  There are individuals working on the signals necessary to control the bladder, which turns out to be quite complicated for such a simple function.  But, yes, I think the same approach could be used for all of these functions.

SHANE HUNTINGTON
What sort of things should we be looking for in the next sort of 10 years out of this work?

JOHN DONOGHUE
Well, my real hope in 10 years is that you will be seeing individuals who are paralysed, cannot used their arms, reaching out and grabbing a glass of water and taking a sip probably in a device that's a little bit clunkier than you'd like to see,  their arm will be supported a bit.  But, in fact, under their own control they will reach out probably slower than normal and take, as I say, a piece of food or a drink of water and be able to help themselves which, you will recall, people who are paralysed can't do at all, they depend on others to do all of those functions.

SHANE HUNTINGTON
You're listening to Up Close coming to you from the University of Melbourne, Australia.  Today our guests are Professor John Donoghue and Professor Robert Shepherd and we're talking about neurotechnology.  Robert, there are many people around the world who obviously have incredible experiences as a result of the Bionic Ear work that you've been involved with over the years, and we know you're now turning your attention to vision and the, I guess, construction of bionic systems for the eye.  How does the Australian approach, this newly funded approach, to this project differ from those that are already going on around the world?

ROBERT SHEPHERD
Shane, that's a good question.  None of us - and you've heard from John's discussion about BrainGate - no one group has all the technology.  So one of the really exciting things about working in this field is that each discipline brings to the table a key piece of the jigsaw, and by working in a very collaborative way across disciplines we can put the big picture together by making sure that we put that jigsaw together in a correct way.  So we're working with ophthalmologists and retinal surgeons, who must play a key role at a very early stage in ensuring we develop the appropriate surgical techniques, our engineering devices are not inappropriate from a surgical approach and that they consider it a safe and reliable surgical approach.  We're also doing the essential pre-clinical trials to make sure that these devices are safe, in animal studies, and working very, very closely with the engineering groups who are helping provide the significant electrode arrays - these are a high number of electrodes in these arrays for a bionic eye.

SHANE HUNTINGTON
In the case of the cochlear implant, we never replaced the entire ear at such.  With the bionic eye are we talking about just a component of the eye being essentially replaced by the implant?

ROBERT SHEPHERD
Exactly.  In the cochlear implant profoundly deaf patients have lost their sensory hair cells, these are the hair cells that would normally convert the mechanical vibrations of sound into nerve impulses, so we're directly stimulating the auditory nerve.  In the retinal implant we will be implanting these devices in blind patients who have lost the photoreceptors, the cells that would normally convert light into nerve impulses within the optic nerve.  So the technology is very similar but it's also very different because of the complexities of the anatomy of the different systems so the electrode arrays must be different.  In the case of the bionic eye, instead of having 22 electrodes we will need hundreds or even thousands of electrodes.

SHANE HUNTINGTON
Which leads into my next question which was how the knowledge you've gained over a decade in developing the cochlear implant will be used in the formation of the new bionic eye implants?

ROBERT SHEPHERD
Yeah, well that knowledge is really important.  We know how to safely electrically stimulate neural tissue and, as we talked about before, individual neurons within the brain react remarkable similar, so that knowledge can be transferred from one area - the auditory nerve - to the optic nerve.  What we need to be able to do now is to apply that knowledge with appropriate electrodes and appropriate electrical stimulation.  So the most important thing is to initiate the safe device and demonstrate that it's safe in preliminary animal studies, and then implant in a small number of patients, as John's doing with BrainGate.  Because once we demonstrate its effectiveness in patients and we get the psychophysical performance from patients, i.e. what they see with an electrical stimulus, we can then start developing sophisticated visual processing strategies.

SHANE HUNTINGTON
When you talk about what they see, what is the goal there, what are we trying to achieve?  Is this a mere scenario where they're essentially seeing shadows light and dark or are we talking about something far more sophisticated?

ROBERT SHEPHERD
Well, initially we will be very excited if they can determine light and dark; for a blind person that's very important, that certainly helps with the diurnal rhythms.  But we believe that with larger numbers of electrodes we can provide the ability for patients to walk unassisted and, eventually, have potentially face recognition.  We've got to recall that when we initially developed a cochlear implant we called it an aid to lip reading.  It still is an aid to lip reading but many of our patients, as a result of brain plasticity, have got so used to the signals that are coming in through a cochlear implant that their performance is far greater than any of our expectations; so they talk on the phone and their performances are outstanding.  So our task is to provide a behaviourally relevant signal to the periphery to the eye and let the brain, through brain plasticity, take advantage of that and these patients, we're confident, will do very well.

SHANE HUNTINGTON
In fact, some of our listeners will remember the last time you were on we actually gave examples of the progress that had been made with regard to processing and what people were hearing as a result of the cochlear implants.  Unfortunately, we won't be able to give similar demonstrations down the track of the visual performance but, certainly, it's an exciting area.  With regards to the signal that comes out of the implant, what happens at that point?  There's obviously a lot of processing and something has to go to the brain that can be understood.  How does that work?

ROBERT SHEPHERD
We will use tiny platinum electrodes to deliver what we call - very small pulses of electricity - we call them charge balanced biphasic current pulses.  That's a safe way of delivering charge to any neural tissue.  The neuron adjacent to the electrode will then be depolarised, as it would be depolarised through the photoreceptor cell in the normal visual process.  Once the neurons depolarised, in our case artificially through an electrical stimulus, it will then conduct the action potentials that we've been talking about earlier in a normal physiological manner and relay that signal through a series of pathways to the visual cortex.  The patient will perceive what we call a phosphene, a small visual percept.

SHANE HUNTINGTON
When we view things normally there are a range of ways in which our vision and our brain can be confused, can be essentially led to believe one thing when we're actually, in essence, seeing something else.  Do we learn from these flaws in the way our vision system works and will the bionic system mimic those flaws or avoid them?

ROBERT SHEPHERD
Absolutely the brain will learn because the human brain is designed to extract as much information from the sensory environment as it can - that's a fundamental in plasticity in learning.  We have many challenges, though, because our eyes are always continually moving and the centre of our eye has a very high resolution; our challenge in a bionic eye is to try and reproduce that in some type of visual processing.  And we really have some ideas but we don’t know how to implement that type of strategy until we implant a small number of patients and do some pilot psychophysical studies on those patients - they provide us with feedback on exactly what they're seeing.

SHANE HUNTINGTON
You mentioned the patients giving you feedback.  How does a patient who hasn't seen before give you feedback on whether or not this is working?

ROBERT SHEPHERD
Initially our patients will all have prior visual experience. So the patients we're mostly targeting are patients with age-related macular degeneration and retinitis pigmentosa.  So they've all had vision until at least young adulthood, so their brain is very familiar with the visual world.  However, our experience with cochlear implants show that the auditory brain in young children who have had no auditory experience before can use that incoming information through a cochlear implant and perform very, very well.  Again it's a result of plasticity and the brain is maximally plastic in a young person.  At the Eye and Ear Hospital children as young as six months of age were implanted with these devices, with these cochlear implants, and the extraordinary thing is that their language development is typically at the same rate as a normal hearing child.  Their brain is a sponge for information and they're programmed for language development.

SHANE HUNTINGTON
Assuming you have the same incredible success you've had with Cochlear, what sort of timeframe are we talking about before you start human trials?

ROBERT SHEPHERD
We anticipate we'll be ready for a clinical trial within two years because we've already done, with our colleagues in Bionic Vision Australia, a considerable amount of work in that area, particularly the engineers at the University of New South Wales.  The second and 'blue sky' device is a thousand electrode device - it's going to be based on boron diamond and those electrodes are being developed in the School of Physics here at Melbourne University - we anticipate that a clinical trial will be ready at the end of four years based on that device, although there's a huge amount of work to be done over the next three years to achieve that.

SHANE HUNTINGTON
Professor John Donoghue from Brown University and Professor Robert Shepherd from the University of Melbourne, I thank you very much for being our guests on Up Close today.  It's exciting work and I know a lot of people are really hedging their hopes on the success of what both of you are doing and we wish you every bit of luck.


ROBERT SHEPHERD
Thanks, Shane.

JOHN DONOGHUE
Thank you.

SHANE HUNTINGTON
Relevant links, a full transcript and more info on this episode can be found at our website at upclose.unimelb.edu.au.  We also invite you to leave your comments or feedback on this or any episode of Up Close, simply click on the Add New Comment link at the bottom of the episode page.  Up Close is brought to you by marketing and communications of the University of Melbourne, Australia.  Our producers for this episode were Kelvin Param and Eric van Bemmel.  Audio engineering by Ben Loveridge.  Up Close is created by Eric van Bemmel and Kelvin Param.  I'm Shane Huntington.  Until next time, goodbye.

VOICEOVER
You’ve been listening to Up Close.  For more information visit upclose.u-n-i-m-e-l-b.edu.au.  Copyright 2010 The University of Melbourne.


show transcript | print transcript | download pdf