Six Paths to the Nonsurgical Future of Brain-Machine Interfaces(darpa.mil)
darpa.mil
Six Paths to the Nonsurgical Future of Brain-Machine Interfaces
https://www.darpa.mil/news-events/2019-05-20
7 comments
The neural links are intended to be bi-directional. While they can't 'write' to the brain, they could provide sensory inputs or thoughts to the user. That seems a little more risky than read-only. (I don't know what I'm talking about though.)
It's about as bad as any other technology. I bet therapy gets a lot easier when I can write my anxiety reduction function for myself.
A malicious firmware update that commands the prosthetic to punch you in the face on an infinite loop will still be a threat. Updates will likely be necessary for a while as the ML stuff gets tuned, though it's not impossible to get the other parts of the controlling software correct the first time and make those read-only.
Also I expect the medical device manufacturers will most likely join this market eventually and they have a less-than-stellar record of security with their devices. Some company is going to connect it to the internet and regulation will be too glacially slow to stop it.
Also I expect the medical device manufacturers will most likely join this market eventually and they have a less-than-stellar record of security with their devices. Some company is going to connect it to the internet and regulation will be too glacially slow to stop it.
That fear feels overblown to me. This isn’t some autonomous vehicle, it’s a learned mapping of electrical signals to specific motions. I don’t see a need for over the air firmware updates. And the model can’t generalize because everyone’s brain works differently so you’ll need to retrain every single time.
Perhaps I wasn’t clear but that technology already exists and is in use. It’s much easier to do prosthetics because you can just hook it up to the nerves on former limb stub.
Perhaps I wasn’t clear but that technology already exists and is in use. It’s much easier to do prosthetics because you can just hook it up to the nerves on former limb stub.
>I don’t see a need for over the air firmware updates.
There isn't a need, but if the same thing happens here that happened everywhere else in the tech industry, there will be a motivation for over-the-air firmware updates. If the average person won't install adblock because they don't realize the end result of ad network tracking, why would they object to a prosthetic that records their every move? Time and time again, the public has shown that they have no practical aversion to being creeped on and abused in ways that don't immediately translate to pain or lost money.
There isn't a need, but if the same thing happens here that happened everywhere else in the tech industry, there will be a motivation for over-the-air firmware updates. If the average person won't install adblock because they don't realize the end result of ad network tracking, why would they object to a prosthetic that records their every move? Time and time again, the public has shown that they have no practical aversion to being creeped on and abused in ways that don't immediately translate to pain or lost money.
I'm thinking when it becomes more widely available. Companies will inevitably try to reduce costs and that likely means firmware updates that aren't delivered by technicians (OTA or downloaded and installed by users).
This is why SF should be mandatory reading in high school. People can't even grasp the consequences of tracking cookies and we are going to sell them stuff to plug in their head.
>we are going to sell them stuff to plug in their head.
Correction, we are going to give away free stuff to plug in their head, with advertising. We'll also sell high end without marketing impulse technology (TM), but that will be a minor sector.
Correction, we are going to give away free stuff to plug in their head, with advertising. We'll also sell high end without marketing impulse technology (TM), but that will be a minor sector.
Why sell advertising when you can directly trigger the pleasure center whenever the user thinks of the brand?
That's what we'll call advertising.
These guys read loads of SF, hire them for the marketting dep.
Nahh, it just appears in my head from somewhere in the cosmos via beams of purple light.
You mean like the free booze we get at company events?
I have worked in the field for 8 years before. This technology is far too immature to be useful beyond medical application. One day it might be used for reading or writing thoughts, but I am quite convinced that this is decades away. But really useful prosthetics are in our reach and such technology could transform many lives for the better.
In hyperion, the AI are very useful to the general population...
Safety nets should be placed now, not as an afterthough.
Safety nets should be placed now, not as an afterthough.
> In hyperion, the AI are very useful to the general population...
Yes, but there is also a large gap between a general purpose AI capable of generating teleportation portals and a device barely capable of discerning electronic noise from the brain signals.
> Safety nets should be placed now, not as an afterthough.
I agree. However:
1. Many of people working on this do read a lot of SF.
2. Initial motivation for BCIs is to bring the dream of being able to walk/talk/interact with the world to people who were deprived from it.
3. The whole purpose of non invasive interfaces is to not need to jack anything in. This is important mainly because there are prohibitive safety nets around anything that goes inside a person's body.
Especially in this case do not overestimate the usefulness of reading brain waves. For example, in absence of disease, it is currently impossible to identify people using their EEG.
This being said, I am somewhat concerned about the writing part, mostly because I have seen experimental protocols which seemed to hand wave quite a lot of potential issues.
Yes, but there is also a large gap between a general purpose AI capable of generating teleportation portals and a device barely capable of discerning electronic noise from the brain signals.
> Safety nets should be placed now, not as an afterthough.
I agree. However:
1. Many of people working on this do read a lot of SF.
2. Initial motivation for BCIs is to bring the dream of being able to walk/talk/interact with the world to people who were deprived from it.
3. The whole purpose of non invasive interfaces is to not need to jack anything in. This is important mainly because there are prohibitive safety nets around anything that goes inside a person's body.
Especially in this case do not overestimate the usefulness of reading brain waves. For example, in absence of disease, it is currently impossible to identify people using their EEG.
This being said, I am somewhat concerned about the writing part, mostly because I have seen experimental protocols which seemed to hand wave quite a lot of potential issues.
What is SF?
Science fiction. The poster you're responding to I think means that the common philosophical issues brought up in science fiction aren't being thought of by the creators of this technology in advance.
Basically, they're too preoccupied with whether they could, and not thinking about whether they should.
Basically, they're too preoccupied with whether they could, and not thinking about whether they should.
Product design's first draft.
[deleted]
[deleted]
As a clinician, this news is extraordinarily exciting. Here's why.
A brain computer interface is something that records neural activity in a person, and then decodes that neural activity to allow them to control objects in their environment. When we think about sources of neural information, it's useful to divide them into non-invasive (EEG, fMRI, MEG, NIRS) technology, and to invasive technology (intracortical electrodes, SEEG, ECoG) systems.
Fundamentally, the DARPA grant is about acquiring new sources of neural information. As the article points out, the latter category is happening almost exclusively as people are hanging out in epilepsy monitoring units with nothing better to do, or as part of early clinical trials looking at safety profiles of implanted devices.
Simply speaking, the divide between invasive and non-invasive systems is about the signal-to-noise ratio of neural information. The closer you get to single neurons, the closer you get to the source of neural activity, the higher the signal quality you get to decode with. Consider that EEG systems (the most commonly used non-invasive methods) average neural activity over the range of several 10^-3 to 10^-2 meters, through the attenuation of spinal fluid, dura mater, skull, five scalp layers, and the electrode interface. Electrodes used in modern intracortical BCI systems are 10^-5 meters in diameter.
To my mind, the DARPA announcement is extraordinary -- these technologies have the possibility of upping the SNR for non-invasive methods, and acquiring fundamentally different sources of neural activity. Importantly, the methods seem to have the ability to work at the bedside, without needing very large and very expensive devices (e.g. fMRI, MEG).
Here are some reasons why this would be important:
1) Someone has been admitted to the intensive care unit, and needs a breathing tube on a ventilator for life saving purposes. Some people in this situation are awake and alert. Literature suggests that being intubated and awake is incredibly scary, since you're aware but can't communicate. Having a non-invasive beside method that could decode your thoughts would be a dramatic increase to the person's quality of life, given their temporary stay. Note that eye-trackers may not be feasible, given the amount of stuff in the ICU room, the fact that they're lying down, etc.
2) If you ask people with locked in syndrome them about their priorities for improving their quality of life, the biggest need is improving their ability to communicate (as opposed to spelling boards, or looking up/down to communicate). Note these people cannot use eye-trackers, due to their neurologic condition.
3) Ongoing clinical trials have had volunteers undergo implantation of electrodes into their brain. The devices implanted have been percutaneous (i.e. had a portion go through their skin). These devices have allowed some participants to feed themselves, either using muscle stimulation or through robotic arm control. Not only would having a non-invasive method obviate the need for a surgery, but having a method for "writing into" the brain would provide closed-loop control of the arm/robotic limb they were controlling, putatively increasing the quality of control.
A brain computer interface is something that records neural activity in a person, and then decodes that neural activity to allow them to control objects in their environment. When we think about sources of neural information, it's useful to divide them into non-invasive (EEG, fMRI, MEG, NIRS) technology, and to invasive technology (intracortical electrodes, SEEG, ECoG) systems.
Fundamentally, the DARPA grant is about acquiring new sources of neural information. As the article points out, the latter category is happening almost exclusively as people are hanging out in epilepsy monitoring units with nothing better to do, or as part of early clinical trials looking at safety profiles of implanted devices.
Simply speaking, the divide between invasive and non-invasive systems is about the signal-to-noise ratio of neural information. The closer you get to single neurons, the closer you get to the source of neural activity, the higher the signal quality you get to decode with. Consider that EEG systems (the most commonly used non-invasive methods) average neural activity over the range of several 10^-3 to 10^-2 meters, through the attenuation of spinal fluid, dura mater, skull, five scalp layers, and the electrode interface. Electrodes used in modern intracortical BCI systems are 10^-5 meters in diameter.
To my mind, the DARPA announcement is extraordinary -- these technologies have the possibility of upping the SNR for non-invasive methods, and acquiring fundamentally different sources of neural activity. Importantly, the methods seem to have the ability to work at the bedside, without needing very large and very expensive devices (e.g. fMRI, MEG).
Here are some reasons why this would be important:
1) Someone has been admitted to the intensive care unit, and needs a breathing tube on a ventilator for life saving purposes. Some people in this situation are awake and alert. Literature suggests that being intubated and awake is incredibly scary, since you're aware but can't communicate. Having a non-invasive beside method that could decode your thoughts would be a dramatic increase to the person's quality of life, given their temporary stay. Note that eye-trackers may not be feasible, given the amount of stuff in the ICU room, the fact that they're lying down, etc.
2) If you ask people with locked in syndrome them about their priorities for improving their quality of life, the biggest need is improving their ability to communicate (as opposed to spelling boards, or looking up/down to communicate). Note these people cannot use eye-trackers, due to their neurologic condition.
3) Ongoing clinical trials have had volunteers undergo implantation of electrodes into their brain. The devices implanted have been percutaneous (i.e. had a portion go through their skin). These devices have allowed some participants to feed themselves, either using muscle stimulation or through robotic arm control. Not only would having a non-invasive method obviate the need for a surgery, but having a method for "writing into" the brain would provide closed-loop control of the arm/robotic limb they were controlling, putatively increasing the quality of control.
These are all interesting approaches. I m surprised there is nothing related to optogenetics, but i guess that is considered too invasive. In reality once we are inside the brain it might prove to be easier to deliver signals, because our brains can learn to "read" a specific sense. This is essentially about creating a sixth sense to humans which can deliver information in new forms to the brain.
You would need to engineer cells for optogenetic. So unless you do it at the embryo stage, or have very efficient and robust delivery and editing mechanisms, it will not work.
“To enable the write function, the team will use a magneto-genetic approach to make neurons sensitive to magnetic fields.“
opsin-based approaches for cochlear implants have been talked about for a decade or more by now.
if that’s not a BCI i don’t know what is.
if that’s not a BCI i don’t know what is.
As long as I can avoid synaptic seepage!
Is this a reference to something cyberpunky?
Let's see, for such communications we already have eyes, ears, fingers, and voice with all or nearly all of these well proven from most mammals from the smallest mouse to the largest elephant with development time 65+ million years.
And with those systems given, DARPA is going to try to do better, just externally, by using sound, optics, magnetic fields, etc.?
They are NOT going to put one of their devices anywhere near MY head!
Ah, DARPA has done some good stuff -- they are allowed some big losers!
Old Advice: The first step in good research is to pick a good problem!
And with those systems given, DARPA is going to try to do better, just externally, by using sound, optics, magnetic fields, etc.?
They are NOT going to put one of their devices anywhere near MY head!
Ah, DARPA has done some good stuff -- they are allowed some big losers!
Old Advice: The first step in good research is to pick a good problem!
If you like science fiction at all, give Accelerando a read. Of course it's just a story, but it feels like a possible real future (at least the first section). In it, there is definitely a place for people who reject integrating with technology, but that place is in the back and left behind. Instant access to all of digital knowledge, unlimited storage for every thought or experience that interests you, performing complicated calculations, visualizing higher dimensions, and so on. Your fingers, ears, and voice aren't that quick, and your eyes are limited to 2D and only have decent resolution around the center. There is certainly room for improvement over what evolved in mammals (or any animal).
It probably won't be real before I die of old age, and I'd certainly let the early adopters find the bugs, but sign me up for version 3.0 or later if I'm still around.
It probably won't be real before I die of old age, and I'd certainly let the early adopters find the bugs, but sign me up for version 3.0 or later if I'm still around.
If I had any faith in the people implementing the software and the people funding the venture, then sure. I would love to live in the bright cheery science fiction future. But at this point I expect anyone with the money to make a proper implementation of the idea will happily and intentionally create something downright abusive and exploitative to the so-called "user". Anyone with the love to make something that would actually genuinely improve the human experience will be relegated to contributing to a half built janky implementation that has to reverse engineer the chip drivers. I would feel lucky to live in the future where you can actually own the chip in your head AND turn off thought analytics.
Yeah, I'm guilty of being pretty cynical/pessimistic from time to time too. However, despite all the nastiness and privacy invading features of the technical world, a lot of things really have just gotten better over time. Maybe just don't run "Braindows 10" or "BrainOS X" with automatic updates.
> Instant access to all of digital knowledge, ...
Yup, I agree. Indeed, that's the core goal of my startup, helping people find that stuff -- for a huge fraction of the content, tough to do now.
Sure, but as I mentioned in another response here, eyes are deep into both the design and the construction of the brain. Sooo, super tough to have a better intake channel.
To be fair to DARPA, my guess is that they have no intention of improving on eyes, ears, etc. but just augmenting them. I didn't mention it, but my reaction would be just go ahead and use just eyes and ears but give them, say, a better user interface.
Yup, I agree. Indeed, that's the core goal of my startup, helping people find that stuff -- for a huge fraction of the content, tough to do now.
Sure, but as I mentioned in another response here, eyes are deep into both the design and the construction of the brain. Sooo, super tough to have a better intake channel.
To be fair to DARPA, my guess is that they have no intention of improving on eyes, ears, etc. but just augmenting them. I didn't mention it, but my reaction would be just go ahead and use just eyes and ears but give them, say, a better user interface.
> eyes are deep into both the design and the construction of the brain. Sooo, super tough to have a better intake channel.
Maybe the optical processing parts of the brain are where they'll inject a lot of the new interfaces. Have you ever had the sensation, when you're really focused on a problem and thinking hard about it, of not being able to see while you're concentrating? To me, it feels like your brain is /borrowing/ the optical lobes for other purposes at that time. I don't have any background in neurology, but it's fun to think about regardless.
Maybe the optical processing parts of the brain are where they'll inject a lot of the new interfaces. Have you ever had the sensation, when you're really focused on a problem and thinking hard about it, of not being able to see while you're concentrating? To me, it feels like your brain is /borrowing/ the optical lobes for other purposes at that time. I don't have any background in neurology, but it's fun to think about regardless.
> "Accelerando"
Singularity book 3 by Stross?
Singularity book 3 by Stross?
By Stross, yes. And it’s online: http://www.antipope.org/charlie/blog-static/fiction/accelera...
Stross, yes. Not sure if it was book 3 of a series or something. I didn't read much else of his.
Not directly related to DARPA, but there are many people who have impairment in one or more of those senses. BCI technology could be transformative to the people who might not otherwise be able to access computers, even with best-in-class assistive technologies. Imagine opening a communication channel with someone who has locked-in syndrome. I just wish we could pour at least as many resources into researching humanistic technology as we do on militaristic "futures".
the idea that this stuff is going to be primarily used for disabled people is, frankly, bullshit. Maybe the grant writing process, and some of the research, but this will be commodified, indeed, things like this already are; check out the emotiv headset. There are hundreds of millions of VC dollars pouring into this right now, they aren't doing it to help locked-in syndrome, whatever they say in interviews or press releases, they intend to get returns on their investments. It drives me absolutely bonkers when I see or hear irl people using the misfortune of a tiny, tiny subset of the population to defend this. And while I appreciate the note at the end about military futures (and I agree!), the idea that there is any "humanistic" use for tech like this is ... blinkered, to put it kindly. No one needs this, at all.
Let's recontextualize that for you. Replace "this technology" with the mouse. No one needs a new way to interact with devices, information, etc indeed.
Holding such luddite views is your right, but the rest of the more transhumanist aligned population can and will leave you behind (and help less abled people along the way).
Holding such luddite views is your right, but the rest of the more transhumanist aligned population can and will leave you behind (and help less abled people along the way).
I'm for doing better than a mouse, but tough to improve on fingers, do better than HTML on a screen, but tough to do better than eyes, do better than a pair of speakers for $30 but tough to improve on ears. Augment? Maybe. Really improve as in replace? For something external? No way.
You got me there.
Sure, the research will likely at least clarify the challenges. For actually being very successful for the military purposes in the four years of the program, I'd say little to no hope.
But where you got me is even if the military goals are not achieved in the four years, maybe there will be some progress for what you mention. And even if there is no such progress, go ahead and spend the money, if only to clarify the challenges also for the non-military hope.
In effect, you have partially made my point: The DARPA program is for healthy people, say, super healthy airplane fighter pilots, each with a $100+ million office. As I noted, for them, they already have eyes, ears, .... And, I didn't mention but obvious, the mammalian visual system, and there fighter pilots are especially good, is deep into the brain and its design so that we can anticipate that nothing just external can hope to compete. Same for hearing. Much the same for speech. But, again, that holds for super healthy fighter pilots. For people without sight, hearing, etc. as you mentioned, the potential is much easier to believe in and much easier to justify whatever the chances.
I don't always laugh at DARPA: (1) They played a constructive role in making TCP/IP real, and now that it has gone commercial, with the rest of the technology of the Internet, it is one of the bigger steps up in civilization. (2) They pushed self-driving vehicles. My view is that for current traffic on current US public roads, self-driving, i.e., without a human right there, is hopeless for a very long time. BUT!!! What DARPA did might actually be good enough for 1000 army trucks charging across a road in a desert in Saudi Arabia as in Gulf War I.
Sure, the research will likely at least clarify the challenges. For actually being very successful for the military purposes in the four years of the program, I'd say little to no hope.
But where you got me is even if the military goals are not achieved in the four years, maybe there will be some progress for what you mention. And even if there is no such progress, go ahead and spend the money, if only to clarify the challenges also for the non-military hope.
In effect, you have partially made my point: The DARPA program is for healthy people, say, super healthy airplane fighter pilots, each with a $100+ million office. As I noted, for them, they already have eyes, ears, .... And, I didn't mention but obvious, the mammalian visual system, and there fighter pilots are especially good, is deep into the brain and its design so that we can anticipate that nothing just external can hope to compete. Same for hearing. Much the same for speech. But, again, that holds for super healthy fighter pilots. For people without sight, hearing, etc. as you mentioned, the potential is much easier to believe in and much easier to justify whatever the chances.
I don't always laugh at DARPA: (1) They played a constructive role in making TCP/IP real, and now that it has gone commercial, with the rest of the technology of the Internet, it is one of the bigger steps up in civilization. (2) They pushed self-driving vehicles. My view is that for current traffic on current US public roads, self-driving, i.e., without a human right there, is hopeless for a very long time. BUT!!! What DARPA did might actually be good enough for 1000 army trucks charging across a road in a desert in Saudi Arabia as in Gulf War I.
The Battelle team, under principal investigator Dr. Gaurav Sharma, aims to develop a minutely invasive interface system that pairs an external transceiver with electromagnetic nanotransducers that are nonsurgically delivered to neurons of interest. The nanotransducers would convert electrical signals from the neurons into magnetic signals that can be recorded and processed by the external transceiver, and vice versa, to enable bidirectional communication.
The Carnegie Mellon University team, under principal investigator Dr. Pulkit Grover, aims to develop a completely noninvasive device that uses an acousto-optical approach to record from the brain and interfering electrical fields to write to specific neurons. The team will use ultrasound waves to guide light into and out of the brain to detect neural activity. The team’s write approach exploits the non-linear response of neurons to electric fields to enable localized stimulation of specific cell types.
The Johns Hopkins University Applied Physics Laboratory team, under principal investigator Dr. David Blodgett, aims to develop a completely noninvasive, coherent optical system for recording from the brain. The system will directly measure optical path-length changes in neural tissue that correlate with neural activity.
The PARC team, under principal investigator Dr. Krishnan Thyagarajan, aims to develop a completely noninvasive acousto-magnetic device for writing to the brain. Their approach pairs ultrasound waves with magnetic fields to generate localized electric currents for neuromodulation. The hybrid approach offers the potential for localized neuromodulation deeper in the brain.
The Rice University team, under principal investigator Dr. Jacob Robinson, aims to develop a minutely invasive, bidirectional system for recording from and writing to the brain. For the recording function, the interface will use diffuse optical tomography to infer neural activity by measuring light scattering in neural tissue. To enable the write function, the team will use a magneto-genetic approach to make neurons sensitive to magnetic fields.
The Teledyne team, under principal investigator Dr. Patrick Connolly, aims to develop a completely noninvasive, integrated device that uses micro optically pumped magnetometers to detect small, localized magnetic fields that correlate with neural activity. The team will use focused ultrasound for writing to neurons.
The Carnegie Mellon University team, under principal investigator Dr. Pulkit Grover, aims to develop a completely noninvasive device that uses an acousto-optical approach to record from the brain and interfering electrical fields to write to specific neurons. The team will use ultrasound waves to guide light into and out of the brain to detect neural activity. The team’s write approach exploits the non-linear response of neurons to electric fields to enable localized stimulation of specific cell types.
The Johns Hopkins University Applied Physics Laboratory team, under principal investigator Dr. David Blodgett, aims to develop a completely noninvasive, coherent optical system for recording from the brain. The system will directly measure optical path-length changes in neural tissue that correlate with neural activity.
The PARC team, under principal investigator Dr. Krishnan Thyagarajan, aims to develop a completely noninvasive acousto-magnetic device for writing to the brain. Their approach pairs ultrasound waves with magnetic fields to generate localized electric currents for neuromodulation. The hybrid approach offers the potential for localized neuromodulation deeper in the brain.
The Rice University team, under principal investigator Dr. Jacob Robinson, aims to develop a minutely invasive, bidirectional system for recording from and writing to the brain. For the recording function, the interface will use diffuse optical tomography to infer neural activity by measuring light scattering in neural tissue. To enable the write function, the team will use a magneto-genetic approach to make neurons sensitive to magnetic fields.
The Teledyne team, under principal investigator Dr. Patrick Connolly, aims to develop a completely noninvasive, integrated device that uses micro optically pumped magnetometers to detect small, localized magnetic fields that correlate with neural activity. The team will use focused ultrasound for writing to neurons.
None of those sound non invasive. How do they get light through the skull?
Typically, it is not wavelengths from the visual light spectrum. It is common to use infrared. For example, high-power infrared light can penetration at least 3 cm into the brain.
Uhh, I'm sure it's fine, but how high power are we talking?
Just when you say "high-power infrared light" my brain thinks "cooking an egg"
Would that not risk heating the brain and injuring the neurons?
"Non invasive" in this context means not physically opening the skull and inserting a device.
One thing I was surprised to learn recently was that mentally controlled prosthetics are simply machine learning algorithms trained to interpret the electrical signals and produce physical outcomes. In hindsight that seems obvious but before that I had assumed something more complicated. I think a lot of great stuff could come from this. Especially art. Either generating things that the device observes you find enjoyable or generating music you’re mentally composing could be awesome. Imagery could be great.