Publications by authors named "Jonathan C Kao"

37 Publications

An artificial intelligence that increases simulated brain-computer interface performance.

J Neural Eng 2021 05 13;18(4). Epub 2021 May 13.

Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90024, United States of America.

Brain-computer interfaces (BCIs) translate neural activity into control signals for assistive devices in order to help people with motor disabilities communicate effectively. In this work, we introduce a new BCI architecture that improves control of a BCI computer cursor to type on a virtual keyboard.Our BCI architecture incorporates an external artificial intelligence (AI) that beneficially augments the movement trajectories of the BCI. This AI-BCI leverages past user actions, at both long (100 s of seconds ago) and short (100 s of milliseconds ago) timescales, to modify the BCI's trajectories.We tested our AI-BCI in a closed-loop BCI simulator with nine human subjects performing a typing task. We demonstrate that our AI-BCI achieves: (1) categorically higher information communication rates, (2) quicker ballistic movements between targets, (3) improved precision control to 'dial in' on targets, and (4) more efficient movement trajectories. We further show that our AI-BCI increases performance across a wide control quality spectrum from poor to proficient control.This AI-BCI architecture, by increasing BCI performance across all key metrics evaluated, may increase the clinical viability of BCI systems.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1088/1741-2552/abfaaaDOI Listing
May 2021

Dorsal periaqueductal gray ensembles represent approach and avoidance states.

Elife 2021 May 6;10. Epub 2021 May 6.

Department of Psychology, University of California, Los Angeles, Los Angeles, United States.

Animals must balance needs to approach threats for risk assessment and to avoid danger. The dorsal periaqueductal gray (dPAG) controls defensive behaviors, but it is unknown how it represents states associated with threat approach and avoidance. We identified a dPAG threatavoidance ensemble in mice that showed higher activity farther from threats such as the open arms of the elevated plus maze and a predator. These cells were also more active during threat avoidance behaviors such as escape and freezing, even though these behaviors have antagonistic motor output. Conversely, the threat approach ensemble was more active during risk assessment behaviors and near threats. Furthermore, unsupervised methods showed that avoidance/approach states were encoded with shared activity patterns across threats. Lastly, the relative number of cells in each ensemble predicted threat avoidance across mice. Thus, dPAG ensembles dynamically encode threat approach and avoidance states, providing a flexible mechanism to balance risk assessment and danger avoidance.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.7554/eLife.64934DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8133778PMC
May 2021

Shared Dorsal Periaqueductal Gray Activation Patterns during Exposure to Innate and Conditioned Threats.

J Neurosci 2021 Jun 21;41(25):5399-5420. Epub 2021 Apr 21.

Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095

The brainstem dorsal periaqueductal gray (dPAG) has been widely recognized as being a vital node orchestrating the responses to innate threats. Intriguingly, recent evidence also shows that the dPAG mediates defensive responses to fear conditioned contexts. However, it is unknown whether the dPAG displays independent or shared patterns of activation during exposure to innate and conditioned threats. It is also unclear how dPAG ensembles encode and predict diverse defensive behaviors. To address this question, we used miniaturized microscopes to obtain recordings of the same dPAG ensembles during exposure to a live predator and a fear conditioned context in male mice. dPAG ensembles encoded not only distance to threat, but also relevant features, such as predator speed and angular offset between mouse and threat. Furthermore, dPAG cells accurately encoded numerous defensive behaviors, including freezing, stretch-attend postures, and escape. Encoding of behaviors and of distance to threat occurred independently in dPAG cells. dPAG cells also displayed a shared representation to encode these behaviors and distance to threat across innate and conditioned threats. Last, we also show that escape could be predicted by dPAG activity several seconds in advance. Thus, dPAG activity dynamically tracks key kinematic and behavioral variables during exposure to threats, and exhibits similar patterns of activation during defensive behaviors elicited by innate or conditioned threats. These data indicate that a common pathway may be recruited by the dPAG during exposure to a wide variety of threat modalities. The dorsal periaqueductal gray (dPAG) is critical to generate defensive behaviors during encounters with threats of multiple modalities. Here we use longitudinal calcium transient recordings of dPAG ensembles in freely moving mice to show that this region uses shared patterns of activity to represent distance to an innate threat (a live predator) and a conditioned threat (a shock grid). We also show that dPAG neural activity can predict diverse defensive behaviors. These data indicate the dPAG uses conserved population-level activity patterns to encode and coordinate defensive behaviors during exposure to both innate and conditioned threats.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1523/JNEUROSCI.2450-20.2021DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8221602PMC
June 2021

Coordination of escape and spatial navigation circuits orchestrates versatile flight from threats.

Neuron 2021 06 15;109(11):1848-1860.e8. Epub 2021 Apr 15.

Department of Psychology, University of California, Los Angeles, Los Angeles, CA 90095, USA. Electronic address:

Naturalistic escape requires versatile context-specific flight with rapid evaluation of local geometry to identify and use efficient escape routes. It is unknown how spatial navigation and escape circuits are recruited to produce context-specific flight. Using mice, we show that activity in cholecystokinin-expressing hypothalamic dorsal premammillary nucleus (PMd-cck) cells is sufficient and necessary for context-specific escape that adapts to each environment's layout. In contrast, numerous other nuclei implicated in flight only induced stereotyped panic-related escape. We reasoned the dorsal premammillary nucleus (PMd) can induce context-specific escape because it projects to escape and spatial navigation nuclei. Indeed, activity in PMd-cck projections to thalamic spatial navigation circuits is necessary for context-specific escape induced by moderate threats but not panic-related stereotyped escape caused by perceived asphyxiation. Conversely, the PMd projection to the escape-inducing dorsal periaqueductal gray projection is necessary for all tested escapes. Thus, PMd-cck cells control versatile flight, engaging spatial navigation and escape circuits.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.neuron.2021.03.033DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8178241PMC
June 2021

Measurement, manipulation and modeling of brain-wide neural population dynamics.

Nat Commun 2021 01 27;12(1):633. Epub 2021 Jan 27.

Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA.

View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41467-020-20371-1DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7840924PMC
January 2021

Decoding and perturbing decision states in real time.

Nature 2021 Mar 20;591(7851):604-609. Epub 2021 Jan 20.

Neurobiology Department, Stanford University, Stanford, CA, USA.

In dynamic environments, subjects often integrate multiple samples of a signal and combine them to reach a categorical judgment. The process of deliberation can be described by a time-varying decision variable (DV), decoded from neural population activity, that predicts a subject's upcoming decision. Within single trials, however, there are large moment-to-moment fluctuations in the DV, the behavioural significance of which is unclear. Here, using real-time, neural feedback control of stimulus duration, we show that within-trial DV fluctuations, decoded from motor cortex, are tightly linked to decision state in macaques, predicting behavioural choices substantially better than the condition-averaged DV or the visual stimulus alone. Furthermore, robust changes in DV sign have the statistical regularities expected from behavioural studies of changes of mind. Probing the decision process on single trials with weak stimulus pulses, we find evidence for time-varying absorbing decision bounds, enabling us to distinguish between specific models of decision making.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41586-020-03181-9DOI Listing
March 2021

Long-Term Characterization of Hippocampal Remapping during Contextual Fear Acquisition and Extinction.

J Neurosci 2020 10 21;40(43):8329-8342. Epub 2020 Sep 21.

Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095

Hippocampal CA1 place cell spatial maps are known to alter their firing properties in response to contextual fear conditioning, a process called "remapping." In the present study, we use chronic calcium imaging to examine remapping during fear retrieval and extinction of an inhibitory avoidance task in mice of both sexes over an extended period of time and with thousands of neurons. We demonstrate that hippocampal ensembles encode space at a finer scale following fear memory acquisition. This effect is strongest near the shock grid. We also characterize the long-term effects of shock on place cell ensemble stability, demonstrating that shock delivery induces several days of high fear and low between-session place field stability, followed by a new, stable spatial representation that appears after fear extinction. Finally, we identify a novel group of CA1 neurons that robustly encode freeze behavior independently from spatial location. Thus, following fear acquisition, hippocampal CA1 place cells sharpen their spatial tuning and dynamically change spatial encoding stability throughout fear learning and extinction. The hippocampus contains place cells that encode an animal's location. This spatial code updates, or remaps, in response to environmental change. It is known that contextual fear can induce such remapping; in the present study, we use chronic calcium imaging to examine inhibitory avoidance-induced remapping over an extended period of time and with thousands of neurons and demonstrate that hippocampal ensembles encode space at a finer scale following electric shock, an effect which is enhanced by threat proximity. We also identify a novel group of freeze behavior-activated neurons. These results suggest that, more than merely shuffling their spatial code following threat exposure, place cells enhance their spatial coding with the possible benefit of improved threat localization.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1523/JNEUROSCI.1022-20.2020DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7577601PMC
October 2020

Structure in Neural Activity during Observed and Executed Movements Is Shared at the Neural Population Level, Not in Single Neurons.

Cell Rep 2020 08;32(6):108006

Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA 90024, USA; Neurosciences Program, University of California, Los Angeles, Los Angeles, CA 90024, USA. Electronic address:

In multiple cortical areas, including the motor cortex, neurons have similar firing rate statistics whether we observe or execute movements. These "congruent" neurons are hypothesized to support action understanding by participating in a neural circuit consistently activated in both observed and executed movements. We examined this hypothesis by analyzing neural population structure and dynamics between observed and executed movements. We find that observed and executed movements exhibit similar neural population covariation in a shared subspace capturing significant neural variance. Further, neural dynamics are more similar between observed and executed movements within the shared subspace than outside it. Finally, we find that this shared subspace has a heterogeneous composition of congruent and incongruent neurons. Together, these results argue that similar neural covariation and dynamics between observed and executed movements do not occur via activation of a subpopulation of congruent single neurons, but through consistent temporal activation of a heterogeneous neural population.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.celrep.2020.108006DOI Listing
August 2020

A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces.

Nat Biomed Eng 2020 10 27;4(10):973-983. Epub 2020 Jul 27.

Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.

The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41551-020-0591-0DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7982996PMC
October 2020

Deep Learning Neural Encoders for Motor Cortex.

IEEE Trans Biomed Eng 2020 08 25;67(8):2145-2158. Epub 2019 Nov 25.

Intracortical brain-machine interfaces (BMIs) transform neural activity into control signals to drive a prosthesis or communication device, such as a robotic arm or computer cursor. To be clinically viable, BMI decoders must achieve high accuracy and robustness. Optimizing these decoders is expensive, traditionally requiring animal or human experiments spanning months to years. This is because BMIs are closed-loop systems, where the user updates his or her motor commands in response to an imperfectly decoded output. Decoder optimization using previously collected "offline" data will therefore not capture this closed-loop response. An alternative approach to significantly accelerate decoder optimization is to use a closed-loop experimental simulator. A key component of this simulator is the neural encoder, which synthetically generates neural population activity from kinematics. Prior neural encoders do not model important features of neural population activity. To overcome these limitations, we use deep learning neural encoders. We find these models significantly outperform prior neural encoders in reproducing peri-stimulus time histograms (PSTHs) and neural population dynamics. We also find that deep learning neural encoders better match neural decoding results in offline data and closed-loop experimental data. We anticipate these deep-learning neural encoders will substantially improve simulators for BMIs, enabling faster evaluation, optimization, and characterization of BMI decoder algorithms.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/TBME.2019.2955722DOI Listing
August 2020

Considerations in using recurrent neural networks to probe neural dynamics.

Authors:
Jonathan C Kao

J Neurophysiol 2019 12 16;122(6):2504-2521. Epub 2019 Oct 16.

Department of Electrical and Computer Engineering, University of California, Los Angeles, California.

Recurrent neural networks (RNNs) are increasingly being used to model complex cognitive and motor tasks performed by behaving animals. RNNs are trained to reproduce animal behavior while also capturing key statistics of empirically recorded neural activity. In this manner, the RNN can be viewed as an in silico circuit whose computational elements share similar motifs with the cortical area it is modeling. Furthermore, because the RNN's governing equations and parameters are fully known, they can be analyzed to propose hypotheses for how neural populations compute. In this context, we present important considerations when using RNNs to model motor behavior in a delayed reach task. First, by varying the network's nonlinear activation and rate regularization, we show that RNNs reproducing single-neuron firing rate motifs may not adequately capture important population motifs. Second, we find that even when RNNs reproduce key neurophysiological features on both the single neuron and population levels, they can do so through distinctly different dynamical mechanisms. To distinguish between these mechanisms, we show that an RNN consistent with a previously proposed dynamical mechanism is more robust to input noise. Finally, we show that these dynamics are sufficient for the RNN to generalize to tasks it was not trained on. Together, these results emphasize important considerations when using RNN models to probe neural dynamics. Artificial neurons in a recurrent neural network (RNN) may resemble empirical single-unit activity but not adequately capture important features on the neural population level. Dynamics of RNNs can be visualized in low-dimensional projections to provide insight into the RNN's dynamical mechanism. RNNs trained in different ways may reproduce neurophysiological motifs but do so with distinctly different mechanisms. RNNs trained to only perform a delayed reach task can generalize to perform tasks where the target is switched or the target location is changed.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1152/jn.00467.2018DOI Listing
December 2019

Publisher Correction: Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements.

Sci Rep 2019 Mar 28;9(1):5528. Epub 2019 Mar 28.

Electrical Engineering Department, Stanford University, Stanford, CA, USA.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41598-018-37930-8DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6437210PMC
March 2019

Single Neuron Firing Rate Statistics in Motor Cortex During Execution and Observation of Movement.

Annu Int Conf IEEE Eng Med Biol Soc 2018 Jul;2018:981-986

Mirror neurons, which fire during both the execution and observation of movement, are believed to play an important role in motor processing and learning. However, much work still remains to understand the similarities and differences in how these neurons compute in the motor cortex during movement execution and observation. Here, we performed experiments where a monkey both executes and observes a center-out-and-back task within the same experimental session. By recording from putatively the same neural population, we were able to analyze and compare single neuron statistics between movement execution and observation. We found that a majority of neurons in the primary motor cortex (M1) and dorsal premotor cortex (PMd) have statistically different firing rate statistics between movement execution and observation. As a result of this difference, we then wondered if neurons during movement observation exhibited a similar characteristic to those during movement execution: changing of preferred directions as a function of movement speed. Interestingly, we found that while observed movement speed is encoded in the neural population, it only alters a small proportion of the neuron's firing rate statistics. These results suggest that neural populations in Ml and PMd process information related to movement differently between execution and observation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2018.8512445DOI Listing
July 2018

Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements.

Sci Rep 2018 11 5;8(1):16357. Epub 2018 Nov 5.

Electrical Engineering Department, Stanford University, Stanford, CA, USA.

Brain-machine interfaces (BMIs) that decode movement intentions should ignore neural modulation sources distinct from the intended command. However, neurophysiology and control theory suggest that motor cortex reflects the motor effector's position, which could be a nuisance variable. We investigated motor cortical correlates of BMI cursor position with or without concurrent arm movement. We show in two monkeys that subtracting away estimated neural correlates of position improves online BMI performance only if the animals were allowed to move their arm. To understand why, we compared the neural variance attributable to cursor position when the same task was performed using arm reaching, versus arms-restrained BMI use. Firing rates correlated with both BMI cursor and hand positions, but hand positional effects were greater. To examine whether BMI position influences decoding in people with paralysis, we analyzed data from two intracortical BMI clinical trial participants and performed an online decoder comparison in one participant. We found only small motor cortical correlates, which did not affect performance. These results suggest that arm movement and proprioception are the major contributors to position-related motor cortical correlates. Cursor position visual feedback is therefore unlikely to affect the performance of BMI-driven prosthetic systems being developed for people with paralysis.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41598-018-34711-1DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6218537PMC
November 2018

Latent Factors and Dynamics in Motor Cortex and Their Application to Brain-Machine Interfaces.

J Neurosci 2018 10;38(44):9390-9401

Department of Electrical and Computer Engineering, and.

In the 1960s, Evarts first recorded the activity of single neurons in motor cortex of behaving monkeys (Evarts, 1968). In the 50 years since, great effort has been devoted to understanding how single neuron activity relates to movement. Yet these single neurons exist within a vast network, the nature of which has been largely inaccessible. With advances in recording technologies, algorithms, and computational power, the ability to study these networks is increasing exponentially. Recent experimental results suggest that the dynamical properties of these networks are critical to movement planning and execution. Here we discuss this dynamical systems perspective and how it is reshaping our understanding of the motor cortices. Following an overview of key studies in motor cortex, we discuss techniques to uncover the "latent factors" underlying observed neural population activity. Finally, we discuss efforts to use these factors to improve the performance of brain-machine interfaces, promising to make these findings broadly relevant to neuroengineering as well as systems neuroscience.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1523/JNEUROSCI.1669-18.2018DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6209846PMC
October 2018

Inferring single-trial neural population dynamics using sequential auto-encoders.

Nat Methods 2018 10 17;15(10):805-815. Epub 2018 Sep 17.

Department of Electrical Engineering, Stanford University, Stanford, CA, USA.

Neuroscience is experiencing a revolution in which simultaneous recording of thousands of neurons is revealing population dynamics that are not apparent from single-neuron responses. This structure is typically extracted from data averaged across many trials, but deeper understanding requires studying phenomena detected in single trials, which is challenging due to incomplete sampling of the neural population, trial-to-trial variability, and fluctuations in action potential timing. We introduce latent factor analysis via dynamical systems, a deep learning method to infer latent dynamics from single-trial neural spiking data. When applied to a variety of macaque and human motor cortical datasets, latent factor analysis via dynamical systems accurately predicts observed behavioral variables, extracts precise firing rate estimates of neural dynamics on single trials, infers perturbations to those dynamics that correlate with behavioral choices, and combines data from non-overlapping recording sessions spanning months to improve inference of underlying dynamics.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41592-018-0109-9DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6380887PMC
October 2018

Augmenting intracortical brain-machine interface with neurally driven error detectors.

J Neural Eng 2017 12;14(6):066007

Department of Electrical Engineering, Stanford University, Stanford, CA 94305, United States of America.

Objective: Making mistakes is inevitable, but identifying them allows us to correct or adapt our behavior to improve future performance. Current brain-machine interfaces (BMIs) make errors that need to be explicitly corrected by the user, thereby consuming time and thus hindering performance. We hypothesized that neural correlates of the user perceiving the mistake could be used by the BMI to automatically correct errors. However, it was unknown whether intracortical outcome error signals were present in the premotor and primary motor cortices, brain regions successfully used for intracortical BMIs.

Approach: We report here for the first time a putative outcome error signal in spiking activity within these cortices when rhesus macaques performed an intracortical BMI computer cursor task.

Main Results: We decoded BMI trial outcomes shortly after and even before a trial ended with 96% and 84% accuracy, respectively. This led us to develop and implement in real-time a first-of-its-kind intracortical BMI error 'detect-and-act' system that attempts to automatically 'undo' or 'prevent' mistakes. The detect-and-act system works independently and in parallel to a kinematic BMI decoder. In a challenging task that resulted in substantial errors, this approach improved the performance of a BMI employing two variants of the ubiquitous Kalman velocity filter, including a state-of-the-art decoder (ReFIT-KF).

Significance: Detecting errors in real-time from the same brain regions that are commonly used to control BMIs should improve the clinical viability of BMIs aimed at restoring motor function to people with paralysis.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1088/1741-2552/aa8dc1DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5742283PMC
December 2017

Leveraging neural dynamics to extend functional lifetime of brain-machine interfaces.

Sci Rep 2017 08 7;7(1):7395. Epub 2017 Aug 7.

Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA.

Intracortical brain-machine interfaces (BMIs) aim to restore lost motor function to people with neurological deficits by decoding neural activity into control signals for guiding prostheses. An important challenge facing BMIs is that, over time, the number of neural signals recorded from implanted multielectrode arrays will decline and result in a concomitant decrease of BMI performance. We sought to extend BMI lifetime by developing an algorithmic technique, implemented entirely in software, to improve performance over state-of-the-art algorithms as the number of recorded neural signals decline. Our approach augments the decoder by incorporating neural population dynamics remembered from an earlier point in the array lifetime. We demonstrate, in closed-loop experiments with two rhesus macaques, that after the loss of approximately 60% of recording electrodes, our approach outperforms state-of-the-art decoders by a factor of 3.2× and 1.7× (corresponding to a 46% and 22% recovery of maximal performance). Further, our results suggest that neural population dynamics in motor cortex are invariant to the number of recorded neurons. By extending functional BMI lifetime, this approach increases the clinical viability of BMIs.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41598-017-06029-xDOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5547077PMC
August 2017

Motor Cortical Visuomotor Feedback Activity Is Initially Isolated from Downstream Targets in Output-Null Neural State Space Dimensions.

Neuron 2017 Jul 15;95(1):195-208.e9. Epub 2017 Jun 15.

Neurosciences Graduate Program, Stanford University, Stanford, CA 94305, USA; Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Neurobiology and Bioengineering Departments, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Stanford Neurosciences Institute, Stanford University, Stanford, CA 94305, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA.

Neural circuits must transform new inputs into outputs without prematurely affecting downstream circuits while still maintaining other ongoing communication with these targets. We investigated how this isolation is achieved in the motor cortex when macaques received visual feedback signaling a movement perturbation. To overcome limitations in estimating the mapping from cortex to arm movements, we also conducted brain-machine interface (BMI) experiments where we could definitively identify neural firing patterns as output-null or output-potent. This revealed that perturbation-evoked responses were initially restricted to output-null patterns that cancelled out at the neural population code readout and only later entered output-potent neural dimensions. This mechanism was facilitated by the circuit's large null space and its ability to strongly modulate output-potent dimensions when generating corrective movements. These results show that the nervous system can temporarily isolate portions of a circuit's activity from its downstream targets by restricting this activity to the circuit's output-null neural dimensions.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.neuron.2017.05.023DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5547570PMC
July 2017

Trial-by-Trial Motor Cortical Correlates of a Rapidly Adapting Visuomotor Internal Model.

J Neurosci 2017 02 13;37(7):1721-1732. Epub 2017 Jan 13.

Neurosciences Graduate Program,

Accurate motor control is mediated by internal models of how neural activity generates movement. We examined neural correlates of an adapting internal model of visuomotor gain in motor cortex while two macaques performed a reaching task in which the gain scaling between the hand and a presented cursor was varied. Previous studies of cortical changes during visuomotor adaptation focused on preparatory and perimovement epochs and analyzed trial-averaged neural data. Here, we recorded simultaneous neural population activity using multielectrode arrays and focused our analysis on neural differences in the period before the target appeared. We found that we could estimate the monkey's internal model of the gain using the neural population state during this pretarget epoch. This neural correlate depended on the gain experienced during recent trials and it predicted the speed of the subsequent reach. To explore the utility of this internal model estimate for brain-machine interfaces, we performed an offline analysis showing that it can be used to compensate for upcoming reach extent errors. Together, these results demonstrate that pretarget neural activity in motor cortex reflects the monkey's internal model of visuomotor gain on single trials and can potentially be used to improve neural prostheses. When generating movement commands, the brain is believed to use internal models of the relationship between neural activity and the body's movement. Visuomotor adaptation tasks have revealed neural correlates of these computations in multiple brain areas during movement preparation and execution. Here, we describe motor cortical changes in a visuomotor gain change task even before a specific movement is cued. We were able to estimate the gain internal model from these pretarget neural correlates and relate it to single-trial behavior. This is an important step toward understanding the sensorimotor system's algorithms for updating its internal models after specific movements and errors. Furthermore, the ability to estimate the internal model before movement could improve motor neural prostheses being developed for people with paralysis.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1523/JNEUROSCI.1091-16.2016DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5320605PMC
February 2017

A Non-Human Primate Brain-Computer Typing Interface.

Proc IEEE Inst Electr Electron Eng 2017 Jan 12;105(1):66-72. Epub 2016 Sep 12.

Electrical Engineering Department, the Bioengineering Department, the Neurobiology Department, and Stanford Neurosciences Institute, Stanford University, Stanford, CA 94305 USA and also with Howard Hughes Medical Institute, Chevy Chase, MD 20815 USA.

Brain-computer interfaces (BCIs) record brain activity and translate the information into useful control signals. They can be used to restore function to people with paralysis by controlling end effectors such as computer cursors and robotic limbs. Communication neural prostheses are BCIs that control user interfaces on computers or mobile devices. Here we demonstrate a communication prosthesis by simulating a typing task with two rhesus macaques implanted with electrode arrays. The monkeys used two of the highest known performing BCI decoders to type out words and sentences when prompted one symbol/letter at a time. On average, Monkeys J and L achieved typing rates of 10.0 and 7.2 words per minute (wpm), respectively, copying text from a newspaper article using a velocity-only two dimensional BCI decoder with dwell-based symbol selection. With a BCI decoder that also featured a discrete click for key selection, typing rates increased to 12.0 and 7.8 wpm. These represent the highest known achieved communication rates using a BCI. We then quantified the relationship between bitrate and typing rate and found it approximately linear: typing rate in wpm is nearly three times bitrate in bits per second. We also compared the metrics of achieved bitrate and information transfer rate and discuss their applicability to real-world typing scenarios. Although this study cannot model the impact of cognitive load of word and sentence planning, the findings here demonstrate the feasibility of BCIs to serve as communication interfaces and represent an upper bound on the expected achieved typing rate for a given BCI throughput.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JPROC.2016.2586967DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7970827PMC
January 2017

Making brain-machine interfaces robust to future neural variability.

Nat Commun 2016 12 13;7:13749. Epub 2016 Dec 13.

Electrical Engineering Department, Stanford University, Stanford, California 94305, USA.

A major hurdle to clinical translation of brain-machine interfaces (BMIs) is that current decoders, which are trained from a small quantity of recent data, become ineffective when neural recording conditions subsequently change. We tested whether a decoder could be made more robust to future neural variability by training it to handle a variety of recording conditions sampled from months of previously collected data as well as synthetic training data perturbations. We developed a new multiplicative recurrent neural network BMI decoder that successfully learned a large variety of neural-to-kinematic mappings and became more robust with larger training data sets. Here we demonstrate that when tested with a non-human primate preclinical BMI model, this decoder is robust under conditions that disabled a state-of-the-art Kalman filter-based decoder. These results validate a new BMI strategy in which accumulated data history are effectively harnessed, and may facilitate reliable BMI use by reducing decoder retraining downtime.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/ncomms13749DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5159828PMC
December 2016

The need for calcium imaging in nonhuman primates: New motor neuroscience and brain-machine interfaces.

Exp Neurol 2017 01 7;287(Pt 4):437-451. Epub 2016 Aug 7.

Department of Electrical Engineering, Stanford University, Stanford, CA 94305, United States; Department of Bioengineering, Stanford University, Stanford, CA 94305, United States; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, United States; Deparment of Neurobiology, Stanford University, Stanford, CA 94305, United States. Electronic address:

A central goal of neuroscience is to understand how populations of neurons coordinate and cooperate in order to give rise to perception, cognition, and action. Nonhuman primates (NHPs) are an attractive model with which to understand these mechanisms in humans, primarily due to the strong homology of their brains and the cognitively sophisticated behaviors they can be trained to perform. Using electrode recordings, the activity of one to a few hundred individual neurons may be measured electrically, which has enabled many scientific findings and the development of brain-machine interfaces. Despite these successes, electrophysiology samples sparsely from neural populations and provides little information about the genetic identity and spatial micro-organization of recorded neurons. These limitations have spurred the development of all-optical methods for neural circuit interrogation. Fluorescent calcium signals serve as a reporter of neuronal responses, and when combined with post-mortem optical clearing techniques such as CLARITY, provide dense recordings of neuronal populations, spatially organized and annotated with genetic and anatomical information. Here, we advocate that this methodology, which has been of tremendous utility in smaller animal models, can and should be developed for use with NHPs. We review here several of the key opportunities and challenges for calcium-based optical imaging in NHPs. We focus on motor neuroscience and brain-machine interface design as representative domains of opportunity within the larger field of NHP neuroscience.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.expneurol.2016.08.003DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5154795PMC
January 2017

A High-Performance Neural Prosthesis Incorporating Discrete State Selection With Hidden Markov Models.

IEEE Trans Biomed Eng 2017 04 21;64(4):935-945. Epub 2016 Jun 21.

Department of Electrical Engineering, Department of Bioengineering, and Department of NeurobiologyStanford University.

Communication neural prostheses aim to restore efficient communication to people with motor neurological injury or disease by decoding neural activity into control signals. These control signals are both analog (e.g., the velocity of a computer mouse) and discrete (e.g., clicking an icon with a computer mouse) in nature. Effective, high-performing, and intuitive-to-use communication prostheses should be capable of decoding both analog and discrete state variables seamlessly. However, to date, the highest-performing autonomous communication prostheses rely on precise analog decoding and typically do not incorporate high-performance discrete decoding. In this report, we incorporated a hidden Markov model (HMM) into an intracortical communication prosthesis to enable accurate and fast discrete state decoding in parallel with analog decoding. In closed-loop experiments with nonhuman primates implanted with multielectrode arrays, we demonstrate that incorporating an HMM into a neural prosthesis can increase state-of-the-art achieved bitrate by 13.9% and 4.2% in two monkeys ( ). We found that the transition model of the HMM is critical to achieving this performance increase. Further, we found that using an HMM resulted in the highest achieved peak performance we have ever observed for these monkeys, achieving peak bitrates of 6.5, 5.7, and 4.7 bps in Monkeys J, R, and L, respectively. Finally, we found that this neural prosthesis was robustly controllable for the duration of entire experimental sessions. These results demonstrate that high-performance discrete decoding can be beneficially combined with analog decoding to achieve new state-of-the-art levels of performance.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/TBME.2016.2582691DOI Listing
April 2017

Leveraging historical knowledge of neural dynamics to rescue decoder performance as neural channels are lost: "Decoder hysteresis".

Annu Int Conf IEEE Eng Med Biol Soc 2015 Aug;2015:1061-6

An intracortical brain-machine interface (BMI) decodes spiking activity recorded from motor cortical neurons to drive a prosthetic device (e.g., a computer cursor or robotic arm). As the number of recorded neurons decreases over time due to decay in recording quality, the performance of a BMI decreases. We asked: can degrading BMI performance be rescued by using prior information from when more neurons were observed? This would entail augmenting a decoder by using previously learned knowledge about motor cortex (at an earlier point in the array lifetime). We implemented this idea by modeling low-dimensional dynamics of the neural population, which describe how the population evolves through time. We posit that if the neural dynamics accurately reflect properties of motor cortex, then having a better estimate of these dynamics should result in a better decoder. Using previously collected (offline) experimental data, we found that a decoder using dynamics inferred in the past (when more neural channels were available) outperformed the same decoder using dynamics inferred from the (fewer) remaining neural channels. These results suggest that neural dynamics capture important features of the neural population responses in motor cortex, and that knowledge of these dynamics may rescue BMI performance even as array signal quality degrades.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2015.7318548DOI Listing
August 2015

Auto-deleting brain machine interface: Error detection using spiking neural activity in the motor cortex.

Annu Int Conf IEEE Eng Med Biol Soc 2015 ;2015:71-5

Brain machine interfaces (BMIs) aim to assist people with paralysis by increasing their independence and ability to communicate, e.g., by using a cursor-based virtual keyboard. Current BMI clinical trials are hampered by modest performance that causes selection of wrong characters (errors) and thus reduces achieved typing rate. If it were possible to detect these errors without explicit knowledge of the task goal, this could be used to automatically "undo" wrong selections or even prevent upcoming wrong selections. We decoded imminent or recent errors during closed-loop BMI control from intracortical spiking neural activity. In our experiment, a non-human primate controlled a neurally-driven BMI cursor to acquire targets on a grid, which simulates a virtual keyboard. In offline analyses of this closed-loop BMI control data, we identified motor cortical neural signals indicative of task error occurrence. We were able to detect task outcomes (97% accuracy) and even predict upcoming task outcomes (86% accuracy) using neural activity alone. This novel strategy may help increase the performance and clinical viability of BMIs.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2015.7318303DOI Listing
November 2016

Single-trial dynamics of motor cortex and their applications to brain-machine interfaces.

Nat Commun 2015 Jul 29;6:7759. Epub 2015 Jul 29.

1] Electrical Engineering Department, Stanford University, Stanford, California 94305, USA [2] Bioengineering Department, Stanford University, Stanford, California 94305, USA [3] Neurosciences Program, Stanford University, Stanford, California, USA [4] Neurobiology Department, Stanford University, Stanford, California 94305, USA [5] Bio-X Program, Stanford University, Stanford, California 94305, USA [6] Stanford Neurosciences Institute, Stanford University, Stanford, California 94305, USA.

Increasing evidence suggests that neural population responses have their own internal drive, or dynamics, that describe how the neural population evolves through time. An important prediction of neural dynamical models is that previously observed neural activity is informative of noisy yet-to-be-observed activity on single-trials, and may thus have a denoising effect. To investigate this prediction, we built and characterized dynamical models of single-trial motor cortical activity. We find these models capture salient dynamical features of the neural population and are informative of future neural activity on single trials. To assess how neural dynamics may beneficially denoise single-trial neural activity, we incorporate neural dynamics into a brain-machine interface (BMI). In online experiments, we find that a neural dynamical BMI achieves substantially higher performance than its non-dynamical counterpart. These results provide evidence that neural dynamics beneficially inform the temporal evolution of neural activity on single trials and may directly impact the performance of BMIs.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/ncomms8759DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4532790PMC
July 2015

A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes.

J Neural Eng 2015 Jun 6;12(3):036009. Epub 2015 May 6.

Neurosciences Program, Stanford University, Stanford, CA USA.

Objective: Brain-machine interfaces (BMIs) seek to enable people with movement disabilities to directly control prosthetic systems with their neural activity. Current high performance BMIs are driven by action potentials (spikes), but access to this signal often diminishes as sensors degrade over time. Decoding local field potentials (LFPs) as an alternative or complementary BMI control signal may improve performance when there is a paucity of spike signals. To date only a small handful of LFP decoding methods have been tested online; there remains a need to test different LFP decoding approaches and improve LFP-driven performance. There has also not been a reported demonstration of a hybrid BMI that decodes kinematics from both LFP and spikes. Here we first evaluate a BMI driven by the local motor potential (LMP), a low-pass filtered time-domain LFP amplitude feature. We then combine decoding of both LMP and spikes to implement a hybrid BMI.

Approach: Spikes and LFP were recorded from two macaques implanted with multielectrode arrays in primary and premotor cortex while they performed a reaching task. We then evaluated closed-loop BMI control using biomimetic decoders driven by LMP, spikes, or both signals together.

Main Results: LMP decoding enabled quick and accurate cursor control which surpassed previously reported LFP BMI performance. Hybrid decoding of both spikes and LMP improved performance when spikes signal quality was mediocre to poor.

Significance: These findings show that LMP is an effective BMI control signal which requires minimal power to extract and can substitute for or augment impoverished spikes signals. Use of this signal may lengthen the useful lifespan of BMIs and is therefore an important step towards clinically viable BMIs.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1088/1741-2560/12/3/036009DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4457459PMC
June 2015
-->