Optical Brain-Computer Interface

Optical neurotechnologies use light to interface with neurons and can monitor and manipulate neural activity with high spatial-temporal precision over large cortical areas. There has been considerable progress in miniaturizing microscopes for head-mounted configurations, but existing devices are bulky and their application in humans will require a more non-invasive, fully implantable form factor. Here we report an ultrathin, miniaturized subdural complementary metal–oxide–semiconductor (CMOS) optical device for bidirectional optical stimulation and recording. We use a custom CMOS application-specific integrated circuit that is capable of both fluorescence imaging and optogenetic stimulation, creating a probe with a total thickness of less than 200 µm, which is thin enough to lie entirely within the subdural space of the primate brain. We show that the device can be used for imaging and optical stimulation in a mouse model and can be used to decode reach movement speed in a non-human primate.

The use of optogenetic stimulation to evoke neuronal activity in targeted neural populations—enabled by opsins with fast kinetics, high sensitivity and cell-type and subcellular specificity—is a powerful tool in neuroscience. However, to interface with the opsins, deep-brain light delivery systems are required that match the scale of the spatial and temporal control offered by the molecular actuators. Here we show that organic light-emitting diodes can be combined with complementary metal–oxide–semiconductor technology to create bright, actively multiplexed emissive elements. We create implantable shanks in which 1,024 individually addressable organic light-emitting diode pixels with a 24.5 µm pitch are integrated with active complementary metal–oxide–semiconductor drive and control circuitry. This integration is enabled by controlled electrode conditioning, monolithic deposition of the organic light-emitting diodes and optimized thin-film encapsulation. The resulting probes can be used to access brain regions as deep as 5 mm and selectively activate individual neurons with millisecond-level precision in mice.

Implantable image sensors have the potential to revolutionize neuroscience. Due to their small form factor requirements; however, conventional filters and optics cannot be implemented. These limitations obstruct high-resolution imaging of large neural densities. Recent advances in angle-sensitive image sensors and single-photon avalanche diodes have provided a path toward ultrathin lens-less fluorescence imaging, enabling plenoptic sensing by extending sensing capabilities to include photon arrival time and incident angle, thereby providing the opportunity for separability of fluorescence point sources within the context of light-field microscopy (LFM). However, the addition of spectral sensitivity to angle-sensitive LFM reduces imager resolution because each wavelength requires a separate pixel subset. Here, we present a 1024-pixel, 50  µm thick implantable shank-based neural imager with color-filter-grating-based angle-sensitive pixels. This angular-spectral sensitive front end combines a metal–insulator–metal (MIM) Fabry–Perot color filter and diffractive optics to produce the measurement of orthogonal light-field information from two distinct colors within a single photodetector. The result is the ability to add independent color sensing to LFM while doubling the effective pixel density. The implantable imager combines angular-spectral and temporal information to demix and localize multispectral fluorescent targets. In this initial prototype, this is demonstrated with 45 μm diameter fluorescently labeled beads in scattering medium. Fluorescent lifetime imaging is exploited to further aid source separation, in addition to detecting pH through lifetime changes in fluorescent dyes. While these initial fluorescent targets are considerably brighter than fluorescently labeled neurons, further improvements will allow the application of these techniques to in-vivo multifluorescent structural and functional neural imaging.

Emerging optical functional imaging and optogenetics are among the most promising approaches in neuroscience to study neuronal circuits. Combining both methods into a single implantable device enables all-optical neural interrogation with immediate applications in freely-behaving animal studies. In this paper, we demonstrate such a device capable of optical neural recording and stimulation over large cortical areas. This implantable surface device exploits lens-less computational imaging and a novel packaging scheme to achieve an ultra-thin (250μm-thick), mechanically flexible form factor. The core of this device is a custom-designed CMOS integrated circuit containing a 160×160 array of time-gated single-photon avalanche photodiodes (SPAD) for low-light intensity imaging and an interspersed array of dual-color (blue and green) flip-chip bonded micro-LED (μLED) as light sources. We achieved 60μm lateral imaging resolution and 0.2mm3 volumetric precision for optogenetics over a 5.4×5.4mm2 field of view (FoV). The device achieves a 125-fps frame-rate and consumes 40 mW of total power.

We propose a new paradigm for dense functional imaging of brain activity to surmount the limitations of present methodologies. We term this approach “integrated neurophotonics”; it combines recent advances in microchip-based integrated photonic and electronic circuitry with those from optogenetics. This approach has the potential to enable lens-less functional imaging from within the brain itself to achieve dense, large-scale stimulation and recording of brain activity with cellular resolution at arbitrary depths. We perform a computational study of several prototype 3D architectures for implantable probe-array modules that are designed to provide fast and dense single-cell resolution (e.g., within a 1-mm3 volume of mouse cortex comprising ∼100,000 neurons). We describe progress toward realizing integrated neurophotonic imaging modules, which can be produced en masse with current semiconductor foundry protocols for chip manufacturing. Implantation of multiple modules can cover extended brain regions.

This paper presents a device for time-gated fluorescence imaging in the deep brain, consisting of two on-chip laser diodes and 512 single-photon avalanche diodes (SPADs). The edge-emitting laser diodes deliver fluorescence excitation above the SPAD array, parallel to the imager. In the time domain, laser diode illumination is pulsed and the SPAD is time-gated, allowing a fluorescence excitation rejection up to O.D. 3 at 1 ns of time-gate delay. Each SPAD pixel is masked with Talbot gratings to enable the mapping of 2D array photon counts into a 3D image. The 3D image achieves a resolution of 40, 35, and 73 μm in the x, y, and z directions, respectively, in a noiseless environment, with a maximum frame rate of 50 kilo-frames-per-second. We present measurement results of the spatial and temporal profiles of the dual-pulsed laser diode illumination and of the photon detection characteristics of the SPAD array. Finally, we show the imager’s ability to resolve a glass micropipette filled with red fluorescent microspheres. The system’s 420 μm-wide cross section allows it to be inserted at arbitrary depths of the brain while achieving a field of view four times larger than fiber endoscopes of equal diameter.

We present an implantable single-photon shank-based imager, monolithically integrated onto a single CMOS IC. The imager comprises of 512 single-photon avalanche diodes distributed along two shanks, with a 6-bit depth in-pixel memory and an on-chip digital-to-time converter. To scale down the system to a minimally invasive form factor, we substitute optical filtering and focusing elements with a time-gated, angle-sensitive detection system. The imager computationally reconstructs the position of fluorescent sources within a 3-D volume of 3.4 mm × 600 μm× 400 μm .