Papers by Bernabe Linares-Barranco
arXiv (Cornell University), Apr 23, 2020
Memristive devices have shown great promise to facilitate the acceleration and improve the power ... more Memristive devices have shown great promise to facilitate the acceleration and improve the power efficiency of Deep Learning (DL) systems. Crossbar architectures constructed using these Resistive Random-Access Memory (RRAM) devices can be used to efficiently implement various in-memory computing operations, such as Multiply Accumulate (MAC) and unrolledconvolutions, which are used extensively in Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs). However, memristive devices face concerns of aging and non-idealities, which limit the accuracy, reliability, and robustness of Memristive Deep Learning Systems (MDLSs), that should be considered prior to circuit-level realization. This Original Software Publication (OSP) presents MemTorch, an open-source 1 framework for customized large-scale memristive DL simulations, with a refined focus on the cosimulation of device non-idealities. MemTorch also facilitates co-modelling of key crossbar peripheral circuitry. MemTorch adopts a modernized software engineering methodology and integrates directly with the well-known PyTorch Machine Learning (ML) library.
Spike-based processing technology is capable of very high speed throughput, as it does not rely o... more Spike-based processing technology is capable of very high speed throughput, as it does not rely on sensing and processing sequences of frames. Besides, it allows building complex and hierarchically structured cortical-like layers for sophisticated processing. In this paper we summarize the fundamental properties of this sensing and processing technology applied to artificial vision systems and the AER (Address Event Representation) protocol used in hardware spiking systems. Finally a four-layer system is described for character recognition. The system is slightly based on the Fukushima´s Neocognitron. Realistic simulations using figures of already existing AER devices are provided, which show recognition delays under 10μs.
arXiv (Cornell University), Jan 20, 2023
For neuromorphic engineering to emulate the human brain, improving memory density with low power ... more For neuromorphic engineering to emulate the human brain, improving memory density with low power consumption is an indispensable but challenging goal. In this regard, emerging RRAMs have attracted considerable interest for their unique qualities like low power consumption, high integration potential, durability, and CMOS compatibility. Using RRAMs to imitate the more analog storage behavior of brain synapses is also a promising strategy for further improving memory density and power efficiency. However, RRAM devices display strong stochastic behavior, together with relaxation effects, making it more challenging to precisely control their multi-level storage capability. To address this, researchers have reported different multi-level programming strategies, mostly involving the precise control of analog parameters like compliance current during write operations and/or programming voltage amplitudes. Here, we present a new fully digital relaxation-aware method for tuning the conductance of analog RRAMs. The method is based on modulating digital pulse widths during erase operations while keeping other parameters fixed, and therefore requires no precise alterations to analog parameters like compliance currents or programming voltage amplitudes. Experimental results, with and without relaxation effect awareness, on a 64 RRAM 1T1R HfOx memory array of cells, fabricated in 130nm CMOS technology, indicate that it is possible to obtain 2-bit memory per cell multi-value storage at the array level, verified 1000 seconds after programming.
2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Despite the parallelism and sparsity in neural network models, their transfer into hardware unavo... more Despite the parallelism and sparsity in neural network models, their transfer into hardware unavoidably makes them susceptible to hardware-level faults. Hardware-level faults can occur either during manufacturing, such as physical defects and process-induced variations, or in the field due to environmental factors and aging. The performance under fault scenarios needs to be assessed so as to develop cost-effective fault-tolerance schemes. In this work, we assess the resilience characteristics of a hardware accelerator for Spiking Neural Networks (SNNs) designed in VHDL and implemented on an FPGA. The fault injection experiments pinpoint the parts of the design that need to be protected against faults, as well as the parts that are inherently fault-tolerant.
2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021
The error-resiliency of Artificial Intelligence (AI) hardware accelerators is a major concern, es... more The error-resiliency of Artificial Intelligence (AI) hardware accelerators is a major concern, especially when they are deployed in mission-critical and safety-critical applications. In this paper, we propose a neuron fault tolerance strategy for Spiking Neural Networks (SNNs). It is optimized for low area and power overhead by leveraging observations made from a largescale fault injection experiment that pinpoints the critical fault types and locations. We describe the fault modeling approach, the fault injection framework, the results of the fault injection experiment, the fault-tolerance strategy, and the fault-tolerant SNN architecture. The idea is demonstrated on two SNNs that we designed for two SNN-oriented datasets, namely the N-MNIST and IBM's DVS128 gesture datasets.
Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, 2000
2020 IEEE International Symposium on Circuits and Systems (ISCAS), 2020
Reading several ReRAMs simultaneously in a neuromorphic circuit increases power consumption and l... more Reading several ReRAMs simultaneously in a neuromorphic circuit increases power consumption and limits scalability. Applying small inference read pulses is a vain attempt when offset voltages of the read-out circuit are decisively more. This paper presents an experimental validation of a three-stage calibration scheme to calibrate the DC offset voltage across the rows of the memristive crossbar. The proposed method is based on biasing the body terminal of one of the differential pair MOSFETs of the buffer through a series of cascaded resistor banks arranged in three stages-coarse, fine and finer stages. The circuit is designed in a 130 nm CMOS technology, where the OxRAM-based binary memristors are built on top of it. A dedicated PCB and other auxiliary boards have been designed for testing the chip. Experimental results validate the presented approach, which is only limited by mismatch and electrical noise.
IEEE Access, 2021
This paper studies the suitability of neuromorphic event-based vision cameras for spaceflight and... more This paper studies the suitability of neuromorphic event-based vision cameras for spaceflight and the effects of neutron radiation on their performance. Neuromorphic event-based vision cameras are novel sensors that implement asynchronous, clockless data acquisition, providing information about the change in illuminance ≥ 120dB with sub-millisecond temporal precision. These sensors have huge potential for space applications as they provide an extremely sparse representation of visual dynamics while removing redundant information, thereby conforming to low-resource requirements. An event-based sensor was irradiated under wide-spectrum neutrons at Los Alamos Neutron Science Center and its effects were classified. Radiation-induced damage of the sensor under wide-spectrum neutrons was tested, as was the radiative effect on the signal-to-noise ratio of the output at different angles of incidence from the beam source. We found that the sensor had very fast recovery during radiation, showing high correlation of noise event bursts with respect to source macro-pulses. No statistically significant differences were observed between the number of events induced at different angles of incidence but significant differences were found in the spatial structure of noise events at different angles. The results show that event-based cameras are capable of functioning in a space-like, radiative environment with a signal-to-noise ratio of 3.355. They also show that radiation-induced noise does not affect event-level computation. Finally, we introduce the Event-based Radiation-Induced Noise Simulation Environment (Event-RINSE), a simulation environment based on the noise-modelling we conducted and capable of injecting the effects of radiation-induced noise from the collected data to any stream of events in order to ensure that developed code can operate in a radiative environment. To the best of our knowledge, this is the first time such analysis of neutron-induced noise has been performed on a neuromorphic vision sensor, and this study shows the advantage of using such sensors for space applications.
2021 IEEE International Symposium on Circuits and Systems (ISCAS), 2021
The emergence of nano-scale memristive devices encouraged many different research areas to exploi... more The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spiketiming-dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation between pre-and postsynaptic spikes. In this work, we demonstrate experimentally that binary stochastic STDP learning can be obtained from a memristor when the appropriate pulses are applied at both sides of the device.
Frontiers in Neuroscience, 2021
Computing paradigm based on von Neuman architectures cannot keep up with the ever-increasing data... more Computing paradigm based on von Neuman architectures cannot keep up with the ever-increasing data growth (also called “data deluge gap”). This has resulted in investigating novel computing paradigms and design approaches at all levels from materials to system-level implementations and applications. An alternative computing approach based on artificial neural networks uses oscillators to compute or Oscillatory Neural Networks (ONNs). ONNs can perform computations efficiently and can be used to build a more extensive neuromorphic system. Here, we address a fundamental problem: can we efficiently perform artificial intelligence applications with ONNs? We present a digital ONN implementation to show a proof-of-concept of the ONN approach of “computing-in-phase” for pattern recognition applications. To the best of our knowledge, this is the first attempt to implement an FPGA-based fully-digital ONN. We report ONN accuracy, training, inference, memory capacity, operating frequency, hardwa...
Frontiers in Neuroscience, 2018
Frontiers in neuroscience, 2015
This article reports on two databases for event-driven object recognition using a Dynamic Vision ... more This article reports on two databases for event-driven object recognition using a Dynamic Vision Sensor (DVS). The first, which we call Poker-DVS and is being released together with this article, was obtained by browsing specially made poker card decks in front of a DVS camera for 2-4 s. Each card appeared on the screen for about 20-30 ms. The poker pips were tracked and isolated off-line to constitute the 131-recording Poker-DVS database. The second database, which we call MNIST-DVS and which was released in December 2013, consists of a set of 30,000 DVS camera recordings obtained by displaying 10,000 moving symbols from the standard MNIST 70,000-picture database on an LCD monitor for about 2-3 s each. Each of the 10,000 symbols was displayed at three different scales, so that event-driven object recognition algorithms could easily be tested for different object sizes. This article tells the story behind both databases, covering, among other aspects, details of how they work and th...
2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings, 2014
We present here an overview of a new vision paradigm where sensors and processors use visual info... more We present here an overview of a new vision paradigm where sensors and processors use visual information not represented by sequences of frames. Event-driven vision is inherently frame-free, as happens in biological systems. We use an event-driven sensor chip (called Dynamic Vision Sensor or DVS) together with event-driven convolution module arrays implemented on high-end FPGAs. Experimental results demonstrate the application of this paradigm to implement Gabor filters and 3D stereo reconstruction systems. This architecture can be applied to real systems which need efficient and high-speed visual perception, like vehicle automatic driving, robotic applications in non-structured environments, or intelligent surveillance in security systems.
2011 IEEE International Symposium of Circuits and Systems (ISCAS), 2011
People rarely put in their papers the things that didn't work, the mistakes they made, and how th... more People rarely put in their papers the things that didn't work, the mistakes they made, and how they found out what went wrong. Such confessions can help others learn how to avoid similar mistakes. Twenty-six confessions were collected to form the bulk of this paper. Themes that arise are errors that result from not understanding the limitations of simulation tools in modeling physical reality, chip verification errors that result from lack of clear communication between designers, and projects that are considered in their own isolated environment of technical challenges rather than the broader context of their environment or application.
The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronous... more The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, applying the matching algorithm to the events generated by the Gabor filters and not to those produced by the DVS. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.
Nowadays spike-based brain processing emulation is taking off. Several EU and others worldwide pr... more Nowadays spike-based brain processing emulation is taking off. Several EU and others worldwide projects are demonstrating this, like SpiNNaker, BrainScaleS, FACETS, or NeuroGrid. The larger the brain process emulation on silicon is, the higher the communication performance of the hosting platforms has to be. Many times the bottleneck of these system implementations is not on the performance inside a chip or a board, but in the communication between boards. This paper describes a novel modular Address-Event-Representation (AER) FPGA-based (Spartan6) infrastructure PCB (the AER-Node board) with 2.5Gbps LVDS high speed serial links over SATA cables that offers a peak performance of 32-bit 62.5Meps (Mega events per second) on board-to-board communications. The board allows back compatibility with parallel AER devices supporting up to x2 28-bit parallel data with asynchronous handshake. These boards also allow modular expansion functionality through several daughter boards. The paper is focused on describing in detail the LVDS serial interface and presenting its performance.
Lecture Notes in Computer Science, 2012
Computation with spiking neurons takes advantage of the abstraction of action potentials into str... more Computation with spiking neurons takes advantage of the abstraction of action potentials into streams of stereotypical events, which encode information through their timing. This approach both reduces power consumption and alleviates communication bottlenecks. A number of such spiking custom mixed-signal address event representation (AER) chips have been developed in recent years. In this paper, we present i) a flexible event-driven platform consisting of the integration of a visual AER sensor and the SpiNNaker system, a programmable massively parallel digital architecture oriented to the simulation of spiking neural networks; ii) the implementation of a neural network for feature-based attentional selection on this platform.
2008 IEEE International Symposium on Circuits and Systems, 2008
This paper presents the design and simulation of a serial AER LVDS communication link. It convert... more This paper presents the design and simulation of a serial AER LVDS communication link. It converts data from classical AER parallel bus with a 4-phase handshaking protocol into a bit stream which is transmitted serially into a single LVDS wire. At the receiver side data from the LVDS cable are transformed back to a parallel AER bus and handshaking signals are also properly managed. The link has been designed in a 90 nms technology. Extensive simulations have been performed demonstrating that the link can operate at a speed of 1 Gbps for all the technology corners, exhibiting a power consumption of 27.8 mW for the transmitter and 12.3 mW for the receiver. In the simulation the transmission channel was modelled as a 50 cm cat5E UTP cable, connected to the AER chip through 5 cm PCB traces modelled as a coupled microstrip transmission line. The design has been completed up to the layout level and has been submitted for fabrication. The transmitter and the receiver take up an area of 311x148 µm 2 and 300x148 µm 2 respectively.
2008 IEEE International Symposium on Circuits and Systems, 2008
We present a neuromorphic fully digital convolution microchip for Address Event Representation (A... more We present a neuromorphic fully digital convolution microchip for Address Event Representation (AER) spike-based processing systems. This microchip computes 2-D convolutions with a programmable kernel in real time. It operates on a pixel array of size 32 x 32, and the kernel is programmable and can be of arbitrary shape and size up to 32 x 32 pixels. The chip receives and generates data in AER format, which is asynchronous and digital. The paper describes the architecture of the chip, the test setup, and experimental results obtained from a fabricated prototype.
Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 2010
Uploads
Papers by Bernabe Linares-Barranco