-
In current sensing-computing systems, sensors are used to acquire information from environments, such data are normally analogue, unstructured and even redundant. After the analogue-to-digital conversion (ADC), the data are transferred into digital computers for processing. In computers with the von Neumann architecture, memories and central processing units (CPUs) are physically separated. Such a separation of sensing terminals, memories and CPUs yields serious problems, such as high energy consumption, long response time, huge data storage, and stringent requirements for the communication bandwidth and security. However, time- and energy-efficient ways are urgently required to process information at where data are generated. On the other hand, biological sensory organs respond to external stimuli in real-time with high efficiency due to the integrated capabilities of sensing, memory and computing. Therefore, the problem of separated sensing units, memories and processing units can be solved by emulating biological sensory organs. In this work, we propose bio-inspired sensory systems with integrated capabilities of sensing, data storage and processing. In such a system, different sensors are used to capture the environmental signals from e.g. gases, light, audio and pressure, then the sensory signals are processed by an analogue signal processor, so that the energy-consuming ADC is avoided, afterwards the sensory signals are processed by a brain-inspired chip which consists of neuron-synapse cores based on memristors. In the neuron-synapse cores, leaky integrate-and-fire (LIF) neurons can be implemented by memristors and capacitors, and adaptive LIF neurons are developed from the LIF neurons to realize unsupervised learning algorithms. The synapses are realized by memristor arrays which can also perform the in-memory computing. By changing the connection between the neurons, the brain-inspired chip can realize different spiking neural networks (SNNs), such as fully connected SNN, convolutional SNN, and recurrent SNN. The synaptic weight in SNNs can be updated according to the spike-timing dependent plasticity (STDP) or the spike-rate dependent plasticity (SRDP). As an example, a bio-inspired olfactory system is demonstrated. In a artificial olfactory system, a sensor array detects and transforms the chemical information about gas molecules into electrical sensory signals. Then the sensory signals are processed by the analogue signal processing unit. After pre-processing, the brain-inspired chip classifies gases by constructing a fully connected SNN with two layers. Such a bio-inspired olfactory system emulates the function of a biological nose, overcoming the low efficiency caused by the frequent sampling, data conversion, transfer and storage under the current sensing-computing architecture. More importantly, the approach of this work can be used to emulate almost all the biological perceptions, such as touch, sight, hearing and taste, through the integration with different types of sensors., Therefore, this work offers a brand new approach to realizing the artificial intelligence (AI). -
Keywords:
- biological sensory organ /
- bio-inspired sensory system /
- neuromorphic computing /
- spiking neural network /
- memristor
[1] Zhou F, Chai Y 2020 Nat. Electron. 3 664
Google Scholar
[2] Wang T, Huang H M, Wang X X, Guo X 2021 InfoMat 3 804
Google Scholar
[3] Pei J, Deng L, Song S, Zhao M, Zhang Y, Wu S, Wang G, Zou Z, Wu Z, He W, Chen F, Deng N, Wu S, Wang Y, Wu Y, Yang Z, Ma C, Li G, Han W, Li H, Wu H, Zhao R, Xie Y, Shi L 2019 Nature 572 106
Google Scholar
[4] LaValle S, Lesser E, Shockley R, Hopkins M S, Kruschwitz N 2011 MIT Sloan Manag. Rev. 52 21
[5] Shulaker M M, Hills G, Park R S, Howe R T, Saraswat K, Wong H S P, Mitra S 2017 Nature 547 74
Google Scholar
[6] Cai F, Correll J M, Lee S H, Lim Y, Bothra V, Zhang Z, Flynn M P, Lu W D 2019 Nat. Electron. 2 290
Google Scholar
[7] Huang H M, Wang Z, Wang T, Xiao Y, Guo X 2020 Adv. Intell. Syst. 2 2000149
Google Scholar
[8] Prezioso M, Merrikh-Bayat F, Hoskins B D, Adam G C, Likharev K K, Strukov D B 2015 Nature 521 61
Google Scholar
[9] Shao Z Y, Huang H M, Guo X 2021 Solid State Ionics 370 115746
Google Scholar
[10] Zhang X, Zhuo Y, Luo Q, Wu Z, Midya R, Wang Z, Song W, Wang R, Upadhyay N K, Fang Y, Kiani F, Rao M, Yang Y, Xia Q, Liu Q, Liu M, Yang J J 2020 Nat. Commun. 11 51
Google Scholar
[11] Lee D, Yun M J, Kim K H, Kim S, Kim H D 2021 ACS Sens. 6 4217
Google Scholar
[12] Imam N, Cleland T A 2020 Nat. Mach. Intell. 2 181
Google Scholar
[13] Hartridge H 1947 Nature 160 538
Google Scholar
[14] Hasler P, Smith P D, Graham D, Ellis R, Anderson D V 2005 IEEE Sens. J. 5 1027
Google Scholar
[15] Kim Y, Chortos A, Xu W, Liu Y, Oh J Y, Son D, Kang J, Foudeh A M, Zhu C, Lee Y, Niu S, Liu J, Pfattner R, Bao Z, Lee T W 2018 Science 360 998
Google Scholar
[16] Fonollosa J, Fernández L, Gutiérrez-Gálvez A, Huerta R, Marco S 2016 Sens. Actuators, B 236 1044
Google Scholar
[17] Kucic M, Hasler P, Dugger J, Anderson D 2001 Proceedings 2001 Conference on Advanced Research in VLSI. ARVLSI 2001 Salt Lake City, UT, March 14-16, 2001 pp148–162
[18] Rüedi P F, Heim P, Kaess F, Grenet E, Heitger F, Burgi P Y, Gyger S, Nussbaum P 2003 IEEE J. Solid-State Circuits 38 2325
[19] Yang R, Huang H M, Guo X 2019 Adv. Electron. Mater. 5 1900287
Google Scholar
[20] Wang W, Pedretti G, Milo V, Carboni R, Calderoni A, Ramaswamy N, Spinelli A S, Ielmini D 2018 Sci. Adv. 4 eaat4752
Google Scholar
[21] Lobo J L, Del Ser J, Bifet A, Kasabov N 2020 Neural Networks 121 88
Google Scholar
[22] Zhang W, Gao B, Tang J, Yao P, Yu S, Chang M F, Yoo H J, Qian H, Wu H 2020 Nat. Electron. 3 371
Google Scholar
[23] Wang Z, Joshi S, Savel’ev S, Song W, Midya R, Li Y, Rao M, Yan P, Asapu S, Zhuo Y, Jiang H, Lin P, Li C, Yoon J H, Upadhyay N K, Zhang J, Hu M, Strachan J P, Barnell M, Wu Q, Wu H, Williams R S, Xia Q, Yang J J 2018 Nat. Electron. 1 137
Google Scholar
[24] Xia Q, Yang J J 2019 Nat. Mater. 18 309
Google Scholar
[25] Huang H M, Yang R, Tan Z H, He H K, Zhou W, Xiong J, Guo X 2019 Adv. Mater. 31 1803849
Google Scholar
[26] Yi W, Tsang K K, Lam S K, Bai X, Crowell J A, Flores E A 2018 Nat. Commun. 9 4661
Google Scholar
[27] Yoon J H, Wang Z, Kim K M, Wu H, Ravichandran V, Xia Q, Hwang C S, Yang J J 2018 Nat. Commun. 9 417
Google Scholar
[28] Zhang X, Lu J, Wang Z, Wang R, Wei J, Shi T, Dou C, Wu Z, Zhu J, Shang D, Xing G, Chan M, Liu Q, Liu M 2021 Sci. Bull. 66 1624
Google Scholar
[29] Shaban A, Bezugam S S, Suri M 2021 Nat. Commun. 12 4234
[30] Wan C, Cai P, Wang M, Qian Y, Huang W, Chen X 2020 Adv. Mater. 32 e1902434
Google Scholar
[31] Yao P, Wu H, Gao B, Tang J, Zhang Q, Zhang W, Yang J J, Qian H 2020 Nature 577 641
Google Scholar
[32] Pfeiffer M, Pfeil T 2018 Front. Neurosci. 12 774
Google Scholar
[33] Roy K, Jaiswal A, Panda P 2019 Nature 575 607
Google Scholar
[34] Bellec G, Scherr F, Subramoney A, Hajek E, Salaj D, Legenstein R, Maass W 2020 Nat. Commun. 11 3625
Google Scholar
[35] Ponghiran W, Roy K 2021 arXiv: 2109.01905 [cs.NE]
[36] Wang T, Wang X, Wen J, Shao Z Y, Huang H M, Guo X 2022 Adv. Intell. Syst. 2200047
-
图 1 生物感官及仿生感觉系统 (a) 生物感官包括感觉神经元、突触及生物神经网络. 感觉神经元的感受器感知环境刺激, 并引起神经元膜电位变化, 当神经元膜电位达到阈值后, 产生神经脉冲, 神经脉冲信号通过突触传递至神经网络进行处理. (b) 仿生感觉系统包含了传感器、模拟信号处理单元以及类脑运算芯片. 传感器负责感知环境信息, 模拟信号处理单元负责对传感器信号进行预处理, 类脑芯片负责特征提取、分类以及预测等高级任务
Figure 1. Biological sensory organ and bio-inspired sensory system. (a) Biological sensory organ includes sensory neurons, synapses and neural network. The receptors of the sensory neurons capture the environmental information and cause the increase of the neuron potential. The neurons generate neural spikes when the neuron potential exceeds its threshold. The neural spikes are then transmitted to the neural network through synapses. (b) Bio-inspired sensory system includes sensors, analogue signal processing units and the brain-inspired chip. Sensors acquire the environmental information and convert the information into analogue electrical signals, which are then directly processed by the analogue signal processing units. Finally, the brain-inspired chip receives the pre-processed signals, and performs advanced tasks, such as feature extraction, classification and prediction.
图 2 传感器及模拟信号处理单元. 用不同的传感器阵列来模拟不同生物感官的感受器, 来实现对气体、光、声音、压力等信号的感知, 随后, 传感器信号直接输入到模拟信号处理单元中进行处理
Figure 2. Sensors and analogue signal processing units. Sensors work like the receptors in a biological sensory organ, detecting the environmental information such as gases, light, audio and pressure, and convert them to electrical sensory signals. The sensory signals are then directly processed by the analogue signal processing units.
图 3 类脑芯片 (a) 人工神经元: 采用忆阻器与电容并联的结构模拟神经元膜电位连续变化的特性, 实现神经元漏电流积分点火的功能; (b) 人工突触: 非易失性忆阻器电导的连续变化可以模拟生物突触权重的连续变化及突触可塑性, 此外, 忆阻器阵列还可以实现存内运算; (c) 类脑芯片架构: 由多个神经元-突触核心组成, 每个核心由基于忆阻器的人工神经元及人工突触组成, 每个核心通过总线连接
Figure 3. Brain-inspired chip. (a) Artificial neuron, consisting of a memristor and a parallel capacitor, emulates the continuous change of the neuron potential and realize the leaky integrate-and-fire function of a biological neuron. (b) Artificial synapses. Non-volatile memristors emulate the continuous change of the synaptic plasticity, in addition, the crossbar array based on memristors can realize the in-memory computing. (c) Architecture of the brain-inspired chip. A brain-inspired chip consists of many neuron-synapse cores based on memristors, and the cores are connected by bus lines.
图 4 脉冲神经网络 (a) 全连接脉冲神经网络; (b) 基于突触可塑性(脉冲时间依赖可塑性(STDP)和脉冲频率依赖可塑性(SRDP))来训练网络, 修改突触权重; c)卷积脉冲神经网络, 由卷积层、池化层和全连接层组成
Figure 4. Spiking neural network (SNN): (a) Fully connected SNN; (b) learning rules of SNN. The synaptic weights in SNN are updated according to STDP and SRDP; (c) convolutional SNN consisting of convolutional layers, pooling layers and fully connect layers.
图 5 仿生嗅觉系统. 气体传感器感知环境中的气体分子, 并将气体分子的化学信息转变为模拟型电信号; 传感信号由模拟信号处理单元作进一步处理, 该单元主要包含气体传感器基线动态校准电路以及运算放大电路; 经过预处理的传感信号传输至类脑芯片的神经元-突触核心中完成气体识别的任务
Figure 5. Bio-inspired olfactory system. In the system, sensor array detects the gas molecules in environment, converts the chemical information of gas molecules into electrical sensory signals, which are analogue. The sensory signals are directly processed by the analogue signal processing units which include baseline cancellation and amplifier circuits. After pre-processing, the sensory signals are sent to a neuron-synapse core of the brain-inspired chip to classify the gases.
-
[1] Zhou F, Chai Y 2020 Nat. Electron. 3 664
Google Scholar
[2] Wang T, Huang H M, Wang X X, Guo X 2021 InfoMat 3 804
Google Scholar
[3] Pei J, Deng L, Song S, Zhao M, Zhang Y, Wu S, Wang G, Zou Z, Wu Z, He W, Chen F, Deng N, Wu S, Wang Y, Wu Y, Yang Z, Ma C, Li G, Han W, Li H, Wu H, Zhao R, Xie Y, Shi L 2019 Nature 572 106
Google Scholar
[4] LaValle S, Lesser E, Shockley R, Hopkins M S, Kruschwitz N 2011 MIT Sloan Manag. Rev. 52 21
[5] Shulaker M M, Hills G, Park R S, Howe R T, Saraswat K, Wong H S P, Mitra S 2017 Nature 547 74
Google Scholar
[6] Cai F, Correll J M, Lee S H, Lim Y, Bothra V, Zhang Z, Flynn M P, Lu W D 2019 Nat. Electron. 2 290
Google Scholar
[7] Huang H M, Wang Z, Wang T, Xiao Y, Guo X 2020 Adv. Intell. Syst. 2 2000149
Google Scholar
[8] Prezioso M, Merrikh-Bayat F, Hoskins B D, Adam G C, Likharev K K, Strukov D B 2015 Nature 521 61
Google Scholar
[9] Shao Z Y, Huang H M, Guo X 2021 Solid State Ionics 370 115746
Google Scholar
[10] Zhang X, Zhuo Y, Luo Q, Wu Z, Midya R, Wang Z, Song W, Wang R, Upadhyay N K, Fang Y, Kiani F, Rao M, Yang Y, Xia Q, Liu Q, Liu M, Yang J J 2020 Nat. Commun. 11 51
Google Scholar
[11] Lee D, Yun M J, Kim K H, Kim S, Kim H D 2021 ACS Sens. 6 4217
Google Scholar
[12] Imam N, Cleland T A 2020 Nat. Mach. Intell. 2 181
Google Scholar
[13] Hartridge H 1947 Nature 160 538
Google Scholar
[14] Hasler P, Smith P D, Graham D, Ellis R, Anderson D V 2005 IEEE Sens. J. 5 1027
Google Scholar
[15] Kim Y, Chortos A, Xu W, Liu Y, Oh J Y, Son D, Kang J, Foudeh A M, Zhu C, Lee Y, Niu S, Liu J, Pfattner R, Bao Z, Lee T W 2018 Science 360 998
Google Scholar
[16] Fonollosa J, Fernández L, Gutiérrez-Gálvez A, Huerta R, Marco S 2016 Sens. Actuators, B 236 1044
Google Scholar
[17] Kucic M, Hasler P, Dugger J, Anderson D 2001 Proceedings 2001 Conference on Advanced Research in VLSI. ARVLSI 2001 Salt Lake City, UT, March 14-16, 2001 pp148–162
[18] Rüedi P F, Heim P, Kaess F, Grenet E, Heitger F, Burgi P Y, Gyger S, Nussbaum P 2003 IEEE J. Solid-State Circuits 38 2325
[19] Yang R, Huang H M, Guo X 2019 Adv. Electron. Mater. 5 1900287
Google Scholar
[20] Wang W, Pedretti G, Milo V, Carboni R, Calderoni A, Ramaswamy N, Spinelli A S, Ielmini D 2018 Sci. Adv. 4 eaat4752
Google Scholar
[21] Lobo J L, Del Ser J, Bifet A, Kasabov N 2020 Neural Networks 121 88
Google Scholar
[22] Zhang W, Gao B, Tang J, Yao P, Yu S, Chang M F, Yoo H J, Qian H, Wu H 2020 Nat. Electron. 3 371
Google Scholar
[23] Wang Z, Joshi S, Savel’ev S, Song W, Midya R, Li Y, Rao M, Yan P, Asapu S, Zhuo Y, Jiang H, Lin P, Li C, Yoon J H, Upadhyay N K, Zhang J, Hu M, Strachan J P, Barnell M, Wu Q, Wu H, Williams R S, Xia Q, Yang J J 2018 Nat. Electron. 1 137
Google Scholar
[24] Xia Q, Yang J J 2019 Nat. Mater. 18 309
Google Scholar
[25] Huang H M, Yang R, Tan Z H, He H K, Zhou W, Xiong J, Guo X 2019 Adv. Mater. 31 1803849
Google Scholar
[26] Yi W, Tsang K K, Lam S K, Bai X, Crowell J A, Flores E A 2018 Nat. Commun. 9 4661
Google Scholar
[27] Yoon J H, Wang Z, Kim K M, Wu H, Ravichandran V, Xia Q, Hwang C S, Yang J J 2018 Nat. Commun. 9 417
Google Scholar
[28] Zhang X, Lu J, Wang Z, Wang R, Wei J, Shi T, Dou C, Wu Z, Zhu J, Shang D, Xing G, Chan M, Liu Q, Liu M 2021 Sci. Bull. 66 1624
Google Scholar
[29] Shaban A, Bezugam S S, Suri M 2021 Nat. Commun. 12 4234
[30] Wan C, Cai P, Wang M, Qian Y, Huang W, Chen X 2020 Adv. Mater. 32 e1902434
Google Scholar
[31] Yao P, Wu H, Gao B, Tang J, Zhang Q, Zhang W, Yang J J, Qian H 2020 Nature 577 641
Google Scholar
[32] Pfeiffer M, Pfeil T 2018 Front. Neurosci. 12 774
Google Scholar
[33] Roy K, Jaiswal A, Panda P 2019 Nature 575 607
Google Scholar
[34] Bellec G, Scherr F, Subramoney A, Hajek E, Salaj D, Legenstein R, Maass W 2020 Nat. Commun. 11 3625
Google Scholar
[35] Ponghiran W, Roy K 2021 arXiv: 2109.01905 [cs.NE]
[36] Wang T, Wang X, Wen J, Shao Z Y, Huang H M, Guo X 2022 Adv. Intell. Syst. 2200047
DownLoad:
Catalog
Metrics
- Abstract views: 6278
- PDF Downloads: 408
- Cited By: 0
