Learning through ferroelectric domain dynamics in solid-state synapses

1Cortical information flows from neuron to neuron through synapses of variable connection strength. The overall distribution of the synaptic strengths provides the neural network with memory, while learning is achieved through the synapses' reconfiguration (that is, plasticity). Several mechanisms regulating the evolution of the synaptic strengths have been proposed. A particularly promising one is spike-timing-dependent plasticity (STDP) through which the synaptic strengths evolve depending on the timing and causality of electrical signals from neighbouring neurons (sketch in Fig. 1a). As observed in biological systems, STDP enables learning without any external control on the synaptic strengths or any previous knowledge of the information to be processed. This makes STDP the basis for autonomous, unsupervised learning. The implementation of STDP in artificial neural networks thus emerges as a crucial milestone towards the realization of self-adaptive electronic architectures.

2An electronic equivalent of the synapse for artificial neural networks is the memristor, a nanoscale device whose resistance depends on the history of electrical signals it was previously subjected to (ref. ). Memristors thus exhibit plasticity and their conductance can emulate synaptic strength, so that a low resistance corresponds to a strong synaptic connection and a high resistance corresponds to a weak synaptic connection, respectively. Most memristors operate through the motion of ions or atoms in binary oxides (TiO2 (refs, ), Ta2O5 (ref. ) and so on), Ag-Si/Ag-S nanocomposites or phase change materials. Recently, STDP was demonstrated in individual devices based on such memristor technologies, confirming the potential of memristors for autonomous learning. However, the connection between the STDP process and the physical mechanisms underlying resistance changes in these memristors is unclear. In addition, continual variations of the memristor conductance are required for learning new features under an incessant information flow. Here we show that resorting to purely electronic memristors with a high endurance, operating on well-established physical principles is a decisive asset for the future implementation of unsupervised learning in high-density memristive crossbar arrays.

3We work with purely electronic memristors based on ferroelectric tunnel junctions (FTJs), in which an ultrathin ferroelectric film is sandwiched between two electrodes. In such devices, sketched in Fig. 1b, the junction resistance sensitively depends on the relative fraction of ferroelectric domains with polarization pointing towards one or the other electrode. Applying voltage pulses to the device modifies the domain population, thereby inducing resistance changes. Furthermore, supertetragonal BiFeO3 (BFO) tunnel barriers combined with (Ca,Ce)MnO3 (CCMO) bottom and Co top electrodes give rise to OFF/ON resistance ratios up to 104 (ref. ) paired with high endurance and operation speed.

4Figure 1c shows the dependence of the junction resistance with the amplitude of 100 ns voltage pulses in a Co/BFO/CCMO junction (Methods). In this hysteresis cycle, one clearly notes the existence of voltage thresholds beyond which switching between low- and high- (high- and low-) resistance states occurs. The existence of such well-defined voltage thresholds (associated with the coercivity of the ferroelectric) makes it possible to implement STDP in these FTJs. According to STDP, if the pre-neuron spikes just before the post-neuron—indicating a ‘causal' relationship—the synapse should be strengthened whereas if the pre-neuron spikes just after the post-neuron—indicating an ‘anticausal' relationship—the synapse should be weakened. We emulate the spikes from pre- and post-neurons (sketched in Fig. 1a) by the waveforms shown in Fig. 1b: rectangular voltage pulses followed by smooth slopes of opposite polarity. Importantly, the voltage never exceeds Vth, so that a single spike cannot induce a change in resistance.

5When both pre- and post-neuron spikes reach the memristor with a delay Δt, their superposition produces the waveforms (Vpre−Vpost) displayed in the inset of Fig. 1d. The resulting combined waveform transitorily exceeds the threshold voltage, leading to an increase (ΔG>0, synapse strengthening) or a decrease (ΔG<0, synapse weakening) of the FTJ conductance (G), depending on the sign of Δt. As can be seen from the experimental STDP curve in Fig. 1d, only closely timed spikes produce a conductance change whereas long delays leave the device unchanged.

6Modelling the shape of the STDP curve requires understanding the physical process underlying the time-dependent variation of the FTJ conductance while the waveform is applied. For this purpose, we image the pseudo real-time evolution of the ferroelectric domain configuration by means of stroboscopic piezoresponse force microscopy (PFM), while simultaneously measuring the electrical properties of the device. Figure 2a shows the PFM phase and amplitude signals after cumulative pulses of constant amplitude (1 V) from the ON to the OFF state. The evolution of the phase images in stroboscopic PFM reveals the gradual reversal of the polarization from up (dark domains) to down (bright domains) and is reminiscent of the polarization reversal with increasing voltages previously observed by PFM. The weak amplitude signal during reversal (for example, stages 2 and 3) indicates that the ferroelectric system is split into many small (≲10 nm) domains and that polarization switching is governed by the inhomogeneous nucleation of new domains rather than by the expansion of existing ones. We complemented these experiments by molecular dynamics (MD) simulations (Methods) on defect-free supertetragonal BFO films. These computations also yield an inhomogeneous character for polarization switching (Supplementary Fig. 1), therefore implying that this process is intrinsic in nature, that is, one does not require the presence of defects to obtain an inhomogeneous switching in supertetragonal BFO. Such findings contrast with the case of bulk-like BFO (rhombohedral phase) for which a homogeneous switching was recently predicted as a result of large oxygen octahedra tilts that provide a specific homogeneous path for polarization reversal.

7We extract the normalized reversed area S from the phase images and plot it as a function of the cumulated pulse duration in Fig. 2b (black squares). Owing to the direct link between the junction resistance R and this normalized reversed area S (well described by a simple model of parallel resistances,, ref. ), one can also extract S from measurements of the junction resistance after consecutive pulses of 1 V (green squares in Fig. 2b; note the excellent agreement between the PFM and transport data). Figure 2c shows transport data sets at different pulse amplitudes. They follow a systematic trend where, for a given cumulated pulse duration, the switched area is larger under higher voltages.

8In ferroelectrics, inhomogeneous polarization switching can be described by a nucleation-limited model, which considers that the ferroelectric film is composed of different zones with independent switching kinetics. Assuming for each voltage V a broad Lorentzian distribution of the logarithm of nucleation times—with width Γ(V) and centred at log(tmean(V))—the normalized reversed area S can be approximated as a function of time t and applied voltage V (ref. ):

9where the index relates to the sign of the applied voltage. Fits obtained with this expression accurately reproduce the experimental data of ferroelectric switching as a function of time and voltage (black lines in Fig. 2c). Figure 2d displays several representative distributions of switching times and illustrates the main trends. For larger voltages, switching occurs earlier and in a narrower time window (decrease of tmean and Γ), in agreement with previous results obtained on thick ferroelectric capacitors. As shown in the inset of Fig. 2d, MD simulations performed at 10 K confirm the relevance of equation (1) (Methods and Supplementary Fig. 2) to characterize the switching process as well as the measured trends of tmean and Γ with the magnitude of the electric field. Figure 2e shows that the evolution of the switching time (tmean) as a function of the inverse electric field, extracted from transport measurements (Fig. 2c), follows the characteristic Merz's law for ferroelectric switching (tmean∝exp(α/E)) where the activation field α is of the order of 3.0 V nm−1. Due to the idealized nature of BFO during MD simulations (no interface, no defects, no tunnel current), the timescale for polarization switching is much shorter than in experiments, while the electric field is larger. Nevertheless, Merz's law can be applied to the MD simulation results (inset of Fig. 2e) and indicates an activation field of 2.4 V nm−1, that is, in the same range as for experimental results. These simulation results strongly suggest that the experimentally observed inhomogeneous polarization switching in ultrathin films of BFO has an intrinsic origin.

10Because of the direct relationship between S and R in these devices, the accurate description of ferroelectric switching by the nucleation-limited model can be further extended to the modelling of resistance changes as a function of voltage amplitude. Figure 3a shows nested resistance hysteresis loops as a function of voltage pulse amplitude, characteristic of memristors. The STDP curve of the same device is displayed in Fig. 3b. Using equation (1) in combination with the parallel resistance model (Methods and Supplementary Fig. 3), we simultaneously fit the resistance versus voltage cycles and the STDP curve. Both resistance changes can be accurately replicated (solid lines in Fig. 3a,b) using the same set of parameters tmean(V) and Γ(V).

11This full description for each memristor device makes it possible to predict the conductance changes driving STDP learning in ferroelectric synapses. In Fig. 3b–d, we apply different voltage waveforms to our memristors to emulate various types of pre- and post-neuron activities. This procedure allows the generation of biologically realistic, though accelerated (Fig. 3b,c), or artificially designed (Fig. 3d) STDP learning curves. Using our model with the parameters extracted previously, we can now predict the conductance changes for these specific types of STDP. Figure 3c,d shows the excellent agreement between these predictions and the measured conductance variations associated with different STDP waveforms.

12We now use this physical model to simulate unsupervised learning in a spiking neural network with ferroelectric synapses. These simulations serve as a test bench to investigate the influence of the STDP waveform shape on the ability of the network to recognize patterns in images (here the horizontal, diagonal, and vertical bars labelled A, B and C, as shown in the inset of Fig. 4a). The simulated network is built around a crossbar of 9 × 5 ferroelectric memristors (Fig. 4a). Each of the nine spiking input neurons codes for one pixel of noisy images that contain one out of the three patterns to recognize (Methods). The output neurons integrate the input signals flowing through the memristors along each row and fire when a threshold is reached. Figure 4b shows that the recognition rate of the network increases with the number of image presentations, reaching 100% for low noise levels and almost 80% for high (of the order of the input amplitude) noise levels. The evolution of the conductance of the nine memristors in each row is shown in the inset of Fig. 4b as coloured 3 × 3 pixel images. As the network is learning, the conductance images in rows 1, 2 and 4 each converge toward one out of the three input patterns, whereas the memristor conductances in rows 3 and 5 remain random. After learning, as illustrated in Fig. 4c, three of the output neurons (neurons 1, 2 and 4 in this example) have specialized to each of the three input patterns and fire when the corresponding input is presented. Our simulations reveal that successful unsupervised learning is highly dependent on the exact shape of the pre- and post-neuron spike waveforms. Figure 4d shows that, for the case presented before, the recognition rate reaches 100% for well-chosen post-neuron spike amplitudes in the range of 0.82–0.98 V. However, when the amplitude of the post-neuron spike waveform is only slightly lower or higher, the recognition rate rapidly drops. Our simulations therefore emphasize the importance of a precise knowledge of the memristor dynamics, and therefore of its accurate description on the basis of a physical model.

13In summary, we have established that STDP can be harnessed from intrinsically inhomogeneous polarization switching in ferroelectric memristors. Combining time-dependent transport measurements, ferroelectric domain imaging, and effective-Hamiltonian-based atomistic MD simulations, we show that the ferroelectric switching underlying resistive changes in these devices can be described by a well-established nucleation-limited model. Using this physical model, we can reliably predict the conductance evolution of ferroelectric synapses with varying neural inputs. These results pave the way toward low-power hardware implementations of billions of reliable and predictable artificial synapses (such as deep neural networks) in future brain-inspired computers.

14The BiFeO3 (4.6 nm)/Ca0.96Ce0.04MnO3 (20 nm) thin films are grown on YAlO3 (YAO) (001) substrates by pulsed laser deposition using a Nd:YAG laser (details of the growth can be found in ref. ). The thin films are epitaxial and fully strained by the substrates as characterized by X-ray diffraction experiments and scanning transmission electron microscopy on cross-section samples. The high compressive strain imposed by the YAO substrate on BiFeO3 (BFO) stabilizes the supertetragonal polymorph of BFO with giant tetragonality and a potentially large polarization pointing initially towards the bottom electrode of Ca0.96Ce0.04MnO3 (CCMO).

15FTJs are then fabricated by defining Pt (10−90 nm)/Co (5−10 nm) nanopillar electrodes (with diameters of 180–500 nm) on top of the unpatterned BFO/CCMO heterostructure, combining conventional electron-beam lithography and lift-off processes. For experiments involving electron transport and ferroelectric domain imaging experiments (as in Fig. 2), arrays of nanopillars are electrically contacted with a conducting atomic force microscopy tip. For all other electronic transport experiments presented in the manuscript, fully patterned junctions are defined by a four-step lithography process following the definition of these nanopillars (details of the fabrication process can be found in ref. ). Such fully patterned junctions are macroscopically connected by standard radio frequency (RF) probes.

16Combined transport and PFM measurements are performed using solid-state FTJs with diameters of 180 nm, as detailed in ref.. These Pt/Co/BFO/CCMO junctions are connected electrically by a conductive atomic force microscopy tip to allow resistance measurements under constant dc voltage (−0.1 V, time constant 100 ms) and the application of 100-ns write pulses in combination with subsequent PFM imaging (Vac=0.6 V, Vdc=−0.17 V, f=14 kHz). The bias voltage is applied to the tip while the bottom electrode is grounded.

17The resistance–voltage cycles and STDP curves were obtained on fully processed FTJs with diameters of 400 and 500 nm, described in ref.. The resistance of the junctions was measured after each pulse at a low bias of −200 mV (time constant 100 ms). To obtain the STDP curve, we always initialize the junction to the ON state for Δt<0 and to an intermediate resistance state between ON and OFF for Δt>0. The exact values of initial resistance states in each figure are

18Figs 1d and 3b: Rinit(Δt<0)=6 × 105 Ω, Rinit(Δt>0)=1.2 × 107 Ω,

19Fig. 3c: Rinit=1.2 × 107 Ω,

20Fig. 3d: Rinit(Δt<0)=6 × 105 Ω, Rinit(Δt>0)=1.2 × 107 Ω.

21The switching process under dc electric fields is simulated using a recently developed scheme, in which the total energy is provided by the first-principles-based effective Hamiltonian method of refs, and inserted into an original MD technique treating on the same footing dynamics of ions and magnetic degrees of freedom. A periodic 48 × 48 × 6 supercell (69,120 atoms) of the supertetragonal BFO (P4mm space group) is adopted and the electric field is constantly applied along the pseudocubic direction (opposite to the initial polarization direction). The temperature is kept at 10 K so as to minimize thermal fluctuations and the time step is set to 0.5 fs. Supplementary Fig. 1a shows polarization patterns at different times under a dc electric field of 2.5 V nm−1, which illustrate the (intrinsic) inhomogeneous switching of polarization in bulk-like supertetragonal BFO. The switched area as function of time is plotted in Supplementary Fig. 1b. Note that the reversed area is defined as any polarization that has a magnitude larger than 7 μC cm−2 (it reaches 100 μC cm−2 when being fully switched) in the direction. Simulation results of polarization switching under different electric fields are summarized in Supplementary Fig. 2. Fits from equation (1) are represented as black lines. Lorentzian switching distributions resulting from the fits are displayed in the inset of Fig. 2d and the switching times are plotted in the inset of Fig. 2e.

22The nucleation-limited model as expressed by equation (1) describes the full reversal of the ferroelectric domains starting from a uniform state under constant voltage amplitude. To describe the ferroelectric switching dynamics under the influence of arbitrary voltage waveforms, the waveform is numerically divided in short segments of width Δt (Supplementary Fig. 3a). For sufficiently small Δt, we can then assume a constant applied voltage Vi. As the reversal dynamics depends on the present domain configuration and therefore on the value Si−1 before the segment i, we compute the time offset corresponding to a voltage Vi and can only then apply equation (1) to calculate the value (Supplementary Fig. 3b). Owing to the highly nonlinear reversal dynamics of FTJs where high voltage amplitudes govern the reversal process, the value of Δt has to be chosen small enough not to smear out these maximal amplitudes. For the results presented here, we chose Δt⩽20 ns.

23A simple yet rigorous model connects the electrical resistance R with the ferroelectric domain configuration characterized by the normalized area of down domains S. Assuming parallel conduction through areas with ferroelectric down domains and areas with up domains allows to calculate one from another using the relation, where RON and ROFF are the resistance values in the low- and high-resistance states, respectively.

24The simulated spiking neural network consists of nine input neurons fully connected to five output neurons by a crossbar of 9 × 5 identical ferroelectric memristors, as depicted in Fig. 4a. Inputs are consecutively presented to the network in the form of grey-level images of 3 × 3 pixels composed of one of the three patterns to recognize (horizontal, diagonal, and vertical bars labelled A, B and C in the inset of Fig. 4a) and an additional uniformly distributed random noise of the specified amplitude. Subsequently, the values of each image are normalized to the interval [0, 1]. Each individual pixel corresponds to one input neuron that integrates the respective input value over time (using a fixed time step) and spikes once it reaches a certain threshold Qth. This implementation effectively results in a frequency coding of the input pixel values. The spikes from each input neuron are transmitted to all output neurons (as in a fully connected network) that integrate them weighted by the corresponding connection strength, that is, by the normalized conductance value of the respective FTJ. Once an output neuron (#1 to #5 in Fig. 4) reaches the threshold value Qth, it spikes on its part and generates an output spike. Lateral inhibition then resets all output neurons. We use the first output spike after the presentation of a new input image to determine if the network has successfully recognized the pattern (A, B or C as shown in Fig. 4a) of the input image. As for unsupervised learning the labels of the output neurons are not known beforehand, the labelling is done algorithmically for each repetition of the simulation. We define a successful learning as when each pattern is immediately recognized after 200 input image presentations. The recognition rates shown in Fig. 4b,d are calculated as the mean of 100 simulation runs for each parameter set. At the beginning of each simulation run, the memristor conductances are initialized to random values between those of their ON and OFF states. Variations between the runs using identical parameters are therefore due to the randomized initial conductance values of the synapses as well as the random noise that is added to the input patterns.

25The data that support the findings of this study are available from the corresponding authors upon request.

26How to cite this article: Boyn, S. et al. Learning through ferroelectric domain dynamics in solid-state synapses. Nat. Commun. 8, 14736 doi: 10.1038/ncomms14736 (2017).

27Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.