No retraction, correction or concern found when last checked on Oct 26, 2026.
An empirical survey of data augmentation for time series classification with neural networks
Brian Kenji Iwana, Seiichi Uchida
PLOS ONE · 2021 · CC BY
doi:10.1371/journal.pone.0254841
Abstract
In recent times, deep artificial neural networks have achieved many successes in pattern recognition. Part of this success can be attributed to the reliance on big data to increase generalization. However, in the field of time series recognition, many datasets are often very small. One method of addressing this problem is through the use of data augmentation. In this paper, we survey data augmentation techniques for time series and their application to time series classification with neural networks. We propose a taxonomy and outline the four families in time series data augmentation, including transformation-based methods, pattern mixing, generative models, and decomposition methods. Furthermore, we empirically evaluate 12 time series data augmentation methods on 128 time series classification datasets with six different types of neural networks. Through the results, we are able to analyze the characteristics, advantages and disadvantages, and recommendations of each data augmentation method. This survey aims to help in the selection of time series data augmentation for neural network applications.
Full text
Its licence does not allow us to show the full text. Read it at PubMed Central:
Read at PubMed CentralReferences 118
- 1.
Ding H, Trajcevski G, Scheuermann P, Wang X, Keogh E. Querying and mining of time series data. Proc Very Larg Data Base Endow. 2008;1(2):1542–1552.
Not in our library
- 2.
Schmidhuber J. Deep learning in neural networks: An overview. Neural Networks. 2015;61:85–117
Not in our library10.1016/j.neunet.2014.09.003
- 3.
Wang Z, Yan W, Oates T. Time series classification from scratch with deep neural networks: A strong baseline. In: IJCNN; 2017.
Not in our library
- 4.
Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nat. 1986;323(6088):533–536
Not in our library10.1038/323533a0
- 5.
Chen Q, Liang B, Wang J. A comparative study of LSTM and phased LSTM for gait prediction. Int J Artificial Intelli & App. 2019;10(4):57–66
Not in our library10.5121/ijaia.2019.10405
- 6.
Kluwak K, Nizynski T. Gait classification using LSTM networks for tagging system. In: IEEE SoSE; 2020.
Not in our library
- 7.
Xu G, Xing G, Jiang J, Jiang J, Ke Y. Arrhythmia detection using gated recurrent unit network with ECG signals. J Medical Imag and Health Inform. 2020;10(3):750–757
Not in our library10.1166/jmihi.2020.2928
- 8.
Kim BH, Pyun JY. ECG identification for personal authentication using LSTM-based deep recurrent neural networks. Sensors. 2020;20(11):3069
Not in our library10.3390/s20113069
- 9.
Sun L, Su T, Liu C, Wang R. Deep LSTM networks for online Chinese handwriting Recognition. In: ICFHR; 2016.
Not in our library
- 10.
Carbune V, Gonnet P, Deselaers T, Rowley HA, Daryin A, Calvo M, et al. Fast multi-language LSTM-based online handwriting recognition. Int J Doc Analy and Recogn. 2020;23(2):89–102
Not in our library10.1007/s10032-020-00350-4
- 11.
Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86(11):2278–2324
Not in our library10.1109/5.726791
- 12.
Bai S, Kolter JZ, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:180301271. 2018;.
Not in our library
- 13.
Iwana BK, Uchida S. Time series classification using local distance-based features in multi-modal fusion networks. Pattern Recogn. 2020;97:107024
Not in our library10.1016/j.patcog.2019.107024
- 14.
Banko M, Brill E. Scaling to very very large corpora for natural language disambiguation. In: AMACL; 2001.
Not in our library
- 15.
Torralba A, Fergus R, Freeman WT. 80 Million tiny images: A large data set for nonparametric object and scene recognition. IEEE Trans Pattern Anal and Mach Intell. 2008;30(11):1958–1970
Not in our library10.1109/tpami.2008.128
- 16.
Dau HA, Keogh E, Kamgar K, Yeh CCM, Zhu Y, Gharghabi S, et al. The UCR time series classification archive; 2018.
Not in our library
- 17.
- 18.
Olson M, Wyner A, Berk R. Modern neural networks generalize on small data sets. In: NeurIPS; 2018. p. 3619–3628.
Not in our library
- 19.
- 20.
Hasibi R, Shokri M, Dehghan M. Augmentation scheme for dealing with imbalanced network traffic classification using deep learning. arXiv preprint arXiv:190100204. 2019;.
Not in our library
- 21.
Krizhevsky A. Learning multiple layers of features from tiny images. University of Tront Thesis. 2009;.
Not in our library
- 22.
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis. 2015;115(3):211–252
Not in our library10.1007/s11263-015-0816-y
- 23.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. 2014;.
Not in our library
- 24.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: IEEE CVPR; 2016.
Not in our library
- 25.
Huang G, Liu Z, Maaten LVD, Weinberger KQ. Densely connected convolutional networks. In: IEEE CVPR; 2017.
Not in our library
- 26.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: IEEE CVPR; 2015.
Not in our library
- 27.
Wen Q, Sun L, Song X, Gao J, Wang X, Xu H. Time series data augmentation for deep learning: A survey. arXiv preprint arXiv:200212478. 2020;.
Not in our library
- 28.
Fields T, Hsieh G, Chenou J. Mitigating drift in time series data with noise augmentation. In: ICCSCI; 2019.
Not in our library
- 29.
Le Guennec A, Malinowski S, Tavenard R. Data augmentation for time series classification using convolutional neural networks. In: IWAATD; 2016.
Not in our library
- 30.
Um TT, Pfister FMJ, Pichler D, Endo S, Lang M, Hirche S, et al. Data augmentation of wearable sensor data for Parkinson’s disease monitoring using convolutional neural networks. In: ACM ICMI; 2017. p. 216–220.
Not in our library
- 31.
Jaitly N, Hinton GE. Vocal tract length perturbation (VTLP) improves speech recognition. In: ICML WDLASLP; 2013.
Not in our library
- 32.
Cao H, Tan VYF, Pang JZF. A parsimonious mixture of Gaussian trees model for oversampling in imbalanced and multimodal time-series classification. IEEE Trans Neural Networks and Learning Sys. 2014;25(12):2226–2239
Not in our library10.1109/tnnls.2014.2308321
- 33.
Wendling F, Bellanger JJ, Bartolomei F, Chauvel P. Relevance of nonlinear lumped-parameter models in the analysis of depth-EEG epileptic signals. Bio Cybernetics. 2000;83(4):367–378
Not in our library10.1007/s004220000160
- 34.
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: NeurIPS; 2014. p. 2672–2680.
Not in our library
- 35.
Bergmeir C, Hyndman RJ, Benítez JM. Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation. Int J Forecasting. 2016;32(2):303–312
Not in our library10.1016/j.ijforecast.2015.07.002
- 36.
Eltoft T. Data augmentation using a combination of independent component analysis and non-linear time-series prediction. In: IJCNN; 2002. p. 448–453.
Not in our library
- 37.
Forestier G, Petitjean F, Dau HA, Webb GI, Keogh E. Generating synthetic time series to augment sparse datasets. In: IEEE ICDM; 2017.
Not in our library
- 38.
Liu B, Zhang Z, Cui R. Efficient Time Series Augmentation Methods. In: CISP-BMEI; 2020.
Not in our library
- 39.
Iwana BK, Uchida S. Time series data augmentation for neural networks by time warping with a discriminative teacher. In: ICPR; 2021.
Not in our library
- 40.
Kamycki K, Kapuscinski T, Oszust M. Data augmentation with suboptimal warping for time-series classification. Sensors. 2019;20(1):98
Not in our library10.3390/s20010098
- 41.
Huang CL. Exploring effective data augmentation with TDNN-LSTM neural network embedding for speaker recognition. In: IEEE ASRUW; 2019. p. 291–295.
Not in our library
- 42.
Rashid KM, Louis J. Time-warping: A time series data augmentation of IMU data for construction equipment activity identification. In: ISARC; 2019.
Not in our library
- 43.
Bishop CM. Training with noise is equivalent to Tikhonov regularization. Neural Computation. 1995;7(1):108–116
Not in our library10.1162/neco.1995.7.1.108
- 44.
An G. The effects of adding noise during backpropagation Training on a Generalization Performance. Neural Computation. 1996;8(3):643–674
Not in our library10.1162/neco.1996.8.3.643
- 45.
Arslan M, Guzel M, Demirci M, Ozdemir S. SMOTE and Gaussian noise based sensor data augmentation. In: ICCSE; 2019.
Not in our library
- 46.
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: Synthetic minority over-sampling technique. J Art Intelli Research. 2002;16:321–357
Not in our library10.1613/jair.953
- 47.
Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller PA. Data augmentation using synthetic data for time series classification with deep residual networks. In: IWAATD; 2018.
Not in our library
- 48.
Ohashi H, Ahmed S, Akiyama T, Sato T, Nguyen P, Nakamura K, et al. Augmenting wearable sensor data with physical constraint for DNN-based human-action recognition. In: ICML Workshops; 2017.
Not in our library
- 49.
Delgado-Escano R, Castro FM, Cozar JR, Marin-Jimenez MJ, Guil N. An end-to-end multi-task and fusion CNN for inertial-based gait recognition. IEEE Access. 2019;7:1897–1908
Not in our library10.1109/access.2018.2886899
- 50.
Tran L, Choi D. Data augmentation for inertial sensor-based gait deep neural network. IEEE Access. 2020;8:12364–12378
Not in our library10.1109/access.2020.2966142
- 51.
Pan Q, Li X, Fang L. Data augmentation for deep learning-based ECG analysis. In: Feature Eng. and Comput. Intell. in ECG Monitor. Springer; 2020. p. 91–111.
Not in our library
- 52.
Eyobu OS, Han D. Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network. Sensors. 2018;18(9):2892
Not in our library10.3390/s18092892
- 53.
Nagano T, Fukuda T, Suzuki M, Kurata G. Data augmentation based on vowel stretch for improving children’s speech recognition. In: IEEE ASRUW; 2019. p. 502–508.
Not in our library
- 54.
Nguyen TS, Stuker S, Niehues J, Waibel A. Improving sequence-to-sequence speech recognition training with on-the-fly data augmentation. In: IEEE ICASSP; 2020.
Not in our library
- 55.
Vachhani B, Bhat C, Kopparapu SK. Data augmentation using healthy speech for dysarthric speech recognition. In: Interspeech; 2018.
Not in our library
- 56.
Cui X, Goel V, Kingsbury B. Data augmentation for deep neural network acoustic modeling. In: IEEE ICASSP; 2014.
Not in our library
- 57.
Lee L, Rose R. A frequency warping approach to speaker normalization. IEEE Trans Speech and Audio Process. 1998;6(1):49–60
Not in our library10.1109/89.650310
- 58.
Adachi S, Takemoto H, Kitamura T, Mokhtari P, Honda K. Vocal tract length perturbation and its application to male-female vocal tract shape conversion. J Acousitical Soc of America. 2007;121(6):3874–3885
Not in our library10.1121/1.2730743
- 59.
Ko T, Peddinti V, Povey D, Khudanpur S. Audio augmentation for speech recognition. In: Interspeech; 2015.
Not in our library
- 60.
Kim C, Shin M, Garg A, Gowda D. Improved vocal tract length perturbation for a state-of-the-art end-to-end speech recognition system. In: Interspeech; 2019.
Not in our library
- 61.
Gao J, Song X, Wen Q, Wang P, Sun L, Xu H. RobustTAD: Robust time series anomaly detection via decomposition and convolutional neural networks. arXiv preprint arXiv:200209545. 2020;.
Not in our library
- 62.
Park DS, Chan W, Zhang Y, Chiu CC, Zoph B, Cubuk ED, et al. SpecAugment: A simple data augmentation method for automatic speech recognition. In: Interspeech; 2019.
Not in our library
- 63.
Khadijah, Endah SN, Kusumaningrum R, Rismiyati. The study of synthetic minority over-sampling technique (SMOTE) and weighted extreme learning machine for handling imbalance problem on multiclass microarray classification. In: ICICoS; 2018.
Not in our library
- 64.
Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. Safe-level-SMOTE: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Adv. in Knowl. Disc. and Data Mining. Springer; 2009. p. 475–482.
Not in our library
- 65.
Tarawneh AS, Hassanat ABA, Almohammadi K, Chetverikov D, Bellinger C. SMOTEFUNA: Synthetic minority over-sampling technique based on furthest neighbour algorithm. IEEE Access. 2020;8:59069–59082
Not in our library10.1109/access.2020.2983003
- 66.
Zhou C, Liu B, Wang S. CMO-SMOTE: Misclassification cost minimization oriented synthetic minority oversampling technique for imbalanced learning. In: IHMSC; 2016.
Not in our library
- 67.
Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. DBSMOTE: Density-based synthetic minority over-sampling technique. Applied Intell. 2011;36(3):664–684
Not in our library10.1007/s10489-011-0287-y
- 68.
Gong Z, Chen H. Model-based oversampling for imbalanced sequence classification. In: ACM ICIKM; 2016.
Not in our library
- 69.
Sawicki A, Zieliński SK. Augmentation of segmented motion capture data for improving generalization of deep neural networks. In: CISIM. Springer; 2020. p. 278–290.
Not in our library
- 70.
DeVries T, Taylor GW. Dataset augmentation in feature space. arXiv preprint arXiv:170205538. 2017;.
Not in our library
- 71.
- 72.
Savitzky A, Golay MJE. Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry. 1964;36(8):1627–1639
Not in our library10.1021/ac60214a047
- 73.
Sakoe H, Chiba S. Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans Acoustics, Speech, and Sig Process. 1978;26(1):43–49
Not in our library10.1109/tassp.1978.1163055
- 74.
Takahashi N, Gygli M, Pfister B, Gool LV. Deep convolutional neural networks and data augmentation for acoustic event recognition. In: Interspeech; 2016.
Not in our library
- 75.
Nanni L, Maguolo G, Paci M. Data augmentation approaches for improving animal audio classification. Ecological Informatics. 2020;57:101084
Not in our library10.1016/j.ecoinf.2020.101084
- 76.
Petitjean F, Ketterlin A, Gançarski P. A global averaging method for dynamic time warping, with applications to clustering. Pattern Recogn. 2011;44(3):678–693
Not in our library10.1016/j.patcog.2010.09.013
- 77.
Smyl S, Kuber K. Data preprocessing and augmentation for multiple short time series forecasting with recurrent neural networks. In: ISF; 2016.
Not in our library
- 78.
Kang Y, Hyndman RJ, Li F. GRATIS: GeneRAting TIme Series with diverse and controllable characteristics. Stat Anal and Data Mining: The ASA Data Sci J. 2020
Not in our library10.1002/sam.11461
- 79.
Tanner MA, Wong WH. The calculation of posterior distributions by data augmentation. J American Stat Assoc. 1987;82(398):528–540
Not in our library10.2307/2289463
- 80.
Fr uhwirth-Schnatter S. Data augmentation and dynamic linear models. J Time Series Anal. 1994;15(2):183–202
Not in our library10.1111/j.1467-9892.1994.tb00184.x
- 81.
Meng XL. Seeking efficient data augmentation schemes via conditional and marginal augmentation. Biometrika. 1999;86(2):301–320
Not in our library10.1093/biomet/86.2.301
- 82.
Hou Y, Liu Y, Che W, Liu T. Sequence-to-sequence data augmentation for dialogue language understanding. arXiv preprint arXiv:180701554. 2018;.
Not in our library
- 83.
Longpre S, Lu Y, Tu Z, DuBois C. An exploration of data augmentation and sampling techniques for domain-agnostic question answering. In: WMRQA. ACL; 2019.
Not in our library
- 84.
Oord Avd, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, et al. Wavenet: A generative model for raw audio. arXiv preprint arXiv:160903499. 2016;.
Not in our library
- 85.
Wang J, Kim S, Lee Y. Speech augmentation using Wavenet in speech recognition. In: IEEE ICASSP; 2019.
Not in our library
- 86.
Tu J, Liu H, Meng F, Liu M, Ding R. Spatial-temporal data augmentation based on LSTM autoencoder network for skeleton-based human action recognition. In: IEEE ICIP; 2018.
Not in our library
- 87.
Lou H, Qi Z, Li J. One-dimensional data augmentation using a Wasserstein generative adversarial network with supervised signal. In: CCDC; 2018.
Not in our library
- 88.
Haradal S, Hayashi H, Uchida S. Biosignal data augmentation based on generative adversarial networks. In: EMBC; 2018.
Not in our library10.1109/embc.2018.8512396
- 89.
Esteban C, Hyland SL, Rätsch G. Real-valued (medical) time series generation with recurrent conditional GANs. arXiv preprint arXiv:170602633. 2017;.
Not in our library
- 90.
Ramponi G, Protopapas P, Brambilla M, Janssen R. T-CGAN: Conditional generative adversarial network for data augmentation in noisy time series with irregular sampling. arXiv preprint arXiv:181108295. 2018;.
Not in our library
- 91.
Che Z, Cheng Y, Zhai S, Sun Z, Liu Y. Boosting deep learning risk prediction with generative adversarial networks for electronic health records. In: IEEE ICDM; 2017.
Not in our library
- 92.
Chen G, Zhu Y, Hong Z, Yang Z. EmotionalGAN: Generating ECG to enhance emotion state classification. In: AICS; 2019.
Not in our library
- 93.
Hatamian FN, Ravikumar N, Vesal S, Kemeth FP, Struck M, Maier A. The effect of data augmentation on classification of atrial fibrillation in short single-lead ECG signals using deep neural networks. In: IEEE ICASSP; 2020.
Not in our library
- 94.
Madhu A, Kumaraswamy S. Data augmentation using generative adversarial network for environmental sound classification. In: ESPC; 2019.
Not in our library
- 95.
Zhu F, Ye F, Fu Y, Liu Q, Shen B. Electrocardiogram generation with a bidirectional LSTM-CNN generative adversarial network. Scientific Reports. 2019;9(1)
Not in our library10.1038/s41598-019-42516-z
- 96.
Mirza M, Osindero S. Conditional generative adversarial nets. arXiv preprint arXiv:14111784. 2014;.
Not in our library
- 97.
Nikolaidis K, Kristiansen S, Goebel V, Plagemann T, Liestøl K, Kankanhalli M. Augmenting physiological time series data: A case study for sleep apnea detection. In: ECML/PKDD; 2019.
Not in our library
- 98.
Harada S, Hayashi H, Uchida S. Biosignal generation and latent variable analysis with recurrent generative adversarial networks. IEEE Access. 2019;7:144292–144302
Not in our library10.1109/access.2019.2934928
- 99.
Sheng P, Yang Z, Qian Y. GANs for children: A generative data augmentation strategy for children speech recognition. In: IEEE ASRUW; 2019. p. 129–135.
Not in our library
- 100.
Wang F, hua Zhong S, Peng J, Jiang J, Liu Y. Data augmentation for EEG-based emotion recognition with deep convolutional neural networks. In: ICMM; 2018. p. 82–93.
Not in our library
- 101.
Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, et al. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc Royal Society of London Series A: Math, Physical and Eng Sci. 1998;454(1971):903–995
Not in our library10.1098/rspa.1998.0193
- 102.
Nam GH, Bu SJ, Park NM, Seo JY, Jo HC, Jeong WT. Data augmentation using empirical mode decomposition on neural networks to classify impact noise in vehicle. In: IEEE ICASSP; 2020.
Not in our library
- 103.
Comon P. Independent component analysis, A new concept? Sig Process. 1994;36(3):287–314
Not in our library10.1016/0165-1684(94)90029-9
- 104.
Cleveland RB, Cleveland WS, McRae JE, Terpenning I. STL: A seasonal-trend decomposition. J Official Stat. 1990;6(1):3–73.
Not in our library
- 105.
Wen Q, Gao J, Song X, Sun L, Xu H, Zhu S. RobustSTL: A robust seasonal-trend decomposition algorithm for long time series. AAAI Conf Artificial Intell. 2019;33:5409–5416.
Not in our library
- 106.
He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In: IEEE ICCV; 2015.
Not in our library
- 107.
Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: ICML; 2015. p. 448–456.
Not in our library
- 108.
Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: AISTATS; 2010. p. 249–256.
Not in our library
- 109.
Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014;.
Not in our library
- 110.
Zeiler MD. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:12125701. 2012;.
Not in our library
- 111.
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation. 1997;9(8):1735–1780
Not in our library10.1162/neco.1997.9.8.1735
- 112.
Reimers N, Gurevych I. Optimal hyperparameters for deep LSTM-networks for sequence labeling tasks. arXiv preprint arXiv:170706799. 2017;.
Not in our library
- 113.
Dozat T. Incorporating Nesterov momentum into Adam. In: ICLR Workshops; 2016.
Not in our library
- 114.
Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Trans Sig Process. 1997;45(11):2673–2681
Not in our library10.1109/78.650093
- 115.
Karim F, Majumdar S, Darabi H, Chen S. LSTM fully convolutional networks for time series classification. IEEE Access. 2018;6:1662–1669
Not in our library10.1109/access.2017.2779939
- 116.
Karim F, Majumdar S, Darabi H. Insights into LSTM fully convolutional networks for time series classification. IEEE Access. 2019;7:67718–67725
Not in our library10.1109/access.2019.2916828
- 117.
Mukaka Mavuto M. A guide to appropriate use of correlation coefficient in medical research Malawi Med. J. 2012;24(3):69–71.
Not in our library
- 118.
Ortigosa-Hernandez J, Inza I, Lozano JA. Measuring the class-imbalance extent of multi-class problems. Pattern Recogn. Letters. 2017;98:32–38
Not in our library10.1016/j.patrec.2017.08.002
Reads
Read by people on this page
None yet.
Checked or read by AI through naulon
None yet.
A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.
Are you an author?
Is this your paper?
Claim it to add your note and be paid when agents cite it. Free.