Estimation for Better Inference in Neuroscience

Abstract

The estimation approach to inference emphasizes reporting effect sizes with expressions of uncertainty (interval estimates). In this perspective we explain the estimation approach and describe how it can help nudge neuroscientists toward a more productive research cycle by fostering better planning, more thoughtful interpretation, and more balanced evaluation of evidence.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 91

1. 1.

   American Psychological Association (2010 ) Publication manual of the American Psychological Association. Washington, DC: American Psychological Association.

   Not in our library

2. 2.

   Anscombe FJ (1954 ) Fixed-sample-size analysis of sequential observations. Biometrics 10:89

   Not in our library10.2307/3001665

3. 3.

   ASA (2016 ) Ethical guidelines for statistical practice: American Statistical Association. Available at http://www.amstat.org/asa/files/pdfs/EthicalGuidelines.pdf.

   Not in our library

4. 4.
   Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies · 2014

   No notice
5. 5.

   Bernard C (2016 ) Editorial: scientific rigor or rigor mortis? eNeuro 3:ENEURO.0176-16.2016.

   Not in our library10.1523/eneuro.0176-16.2016

6. 6.

   Bliss TV, Lomo T (1973 ) Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J Physiol 232:331–356.

   Not in our library10.1113/jphysiol.1973.sp010273

7. 7.

   Boekel W, Wagenmakers EJ, Belay L, Verhagen J, Brown S, Forstmann BU (2015 ) A purely confirmatory replication study of structural brain-behavior correlations. Cortex 66:115–133.

   Not in our library10.1016/j.cortex.2014.11.019

8. 8.

   Borota D, Murray E, Keceli G, Chang A, Watabe JM, Ly M, Toscano JP, Yassa MA ( 2014 ) Post-study caffeine administration enhances memory consolidation in humans. Nat Neurosci 17:201–203.

   Not in our library10.1038/nn.3623

9. 9.

   Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, Munafò MR (2013 ) Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14:365–376.

   Not in our library10.1038/nrn3475

10. 10.

    Calin-Jageman RJ (2017 ) After p values: the new statistics for undergraduate neuroscience education. J Undergrad Neurosci Educ 16:E1–E4.

    Not in our library

11. 11.

    Calin-Jageman RJ, Cumming G (2019 ) The new statistics for better science: ask how much, how uncertain, and what else is known. Am Stat 73:271–280.

    Not in our library10.1080/00031305.2018.1518266

12. 12.

    Coan JA, Schaefer HS, Davidson RJ (2006 ) Lending a hand of the neural response to threat. Psychol Sci 17:1032–1039.

    Not in our library10.1111/j.1467-9280.2006.01832.x

13. 13.

    Conlisk J (2011 ) Professor Zak’s empirical studies on trust and oxytocin. J Econ Behav Organ 78:160–166.

    Not in our library10.1016/j.jebo.2011.01.002

14. 14.

    Cumming G (2008 ) Replication and p intervals. Perspect Psychol Sci 3:286–300.

    Not in our library10.1111/j.1745-6924.2008.00079.x

15. 15.

    Cumming G (2012 ) Understanding the new statistics: effect sizes, confidence intervals, and meta-analysis. New York: Routledge.

    Not in our library

16. 16.

    Cumming G (2014 ) The new statistics: why and how. Psychol Sci 25:7–29.

    Not in our library10.1177/0956797613504966

17. 17.

    Cumming G, Calin-Jageman RJ (2017 ) Introduction to the new statistics: estimation, open science, and beyond. New York: Routledge.

    Not in our library

18. 18.

    Cumming G, Finch SU (2001 ) Four reasons to use CIs. Educ Psychol Meas 61:532–574.

    Not in our library

19. 19.

    Cumming G, Finch S (2005 ) Inference by eye: confidence intervals and how to read pictures of data. Am Psychol 60:170–180.

    Not in our library10.1037/0003-066x.60.2.170

20. 20.

    Cumming G, Maillardet R (2006 ) Confidence intervals and replication: where will the next mean fall? Psychol Methods 11:217–227.

    Not in our library10.1037/1082-989x.11.3.217

21. 21.
    Low statistical power in biomedical science: a review of three human research domains · 2017

    No notice
22. 22.

    Ellis PD (2010 ) The essential guide to effect sizes. New York: Cambridge UP.

    Not in our library

23. 23.

    Fanelli D (2012 ) Negative results are disappearing from most disciplines and countries. Scientometrics 90:891–904.

    Not in our library10.1007/s11192-011-0494-7

24. 24.

    Ferguson CJ, Heene M (2012 ) A vast graveyard of undead theories: publication bias and psychological science’s aversion to the null. Perspect Psychol Sci 7:555–561.

    Not in our library10.1177/1745691612459059

25. 25.

    Fidler F (2010 ) The American Psychological Association publication manual sixth edition: implications for statistics education. Proceedings of the Eighth International Conference on Teaching Statistics (ICOTS8, July, 2010), Ljubljana, Slovenia. Voorburg, The Netherlands: International Statistical Institute.

    Not in our library

26. 26.

    Fidler F, Thomason N, Cumming G, Finch S, Leeman J (2004 ) Editors can lead researchers to confidence intervals, but can’t make them think: statistical reform lessons from medicine. Psychol Sci 15:119–126.

    Not in our library10.1111/j.0963-7214.2004.01502008.x

27. 27.

    Fisher RA (1926 ) The arrangement of field experiments. J Minist Agric Gt Britain 33:503–513.

    Not in our library

28. 28.

    Fritz A, Scherndl T, Kühberger A (2013 ) A comprehensive review of reporting practices in psychological journals: are effect sizes really enough? Theory Psychol 23:98–122.

    Not in our library10.1177/0959354312436870

29. 29.

    Gelman A, Stern H (2006 ) The difference between “significant” and “not significant” is not itself statistically significant. Am Stat 60:328–331.

    Not in our library10.1198/000313006x152649

30. 30.

    Gigerenzer G (2018 ) Statistical rituals: the replication delusion and how we got there. Adv Methods Pract Psychol Sci 1:198–218.

    Not in our library10.1177/2515245918771329

31. 31.

    Giofrè D, Cumming G, Fresc L, Boedker I, Tressoldi P (2017 ) The influence of journal submission guidelines on authors’ reporting of statistics and use of open research practices. PLoS One 12:e0175583.

    Not in our library10.1371/journal.pone.0175583

32. 32.

    Goodhill GJ (2017 ) Is neuroscience facing up to statistical power? arXiv:1701.01219. Available at http://arxiv.org/abs/1701.01219.

    Not in our library

33. 33.

    Goodman SN (1994 ) The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 121:200.

    Not in our library10.7326/0003-4819-121-3-199408010-00008

34. 34.

    Halsey LG, Curran-Everett D, Vowler SL, Drummond GB (2015 ) The fickle p value generates irreproducible results. Nat Methods 12:179–185.

    Not in our library10.1038/nmeth.3288

35. 35.

    Ho J, Tumkaya T, Aryal S, Choi H, Claridge-Chang A (2018 ) Moving beyond p values: everyday data analysis with estimation plots. bioRxiv. Advance online publication. Retrieved July 26, 2018. doi: https://doi.org/10.1101/377978.

    Not in our library

36. 36.

    Hodgkin AL, Huxley AF (1952 ) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117:500–544.

    Not in our library10.1113/jphysiol.1952.sp004764

37. 37.

    Hogben LT (1957 ) Statistical theory: the relationship of probability, credibility, and error. New York: Norton.

    Not in our library

38. 38.

    Horvath JC, Forte JD, Carter O (2015 ) Quantitative review finds no evidence of cognitive effects in healthy populations from single-session transcranial direct current stimulation (tDCS). Brain Stimul 8:535–550.

    Not in our library10.1016/j.brs.2015.01.400

39. 39.

    International Committee of Medical Journal Editors (1997 ) Uniform requirements for manuscripts submitted to biomedical journals. N Engl J Med 336:309–316.

    Not in our library10.1056/nejm199701233360422

40. 40.

    Katz B, Miledi R (1968 ) The role of calcium in neuromuscular facilitation. J Physiol 195:481–492.

    Not in our library10.1113/jphysiol.1968.sp008469

41. 41.

    Kelley K (2007 ) Methods for the behavioral, educational, and social sciences: an R package. Behav Res Methods 39:979–984.

    Not in our library10.3758/bf03192993

42. 42.

    Kelley K, Darku FB, Chattopadhyay B (2018 ) Accuracy in parameter estimation for a general class of effect sizes: a sequential approach. Psychol Methods 23:226–243.

    Not in our library10.1037/met0000127

43. 43.

    Kelley K, Maxwell SE, Rausch JR (2003 ) Obtaining power or obtaining precision. Eval Health Prof 26:258–287.

    Not in our library10.1177/0163278703255242

44. 44.

    Kline RB (2004 ) Beyond significance testing. Washington, DC: American Psychological Association.

    Not in our library

45. 45.

    Kosfeld M, Heinrichs M, Zak PJ, Fischbacher U, Fehr E (2005 ) Oxytocin increases trust in humans. Nature 435:673–676.

    Not in our library10.1038/nature03701

46. 46.

    Kraemer HC (1991 ) To increase power in randomized clinical trials without increasing sample size. Psychopharmacol Bull 27:217–224.

    Not in our library

47. 47.

    Kruschke JK (2014 ) Doing Bayesian data analysis: a tutorial with R, JAGS, and Stan, Ed 2. Amsterdam; London: Elsevier.

    Not in our library

48. 48.

    Kruschke JK (2013 ) Bayesian estimation supersedes the t test. J Exp Psychol Gen 142:573–603.

    Not in our library10.1037/a0029146

49. 49.

    Kruschke JK, Liddell TM (2018 ) The Bayesian new statistics: hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychon Bull Rev 25:178–206.

    Not in our library10.3758/s13423-016-1221-4

50. 50.

    Lai J, Fidler F, Cumming G (2012 ) Subjective p intervals researchers underestimate the variability of p values over replication. Methodology 8:51–62.

    Not in our library10.1027/1614-2241/a000037

51. 51.
    Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs · 2013

    No notice
52. 52.

    Lakens D, Evers ERK (2014 ) Sailing from the seas of chaos into the corridor of stability: practical recommendations to increase the informational value of studies. Perspect Psychol Sci 9:278–292.

    Not in our library10.1177/1745691614528520

53. 53.

    Lakens D, Scheel AM, Isager PM (2018 ) Equivalence testing for psychological research: a tutorial. Adv Methods Pract Psychol Sci 1:259–269.

    Not in our library10.1177/2515245918770963

54. 54.

    Lane A, Luminet O, Nave G, Mikolajczak M (2016 ) Is there a publication bias in behavioural intranasal oxytocin research on humans? Opening the file drawer of one laboratory. J Neuroendocrinol 28.

    Not in our library10.1111/jne.12384

55. 55.

    Leng G, Ludwig M (2016 ) Intranasal oxytocin: myths and delusions. Biol Psychiatry 79:243–250.

    Not in our library10.1016/j.biopsych.2015.05.003

56. 56.

    Liu Y, Li S, Lin W, Li W, Yan X, Wang X, Pan X, Rutledge RB, Ma Y (2019 ) Oxytocin modulates social value representations in the amygdala. Nat Neurosci 22:633–641.

    Not in our library10.1038/s41593-019-0351-1

57. 57.

    MacKinnon S (2013 ) Increasing statistical power in psychological research without increasing sample size. Open Science Collaboration. Available at: http://osc.centerforopenscience.org/2013/11/03/Increasing-statistical-power/.

    Not in our library

58. 58.

    Marsman M, Wagenmakers EJ (2017 ) Bayesian benefits with JASP. Eur J Dev Psychol 14:545–555.

    Not in our library10.1080/17405629.2016.1259614

59. 59.

    Maxwell SE, Kelley K, Rausch JR (2008 ) Sample size planning for statistical power and accuracy in parameter estimation. Annu Rev Psychol 59:537–563.

    Not in our library10.1146/annurev.psych.59.103006.093735

60. 60.

    McShane BB, Gal D, Gelman A, Robert C, Tackett JL (2019 ) Abandon statistical significance. Am Stat 73:235–245.

    Not in our library10.1080/00031305.2018.1527253

61. 61.

    Medina J, Cason S (2017 ) No evidential value in samples of transcranial direct current stimulation (tDCS) studies of cognition and working memory in healthy populations. Cortex 94:131–141.

    Not in our library10.1016/j.cortex.2017.06.021

62. 62.

    Meyvis T, Van Osselaer SMJ (2018 ) Increasing the power of your study by increasing the effect size. J Consum Res. 1173 44:1157.

    Not in our library

63. 63.
    The Importance of Sample Size for Reproducibility of tDCS Effects · 2016

    No notice
64. 64.

    Mole (2018 ) Mea culpa, mea powerful culpa. J Cell Sci 131:jcs215707.

    Not in our library

65. 65.

    Mole (2017 ) Peerless I. J Cell Sci 130:2079–2080.

    Not in our library

66. 66.

    Nave G, Camerer C, McCullough M (2015 ) Does oxytocin increase trust in humans? A critical review of research. Perspect Psychol Sci 10:772–789.

    Not in our library10.1177/1745691615600138

67. 67.

    Nieuwenhuis S, Forstmann BU, Wagenmakers EJ (2011 ) Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 14:1105–1107.

    Not in our library10.1038/nn.2886

68. 68.

    O’Boyle EH, Banks GC, Gonzalez-Mulé E (2017 ) The chrysalis effect. J Manage 43:376–399.

    Not in our library10.1177/0149206314527133

69. 69.

    Olds J, Milner P (1954 ) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427.

    Not in our library10.1037/h0058775

70. 70.

    Pek J, Flora DB (2018 ) Reporting effect sizes in original psychological research: a discussion and tutorial. Psychol Methods 23:208–225.

    Not in our library10.1037/met0000126

71. 71.

    Peters GJ, Crutzen R (2017 ) Knowing exactly how effective an intervention, treatment, or manipulation is and ensuring that a study replicates: accuracy in parameter estimation as a partial solution to the replication crisis. PsyArxiv. Advance online publication. Retrieved September 7, 2018. doi: 10.31234/osf.io/cjsk2.

    Not in our library

72. 72.

    Rothman KJ, Greenland S (2018 ) Planning study size based on precision rather than power. Epidemiology 29:599–603.

    Not in our library10.1097/ede.0000000000000876

73. 73.

    Scoville WB, Milner B (1957 ) Loss of recent memory after bilateral hippocampal lesions. J Neurol Neurosurg Psychiatry 20:11–21.

    Not in our library10.1136/jnnp.20.1.11

74. 74.

    Sherrington R, Rogaev EI, Liang Y, Rogaeva EA, Levesque G, Ikeda M, Chi H, Lin C, Li G, Holman K, Tsuda T, Mar L, Foncin JF, Bruni AC, Montesi MP, Sorbi S, Rainero I, Pinessi L, Nee L, Chumakov I, Pollen D, et al (2003 ) Cloning of a gene bearing missense mutations in early-onset familial Alzheimer's disease. Nature 375:754–760.

    Not in our library10.1038/375754a0

75. 75.

    Shuirmann DJ (1987 ) A comparison of the two one-sided tests procedure and the power. J Pharmacokinet Biopharm 15:657–680.

    Not in our library10.1007/bf01068419

76. 76.

    Simmons JP, Nelson LD, Simonsohn U (2011 ) False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22:1359–66.

    Not in our library10.1177/0956797611417632

77. 77.

    Smithson M (2002 ) Confidence intervals. Thousand Oaks, CA: SAGE.

    Not in our library

78. 78.

    Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, Hing C, Kwok CS, Pang C, Harvey I (2010 ) Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 14:1–220.

    Not in our library10.3310/hta14080

79. 79.

    Sterling TD (1959 ) Publication decisions and their possible effects on inferences drawn from tests of significance: or vice versa. J Am Stat Assoc 54:30–34.

    Not in our library10.2307/2282137

80. 80.

    Sterling TD, Rosenbaum WL, Weinkam JJ (1995 ) Publication decisions revisited: the effect of the outcome of statistical tests on the decision to publish and vice versa. Am Stat 49:108

    Not in our library10.2307/2684823

81. 81.
    When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment · 2017

    No notice
82. 82.
    Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature · 2017

    Corrected
83. 83.
    The pervasive avoidance of Prospective Statistical Power: major consequences and practical solutions · 2015

    Not checked
84. 84.

    Tressoldi PE, Giofré D, Sella F, Cumming G (2013 ) High impact = high statistical standards? Not necessarily so. PLoS One 8:e56180.

    Not in our library10.1371/journal.pone.0056180

85. 85.

    Vankov I, Bowers J, Munafò MR (2014 ) On the persistence of low power in psychological science. Q J Exp Psychol 67:1037–1040.

    Not in our library10.1080/17470218.2014.885986

86. 86.
    Bayesian inference for psychology. Part II: Example applications with JASP · 2017

    No notice
87. 87.

    Wagenmakers EJ, Gronau QF, Dablender F, Etz A (2018b ) The support interval. PsyArXiv. Advance online publication. Retrieved December 22, 2018. doi: 10.31234/osf.io/zwnxb.

    Not in our library

88. 88.

    Walum H, Waldman ID, Young LJ (2016 ) Statistical and methodological considerations for the interpretation of intranasal oxytocin studies. Biol Psychiatry 79:251–257.

    Not in our library10.1016/j.biopsych.2015.06.016

89. 89.

    Wasserstein RL, Schirm AL, Lazar NA (2019 ) Moving to a world beyond “p < 0.05”. Am Stat 73:1–19.

    Not in our library10.1080/00031305.2019.1583913

90. 90.

    Westlake WJ (1972 ) Use of confidence intervals in analysis of comparative bioavailability trials. J Pharm Sci 61:1340–1341.

    Not in our library10.1002/jps.2600610845

91. 91.

    Zak P, Kurzban R, Matzner W (2005 ) Oxytocin is associated with human trustworthiness. Horm Behav 48:522–527.

    Not in our library10.1016/j.yhbeh.2005.07.009

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.