Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research

Abstract

The estimation of the correct number of dimensions is a long-standing problem in psychometrics. Several methods have been proposed, such as parallel analysis (PA), Kaiser-Guttman's eigenvalue-greater-than-one rule, multiple average partial procedure (MAP), the maximum-likelihood approaches that use fit indexes as BIC and EBIC and the less used and studied approach called very simple structure (VSS). In the present paper a new approach to estimate the number of dimensions will be introduced and compared via simulation to the traditional techniques pointed above. The approach proposed in the current paper is called exploratory graph analysis (EGA), since it is based on the graphical lasso with the regularization parameter specified using EBIC. The number of dimensions is verified using the walktrap, a random walk algorithm used to identify communities in networks. In total, 32,000 data sets were simulated to fit known factor structures, with the data sets varying across different criteria: number of factors (2 and 4), number of items (5 and 10), sample size (100, 500, 1000 and 5000) and correlation between factors (orthogonal, .20, .50 and .70), resulting in 64 different conditions. For each condition, 500 data sets were simulated using lavaan. The result shows that the EGA performs comparable to parallel analysis, EBIC, eBIC and to Kaiser-Guttman rule in a number of situations, especially when the number of factors was two. However, EGA was the only technique able to correctly estimate the number of dimensions in the four-factor structure when the correlation between factors were .7, showing an accuracy of 100% for a sample size of 5,000 observations. Finally, the EGA was used to estimate the number of factors in a real dataset, in order to compare its performance with the other six techniques tested in the simulation study.

References 69

1. 1.

   Preacher KJ, MacCallum RC. Repairing Tom Swift's electric factor analysis machine. Understanding statistics: Statistical issues in psychology, education, and the social sciences. 2003 2 3;2(1):13–43.

   Not in our library

2. 2.

   Ruscio J, Roche B. Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological assessment. 2012 6;24(2):282 doi: 10.1037/a0025697

   Not in our library10.1037/a0025697

3. 3.

   Velicer WF, Jackson DN. Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure. Multivariate behavioral research. 1990 1 1;25(1):1–28. doi: 10.1207/s15327906mbr2501_1

   Not in our library10.1207/s15327906mbr2501_1

4. 4.

   James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning. New York: springer; 2013 2 11.

   Not in our library

5. 5.

   Friedman J, Hastie T, Tibshirani R. The elements of statistical learning. Springer, Berlin: Springer series in statistics; 2001.

   Not in our library

6. 6.

   Guttman L. Some necessary conditions for common-factor analysis. Psychometrika. 1954 6 1;19(2):149–61.

   Not in our library

7. 7.

   Kaiser HF. The application of electronic computers to factor analysis. Educational and psychological measurement. 1960, 20:141–151.

   Not in our library

8. 8.

   Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965 6 1;30(2):179–85.

   Not in our library10.1007/bf02289447

9. 9.

   Velicer WF. Determining the number of components from the matrix of partial correlations. Psychometrika. 1976 9 1;41(3):321–7.

   Not in our library

10. 10.

    Schwarz G. Estimating the dimension of a model. The annals of statistics. 1978;6(2):461–4.

    Not in our library

11. 11.

    Chen J, Chen Z. Extended Bayesian information criteria for model selection with large model spaces. Biometrika. 2008 9 1;95(3):759–71.

    Not in our library

12. 12.

    Revelle W, Rocklin T. Very simple structure: An alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behavioral Research. 1979 10 1;14(4):403–14. doi: 10.1207/s15327906mbr1404_2

    Not in our library10.1207/s15327906mbr1404_2

13. 13.

    Lauritzen SL. Graphical models. Clarendon Press; 1996 5 2.

    Not in our library

14. 14.

    Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics. 2008 7 1;9(3):432–41. doi: 10.1093/biostatistics/kxm045

    Not in our library10.1093/biostatistics/kxm045

15. 15.

    Foygel R, Drton M. Extended Bayesian information criteria for Gaussian graphical models. InAdvances in neural information processing systems 2010 (pp. 604–612).

    Not in our library

16. 16.

    Golino, HF. EGA package. Available at github.com/hfgolino/EGA.

    Not in our library

17. 17.

    Garrido LE, Abad FJ, Ponsoda V. Are fit indices really fit to estimate the number of factors with categorical variables? Some cautionary findings via Monte Carlo simulation. Psychological methods. 2016 3;21(1):93 doi: 10.1037/met0000064

    Not in our library10.1037/met0000064

18. 18.

    Keith TZ, Caemmerer JM, Reynolds MR. Comparison of methods for factor extraction for cognitive test-like data: Which overfactor, which underfactor?. Intelligence. 2016 2 29;54:37–54.

    Not in our library

19. 19.

    Osborne JW, Costello AB. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pan-Pacific Management Review. 2009 7 1;12(2):131–46.

    Not in our library

20. 20.

    Bandalos DL, Boehm-Kaufman MR. Four common misconceptions in exploratory factor analysis. Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences. 2009:61–87.

    Not in our library

21. 21.

    Crawford AV, Green SB, Levy R, Lo WJ, Scott L, Svetina D, et al Evaluation of parallel analysis methods for determining the number of factors. Educational and Psychological Measurement. 2010 9 1 doi: 10.1177/0013164410387338

    Not in our library

22. 22.

    Green SB, Redell N, Thompson MS, Levy R. Accuracy of revised and traditional parallel analyses for assessing dimensionality with binary data. Educational and Psychological Measurement. 2016 2 1;76(1):5–21.

    Not in our library10.1177/0013164415581898

23. 23.

    Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods. 2011 6;16(2):209 doi: 10.1037/a0023353

    Not in our library10.1037/a0023353

24. 24.

    Velicer WF, Eaton CA, Fava JL. Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. InProblems and solutions in human assessment 2000 (pp. 41–71). Springer US.

    Not in our library

25. 25.

    Zwick WR, Velicer WF. Comparison of five rules for determining the number of components to retain. Psychological bulletin. 1986 5;99(3):432.

    Not in our library

26. 26.

    Garrido LE, Abad FJ, Ponsoda V. Performance of Velicer's minimum average partial factor retention method with categorical variables. Educational and Psychological Measurement. 2011 4 5:0013164410389489.

    Not in our library

27. 27.

    Dziak JJ, Coffman DL, Lanza ST, Li R. Sensitivity and specificity of information criteria. The Methodology Center and Department of Statistics, Penn State, The Pennsylvania State University. 2012 Jun 27:1–0.

    Not in our library

28. 28.

    Lopes HF, West M. Bayesian model assessment in factor analysis. Statistica Sinica. 2004 1 1:41–67.

    Not in our library

29. 29.

    Preacher KJ, Zhang G, Kim C, Mels G. Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research. 2013 1 1;48(1):28–56. doi: 10.1080/00273171.2012.710386

    Not in our library10.1080/00273171.2012.710386

30. 30.

    Song J, Belin TR. Choosing an appropriate number of factors in factor analysis with incomplete data. Computational Statistics & Data Analysis. 2008 3 15;52(7):3560–9.

    Not in our library

31. 31.

    Hakstian AR, Rogers WT, Cattell RB. The behavior of number-of-factors rules with simulated data. Multivariate Behavioral Research. 1982 4 1;17(2):193–219. doi: 10.1207/s15327906mbr1702_3

    Not in our library10.1207/s15327906mbr1702_3

32. 32.

    Buja A, Eyuboglu N. Remarks on parallel analysis. Multivariate behavioral research. 1992 10 1;27(4):509–40. doi: 10.1207/s15327906mbr2704_2

    Not in our library10.1207/s15327906mbr2704_2

33. 33.

    Horn JL, Engstrom R. Cattell's scree test in relation to Bartlett's chi-square test and other observations on the number of factors problem. Multivariate Behavioral Research. 1979 7 1;14(3):283–300. doi: 10.1207/s15327906mbr1403_1

    Not in our library10.1207/s15327906mbr1403_1

34. 34.
    The Small World of Psychopathology · 2011

    No notice
35. 35.

    Borsboom D, Cramer AO. Network analysis: an integrative approach to the structure of psychopathology. Annual review of clinical psychology. 2013 3 28;9:91–121. doi: 10.1146/annurev-clinpsy-050212-185608

    Not in our library10.1146/annurev-clinpsy-050212-185608

36. 36.

    Isvoranu AM, van Borkulo CD, Boyette LL, Wigman JT, Vinkers CH, Borsboom D. A Network Approach to Psychosis: Pathways Between Childhood Trauma and Psychotic Symptoms. Schizophrenia bulletin. 2016 5 10:sbw055.

    Not in our library10.1093/schbul/sbw055

37. 37.

    McNally RJ, Robinaugh DJ, Wu GW, Wang L, Deserno MK, Borsboom D. Mental disorders as causal systems a network approach to posttraumatic stress disorder. Clinical Psychological Science. 2015 11 1;3(6):836–49.

    Not in our library

38. 38.

    Fried EI, Bockting C, Arjadi R, Borsboom D, Amshoff M, Cramer AO, et al From loss to loneliness: The relationship between bereavement and depressive symptoms. Journal of abnormal psychology. 2015 5;124(2):256 doi: 10.1037/abn0000028

    Not in our library10.1037/abn0000028

39. 39.

    Dalege J, Borsboom D, van Harreveld F, van den Berg H, Conner M, van der Maas HL. Toward a formalized account of attitudes: The Causal Attitude Network (CAN) model. Psychological review. 2016 1;123(1):2 doi: 10.1037/a0039802

    Not in our library10.1037/a0039802

40. 40.

    Kossakowski JJ, Epskamp S, Kieffer JM, van Borkulo CD, Rhemtulla M, Borsboom D. The application of a network approach to Health-Related Quality of Life (HRQoL): introducing a new method for assessing HRQoL in healthy adults and cancer patients. Quality of Life Research. 2016 4 1;25(4):781–92. doi: 10.1007/s11136-015-1127-z

    Not in our library10.1007/s11136-015-1127-z

41. 41.

    Van Der Maas HL, Dolan CV, Grasman RP, Wicherts JM, Huizenga HM, Raijmakers ME. A dynamical model of general intelligence: the positive manifold of intelligence by mutualism. Psychological review. 2006 10;113(4):842 doi: 10.1037/0033-295X.113.4.842

    Not in our library10.1037/0033-295x.113.4.842

42. 42.

    Epskamp S, Maris G, Waldorp LJ, Borsboom D. Network psychometrics. Handbook of psychometrics. New York: Wiley 2015.

    Not in our library

43. 43.

    Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological). 1996 1 1:267–88.

    Not in our library

44. 44.

    van Borkulo CD, Borsboom D, Epskamp S, Blanken TF, Boschloo L, Schoevers RA, et al A new method for constructing networks from binary data. Scientific Reports. 2014 8 1;4.

    Not in our library10.1038/srep05918

45. 45.

    Epskamp S, Cramer AO, Waldorp LJ, Schmittmann VD, Borsboom D. qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software. 2012 5 24;48(4):1–8.

    Not in our library

46. 46.

    Epskamp S, Fried EI. Estimating Regularized Psychological Networks Using qgraph. arXiv preprint arXiv:1607.01367. 2016 Sep 25.

    Not in our library

47. 47.

    Cramer AO, Sluis S, Noordhof A, Wichers M, Geschwind N, Aggen SH, et al Dimensions of normal personality as networks in search of equilibrium: You can't like parties if you don't like people. European Journal of Personality. 2012 7 1;26(4):414–31.

    Not in our library

48. 48.

    Costantini G, Epskamp S, Borsboom D, Perugini M, Mõttus R, Waldorp LJ, et al State of the aRt personality research: A tutorial on network analysis of personality data in R. Journal of Research in Personality. 2015 2 28;54:13–29.

    Not in our library

49. 49.

    Marsman M, Maris G, Bechger T, Glas C. Bayesian inference for low-rank Ising networks. Scientific reports. 2015;5.

    Not in our library10.1038/srep09050

50. 50.

    Chandrasekaran V, Parrilo PA, Willsky AS. Latent variable graphical model selection via convex optimization. InCommunication, Control, and Computing (Allerton), 2010 48th Annual Allerton Conference on 2010 Sep 29 (pp. 1610–1613). IEEE.

    Not in our library

51. 51.

    Epskamp S, Rhemtulla M, Borsboom D. Generalized Network Psychometrics: Combining Network and Latent Variable Models. arXiv preprint arXiv:1605.09288. 2016 May 30.

    Not in our library10.1007/s11336-017-9557-x

52. 52.

    Pons P, Latapy M. Computing communities in large networks using random walks. InInternational Symposium on Computer and Information Sciences 2005 Oct 26 (pp. 284–293). Springer Berlin Heidelberg.

    Not in our library

53. 53.

    R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2016. URL http://www.R-project.org/.

    Not in our library

54. 54.

    Rosseel Y. lavaan: An R package for structural equation modeling. Journal of Statistical Software. 2012 5 24;48(2):1–36.

    Not in our library

55. 55.

    Raven JC. Progressive matrices: A perceptual test of intelligence. London: HK Lewis 1938.

    Not in our library

56. 56.

    Formann AK, Waldherr K, Piswanger K. Wiener Matrizen-Test 2 (WMT-2): Ein Rasch-skalierter sprachfreier Kurztest zur Erfassung der Intelligenz. Beltz Test; 2011.

    Not in our library

57. 57.

    Condon DM, Revelle W. The International Cognitive Ability Resource: Development and initial validation of a public-domain measure. Intelligence. 2014 4 30;43:52–64.

    Not in our library

58. 58.

    Condon DM, Revelle W. Selected ICAR data from the SAPA-Project: Development and initial validation of a public-domain measure. Journal of Open Psychology Data. 2016 1 26;4(1).

    Not in our library

59. 59.

    Revelle W. psych: Procedures for personality and psychological research. Northwestern University, Evanston. R package version. 2014 Jan;1(1).

    Not in our library

60. 60.

    Csardi G, Nepusz T. The igraph software package for complex network research. InterJournal, Complex Systems. 2006 1 11;1695(5):1–9.

    Not in our library

61. 61.

    Cohen J. Statistical power analysis for the behavior science Lawrance Eribaum Association 1988.

    Not in our library

62. 62.

    Golino, HF, & Gomes, CM. Investigando estágios de desenvolvimento do raciocínio indutivo usando a análise fatorial confimatória, o Modelo Logístico Simples de Rasch e o modelo de teste logístico linear (Rasch Estendido). In H. F. Golino, C. M. Gomes, A. Amantes, & G. Coelho. Psicometria Contemporânea: Compreendendo os Modelos Rasch. São Paulo: Casa do Psicólogo/Pearson, 2015. 283–331.

    Not in our library

63. 63.

    Commons ML, Richards FA. Applying the general stage model. Beyond formal operations. 1984;1:141–57.

    Not in our library

64. 64.

    Commons ML, Pekker A. Presenting the formal theory of hierarchical complexity. World Futures. 2008 10 15;64(5–7):375–82.

    Not in our library

65. 65.

    Fischer KW. A theory of cognitive development: The control and construction of hierarchies of skills. Psychological review. 1980 11;87(6):477.

    Not in our library

66. 66.

    Fischer K, Yan Z. The development of dynamic skill theory. Conceptions of development: Lessons from the laboratory. 2002:279–312.

    Not in our library

67. 67.

    Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001 12 1;66(4):507–14.

    Not in our library10.1007/s11336-009-9135-y

68. 68.

    Epskamp S. semPlot: Path diagrams and visual analysis of various SEM packages' output (Version 1.0. 0.). http://CRAN.R-project.org/package=semPlot. 2014.

    Not in our library

69. 69.
    A Comparative Analysis of Community Detection Algorithms on Artificial Networks · 2016

    Corrected

Cited by papers we hold 6

• Psychometric validation of a brief self-report measure of misophonia symptoms and functional impairment: The duke-vanderbilt misophonia screening questionnaire · 2022
  Not checked
• Análisis de Redes: una Alternativa a los Enfoques Clásicos de Evaluación de los Sistemas Educativos · 2022
  Not checked
• Psychological Differences Among Healthcare Workers of a Rehabilitation Institute During the COVID-19 Pandemic: A Two-Step Study · 2021
  Not checked
• Psychometric Network Models from Time-Series and Panel Data · 2020
  No notice
• Searching for G: A New Evaluation of SPM-LS Dimensionality · 2019
  Not checked
• Networks of Depression and Anxiety Symptoms Across Development · 2018
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.