Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research

Abstract

The estimation of the correct number of dimensions is a long-standing problem in psychometrics. Several methods have been proposed, such as parallel analysis (PA), Kaiser-Guttman's eigenvalue-greater-than-one rule, multiple average partial procedure (MAP), the maximum-likelihood approaches that use fit indexes as BIC and EBIC and the less used and studied approach called very simple structure (VSS). In the present paper a new approach to estimate the number of dimensions will be introduced and compared via simulation to the traditional techniques pointed above. The approach proposed in the current paper is called exploratory graph analysis (EGA), since it is based on the graphical lasso with the regularization parameter specified using EBIC. The number of dimensions is verified using the walktrap, a random walk algorithm used to identify communities in networks. In total, 32,000 data sets were simulated to fit known factor structures, with the data sets varying across different criteria: number of factors (2 and 4), number of items (5 and 10), sample size (100, 500, 1000 and 5000) and correlation between factors (orthogonal, .20, .50 and .70), resulting in 64 different conditions. For each condition, 500 data sets were simulated using lavaan. The result shows that the EGA performs comparable to parallel analysis, EBIC, eBIC and to Kaiser-Guttman rule in a number of situations, especially when the number of factors was two. However, EGA was the only technique able to correctly estimate the number of dimensions in the four-factor structure when the correlation between factors were .7, showing an accuracy of 100% for a sample size of 5,000 observations. Finally, the EGA was used to estimate the number of factors in a real dataset, in order to compare its performance with the other six techniques tested in the simulation study.

References 69

1. 1.

   Preacher KJ, MacCallum RC. Repairing Tom Swift's electric factor analysis machine. Understanding statistics: Statistical issues in psychology, education, and the social sciences. 2003 2 3;2(1):13–43.

   Not in our library

2. 2.

   Ruscio J, Roche B. Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological assessment. 2012 6;24(2):282 doi: 10.1037/a0025697

   Not in our library10.1037/a0025697 (opens in a new tab)

3. 3.

   Velicer WF, Jackson DN. Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure. Multivariate behavioral research. 1990 1 1;25(1):1–28. doi: 10.1207/s15327906mbr2501_1

   Not in our library10.1207/s15327906mbr2501_1 (opens in a new tab)

4. 4.

   James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning. New York: springer; 2013 2 11.

   Not in our library

5. 5.

   Friedman J, Hastie T, Tibshirani R. The elements of statistical learning. Springer, Berlin: Springer series in statistics; 2001.

   Not in our library

6. 6.

   Guttman L. Some necessary conditions for common-factor analysis. Psychometrika. 1954 6 1;19(2):149–61.

   Not in our library

7. 7.

   Kaiser HF. The application of electronic computers to factor analysis. Educational and psychological measurement. 1960, 20:141–151.

   Not in our library

8. 8.

   Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965 6 1;30(2):179–85.

   Not in our library10.1007/bf02289447 (opens in a new tab)

9. 9.

   Velicer WF. Determining the number of components from the matrix of partial correlations. Psychometrika. 1976 9 1;41(3):321–7.

   Not in our library

10. 10.

    Schwarz G. Estimating the dimension of a model. The annals of statistics. 1978;6(2):461–4.

    Not in our library

11. 11.

    Chen J, Chen Z. Extended Bayesian information criteria for model selection with large model spaces. Biometrika. 2008 9 1;95(3):759–71.

    Not in our library

12. 12.

    Revelle W, Rocklin T. Very simple structure: An alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behavioral Research. 1979 10 1;14(4):403–14. doi: 10.1207/s15327906mbr1404_2

    Not in our library10.1207/s15327906mbr1404_2 (opens in a new tab)

13. 13.

    Lauritzen SL. Graphical models. Clarendon Press; 1996 5 2.

    Not in our library

14. 14.

    Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics. 2008 7 1;9(3):432–41. doi: 10.1093/biostatistics/kxm045

    Not in our library10.1093/biostatistics/kxm045 (opens in a new tab)

15. 15.

    Foygel R, Drton M. Extended Bayesian information criteria for Gaussian graphical models. InAdvances in neural information processing systems 2010 (pp. 604–612).

    Not in our library

16. 16.

    Golino, HF. EGA package. Available at github.com/hfgolino/EGA.

    Not in our library

17. 17.

    Garrido LE, Abad FJ, Ponsoda V. Are fit indices really fit to estimate the number of factors with categorical variables? Some cautionary findings via Monte Carlo simulation. Psychological methods. 2016 3;21(1):93 doi: 10.1037/met0000064

    Not in our library10.1037/met0000064 (opens in a new tab)

18. 18.

    Keith TZ, Caemmerer JM, Reynolds MR. Comparison of methods for factor extraction for cognitive test-like data: Which overfactor, which underfactor?. Intelligence. 2016 2 29;54:37–54.

    Not in our library

19. 19.

    Osborne JW, Costello AB. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pan-Pacific Management Review. 2009 7 1;12(2):131–46.

    Not in our library

20. 20.

    Bandalos DL, Boehm-Kaufman MR. Four common misconceptions in exploratory factor analysis. Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences. 2009:61–87.

    Not in our library

21. 21.

    Crawford AV, Green SB, Levy R, Lo WJ, Scott L, Svetina D, et al Evaluation of parallel analysis methods for determining the number of factors. Educational and Psychological Measurement. 2010 9 1 doi: 10.1177/0013164410387338

    Not in our library

22. 22.

    Green SB, Redell N, Thompson MS, Levy R. Accuracy of revised and traditional parallel analyses for assessing dimensionality with binary data. Educational and Psychological Measurement. 2016 2 1;76(1):5–21.

    Not in our library10.1177/0013164415581898 (opens in a new tab)

23. 23.

    Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods. 2011 6;16(2):209 doi: 10.1037/a0023353

    Not in our library10.1037/a0023353 (opens in a new tab)

24. 24.

    Velicer WF, Eaton CA, Fava JL. Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. InProblems and solutions in human assessment 2000 (pp. 41–71). Springer US.

    Not in our library

25. 25.

    Zwick WR, Velicer WF. Comparison of five rules for determining the number of components to retain. Psychological bulletin. 1986 5;99(3):432.

    Not in our library

26. 26.

    Garrido LE, Abad FJ, Ponsoda V. Performance of Velicer's minimum average partial factor retention method with categorical variables. Educational and Psychological Measurement. 2011 4 5:0013164410389489.

    Not in our library

27. 27.

    Dziak JJ, Coffman DL, Lanza ST, Li R. Sensitivity and specificity of information criteria. The Methodology Center and Department of Statistics, Penn State, The Pennsylvania State University. 2012 Jun 27:1–0.

    Not in our library

28. 28.

    Lopes HF, West M. Bayesian model assessment in factor analysis. Statistica Sinica. 2004 1 1:41–67.

    Not in our library

29. 29.

    Preacher KJ, Zhang G, Kim C, Mels G. Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research. 2013 1 1;48(1):28–56. doi: 10.1080/00273171.2012.710386

    Not in our library10.1080/00273171.2012.710386 (opens in a new tab)

30. 30.

    Song J, Belin TR. Choosing an appropriate number of factors in factor analysis with incomplete data. Computational Statistics & Data Analysis. 2008 3 15;52(7):3560–9.

    Not in our library

31. 31.

    Hakstian AR, Rogers WT, Cattell RB. The behavior of number-of-factors rules with simulated data. Multivariate Behavioral Research. 1982 4 1;17(2):193–219. doi: 10.1207/s15327906mbr1702_3

    Not in our library10.1207/s15327906mbr1702_3 (opens in a new tab)

32. 32.

    Buja A, Eyuboglu N. Remarks on parallel analysis. Multivariate behavioral research. 1992 10 1;27(4):509–40. doi: 10.1207/s15327906mbr2704_2

    Not in our library10.1207/s15327906mbr2704_2 (opens in a new tab)

33. 33.

    Horn JL, Engstrom R. Cattell's scree test in relation to Bartlett's chi-square test and other observations on the number of factors problem. Multivariate Behavioral Research. 1979 7 1;14(3):283–300. doi: 10.1207/s15327906mbr1403_1

    Not in our library10.1207/s15327906mbr1403_1 (opens in a new tab)

34. 34.
    The Small World of Psychopathology · 2011

    No notice
35. 35.

    Borsboom D, Cramer AO. Network analysis: an integrative approach to the structure of psychopathology. Annual review of clinical psychology. 2013 3 28;9:91–121. doi: 10.1146/annurev-clinpsy-050212-185608

    Not in our library10.1146/annurev-clinpsy-050212-185608 (opens in a new tab)

36. 36.

    Isvoranu AM, van Borkulo CD, Boyette LL, Wigman JT, Vinkers CH, Borsboom D. A Network Approach to Psychosis: Pathways Between Childhood Trauma and Psychotic Symptoms. Schizophrenia bulletin. 2016 5 10:sbw055.

    Not in our library10.1093/schbul/sbw055 (opens in a new tab)

37. 37.

    McNally RJ, Robinaugh DJ, Wu GW, Wang L, Deserno MK, Borsboom D. Mental disorders as causal systems a network approach to posttraumatic stress disorder. Clinical Psychological Science. 2015 11 1;3(6):836–49.

    Not in our library

38. 38.

    Fried EI, Bockting C, Arjadi R, Borsboom D, Amshoff M, Cramer AO, et al From loss to loneliness: The relationship between bereavement and depressive symptoms. Journal of abnormal psychology. 2015 5;124(2):256 doi: 10.1037/abn0000028

    Not in our library10.1037/abn0000028 (opens in a new tab)

39. 39.

    Dalege J, Borsboom D, van Harreveld F, van den Berg H, Conner M, van der Maas HL. Toward a formalized account of attitudes: The Causal Attitude Network (CAN) model. Psychological review. 2016 1;123(1):2 doi: 10.1037/a0039802

    Not in our library10.1037/a0039802 (opens in a new tab)

40. 40.

    Kossakowski JJ, Epskamp S, Kieffer JM, van Borkulo CD, Rhemtulla M, Borsboom D. The application of a network approach to Health-Related Quality of Life (HRQoL): introducing a new method for assessing HRQoL in healthy adults and cancer patients. Quality of Life Research. 2016 4 1;25(4):781–92. doi: 10.1007/s11136-015-1127-z

    Not in our library10.1007/s11136-015-1127-z (opens in a new tab)

Cited by papers we hold 7

• Psychometric validation of a brief self-report measure of misophonia symptoms and functional impairment: The duke-vanderbilt misophonia screening questionnaire · 2022
  Not checked
• Análisis de Redes: una Alternativa a los Enfoques Clásicos de Evaluación de los Sistemas Educativos · 2022
  Not checked
• Item response theory and differential test functioning analysis of the HBSC-Symptom-Checklist across 46 countries · 2022
  Not checked
• Psychological Differences Among Healthcare Workers of a Rehabilitation Institute During the COVID-19 Pandemic: A Two-Step Study · 2021
  Not checked
• Psychometric Network Models from Time-Series and Panel Data · 2020
  No notice
• Searching for G: A New Evaluation of SPM-LS Dimensionality · 2019
  Not checked
• Networks of Depression and Anxiety Symptoms Across Development · 2018
  No notice

Questions about this paper

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.