A theoretical model of the relationship between the h-index and other simple citation indicators

1Some simple and basic bibliometric indicators, such as the total number of citations C, the total number of publications with at least a number of citations k each, T k, the total number of citations for the t most cited papers, C t, the average number of citations per paper (ACPP), (where, hereafter, T stands for T 0), as well as the h-index (Hirsch; Braun et al.; Schubert and Glänzel; Harzing and van der Wal ), are routinely used to measure the relevance and citation impact of journals when computed according to suitable, pre-specified timeframes. In particular, time-limited versions of the ACPP lead to different types of “impact factors”, with possible variants defined according to different pre-specified publication and citation time windows, and also depending on the degree of overlap between these timeframes (synchronous and diachronous impact factors; Ingwersen et al. ). Similarly, alternative versions of the h-index have been defined (synchronous and diachronous h-indexes; Bar-Ilan ). In general, all these indicators merge information about the number of citations received by a journal within a pre-specified time window—typically a huge amount of data—into a single representative value interpretable as a measure of a journal’s “quality”. Their computation requires knowledge of the entire citation pattern, or at least most of it. In recent years, a certain interest has been shown in developing theoretical models with which to “estimate” one such indicator given the values of certain others. Well-known representative examples are theoretical models with which to obtain the value of the h-index, h:as a function of C (Hirsch ),as a function of T (Egghe and Rousseau ),as a function of T 1 (Burrell ),as a function of C and T (Glänzel; Iglesias and Pecharroman; Schubert and Glänzel; Bletsas and Sahalos; Egghe et al.; Egghe and Rousseau ),as a function of C, T 1 and C 1 Bertoli-Barsotti and Lando; but also theoretical models with which to estimate C, as a function of h (Petersen et al. ), or as a function of m and h (Egghe et al. ), or as a function of T and h (Burrell ), and so on. These models—usually based, in their turn, on the assumption of a specific probabilistic model for the citation distribution—may be effective, for instance, when the indicator of interest cannot be obtained directly because it is not accessible, or when the availability of citation data is incomplete. For example, there may be the case in which h is not available but we know C and T (Glänzel; Schubert and Glänzel; Bletsas and Sahalos ), or the case in which we have to impute missing values of impact factors using the availability of the h-index as a predictor (Bertocchi et al. ).

2as a function of C (Hirsch ),

3as a function of T (Egghe and Rousseau ),

4as a function of T 1 (Burrell ),

5as a function of C and T (Glänzel; Iglesias and Pecharroman; Schubert and Glänzel; Bletsas and Sahalos; Egghe et al.; Egghe and Rousseau ),

6as a function of C, T 1 and C 1 Bertoli-Barsotti and Lando;

7In particular, in this paper we focus mainly on the problem of obtaining an explicit “universal” formula for estimating the actual value of the h-index. Recently, Burrell and Bertoli-Barsotti and Lando introduced a model that has proved very effective in estimating the actual value of the h-index for individual scientists. More precisely, these approaches lead (or may lead) to two slightly different formulas, being based, respectively, on a “standard” and a “shifted” version of the geometric distribution. In the first part of section ‘Methods’ we present a (functional) equation, based on the geometric distribution, that constitutes a theoretical basis for both these approaches. Indeed, this equation allows us to derive a closed-form estimator of the h-index, expressed as a function of (some of) the above citation metrics. We shall call this estimator, for reasons which will be apparent below, the Lambert-W formula for the h-index.

8In the related scientific literature, authors often limit their analysis to the problem of estimating the unknown parameters of a suggested theoretical parametric model for the h-index, under the assumption of knowing the real values of the h-index. Instead, in this paper we consider the more practical (and in a certain sense, opposing) problem of determining the (unknown) h-index on the basis of a ready-to-use formula for it. Then, in our empirical analyses we will use the actual values of the h-index but only to evaluate, a posteriori, the performance of the proposed ready-to-use formulas and not to determine (maybe for interpretative reasons) unknown parameters of a theoretical parametric model. In this paper, we will concentrate on the case of the h-index for journals (Braun et al. ). One of the major differences between the cases of an individual scientist and a journal is that, in the latter, the h-index should be computed in a “timed” version, i.e. limited to suitable, usually relatively short, publication and citation time windows. In this regard, it should be noted that a familiar definition such as “a journal has index h if h of its publications each have at least h citations and the other publications each have no more than h citations” is somewhat inaccurate because it does not specify the time windows to be considered for the calculation of h. One of the aims of our study will also be to test the robustness of the formula empirically against different possible choices of (1) length of the time windows and (2) type of approach adopted for analyzing the citation process: “prospective” (diachronous) or “retrospective” (synchronous) (Glänzel ). We shall also focus on a comparison of effectiveness between the Lambert-W formula for the h-index and a popular class of alternative models, related to the so-called Glänzel–Schubert formula, that have already been proved to be highly correlated to the h-index.

9In the second part of section ‘Methods’ we review the existing literature on the Glänzel–Schubert family of models (and related models) and discuss some problematic aspects linked to the presence of unknown parameters in their expressions. Then, in section ‘Two empirical studies’, we report the results of an empirical comparison between the Lambert-W formula for the h-index and these alternative models, using two different dataset of journals. For this task, we downloaded citation data from the Scopus database on about 100,000 and 20,000 publications, respectively, for the first and the second dataset. Based on the results of our research study, we conclude that the Lambert-W formula for the h-index provides an effective ready-to-use rule that should be preferred to other known formulas if one’s goal is (simply) to derive a reliable estimate of the h-index.

10A model of a hypothetical equation of the typeis sought, connecting h, T and C. Naturally, we do not assume a deterministic relationship among observed values of h, T and C, rather, we shall determine a “probabilistic” relationship. Indeed, the problem addressed here is that of deriving a formula for predictions. In particular, we try to identify a model that is able to predict one input-term given the other two (e.g. h given T and C, or C given h and T, or, which is the same, C/T given h and T, and so on). A preliminary solution of the functional Eq. (1) can be obtained by “assuming” (which here represents a simple working hypothesis) the geometric distribution (GD) with parameter P,where p(x) gives the probability of observing x and P, P > 0, represents the expectation of the GD (Johnson et al., p. 210). Then the value expresses the “expected” number of articles with x citations (size-frequency function). Now, since for every k,,, the predicted number of papers with at least k citations is

11By definition of the h-index, h, this yields the equation. Then, assuming as an estimate of the expectation P (see Johnson et al., Eq. 5.12, p. 211), we derive the following model of functional equation

12We note in passing that this model yields, as a byproduct, the formula for the “uncitedness factor”, providing proof of the result conjectured by Hsu and Huang (see also Egghe; Burrell ). This equation represents a theoretical model of the relationship among the h-index, the number of publications T and the ACPP, m. Equation (4) can be solved with respect to any of its arguments. In particular,Given h and T, we easily obtain an estimate of the expectation P as follows:andGiven T and C, we obtain an estimate of h as follows. Equation (4) is equivalent to, where and. Then, multiplying each side of the latter equation by log a, and substituting, we obtain, which leads immediately to the solutionwhere represents the so-called Lambert-W function (Corless and Jeffrey ). Remember that the Lambert-W function is the function W(y) satisfying, and can be currently computed using mathematical software, for example the Mathematica® 10.0 software package (Wolfram Research, Inc.; it is implemented in the Wolfram Language as “LambertW”), or also using the R statistical computing environment (R Development Core Team ).Hencethat is, equivalently,where we have adopted a new symbol for differentiating the “predicted” h-index,, from the actual value h of the h-index. Note that the GD approach has been previously suggested by Burrell but without giving an explicit formula, in closed form, for the estimation of the h-index.

13Given h and T, we easily obtain an estimate of the expectation P as follows:and

14Given T and C, we obtain an estimate of h as follows. Equation (4) is equivalent to, where and. Then, multiplying each side of the latter equation by log a, and substituting, we obtain, which leads immediately to the solutionwhere represents the so-called Lambert-W function (Corless and Jeffrey ). Remember that the Lambert-W function is the function W(y) satisfying, and can be currently computed using mathematical software, for example the Mathematica® 10.0 software package (Wolfram Research, Inc.; it is implemented in the Wolfram Language as “LambertW”), or also using the R statistical computing environment (R Development Core Team ).

15Hencethat is, equivalently,where we have adopted a new symbol for differentiating the “predicted” h-index,, from the actual value h of the h-index. Note that the GD approach has been previously suggested by Burrell but without giving an explicit formula, in closed form, for the estimation of the h-index.

16As a general rule, one should expect that knowledge of other (i.e., other than m and T) simple summary statistics of the raw citation data will help increase the precision of the h-index estimate. Indeed, if we also assume that we know T 1, a modified version of the above formulas can be easily introduced by taking the shifted-geometric distribution (SGD) with parameter Qwhere p(y) represents the probability of observing the number of citations y of a paper cited at least once, and Q, Q > 1, represents the expectation of the SGD. Since for every k,,, then represents the number of papers with at least k + 1 citations. Then, assuming, the average number of citations of articles that have been cited at least once, as a proxy for the expectation Q, we derive the following functional equation

17This equation can be solved with respect to any of its arguments. In particular,Given h and T 1, we obtainandGiven T 1 and C, and following a completely analogous sequence of steps as in the above point (b), we obtain the estimate of h

18Given h and T 1, we obtainand

19Given T 1 and C, and following a completely analogous sequence of steps as in the above point (b), we obtain the estimate of h

20If we also know the total number of citations of the most cited paper, C 1, we can hope to improve the accuracy of the above formula further. Indeed, with the use of the trimmed mean—that is, the sample mean obtained omitting the most highly cited paper— instead of m 1, we obtain a modified (improved) version of the above formula, which we shall define,

21As is well known, citation distributions are highly skewed; hence the sample mean is distorted by extreme values. In particular, the presence of individual highly-cited papers tends to overestimate C, and consequently, in comparison to the true h-index—that is clearly insensitive to a single very highly cited paper. In this sense, the use of a trimmed mean is simply a technique for reducing this possible bias.

22To summarize, we have: or also, equivalently,, and or also, equivalently,. We shall refer to these formulas as Lambert-W formulas for the h-index, respectively, in a “basic”,, and an “improved” version,. The formula has been considered elsewhere Bertoli-Barsotti and Lando for the estimation of the h-index for individual scientists.

23A well-known alternative “theoretical model of the dependence of the citation h-index on the sample size and the sample’s mean citation rate” (Schubert et al. ) is the one proposed by Schubert and Glänzel, who noted that the h-index is approximately proportional to “a power function of the sample size and the sample mean”, namely to the function (Schubert et al.; see also Glänzel, ). In applications, this fact has given rise to a plethora of “variants”, as possible parametric models for the h-index. It is useful to distinguish each of them with the following nine cases.Iglesias and Pecharroman derived the following one-parameter family of models of the h-index: where (the formula was reported by Iglesias and Pecharroman with parameter ). Glänzel estimated this model in an empirical comparative study of h-index for journals. He found that the estimate of the power parameter depends on the length of the citation window considered. In particular, he found that the formula (α = 2 in his notation, which corresponds to η = 2/3 in ours) is appropriate “for small windows comprising an initial period of about 3 years after publication”.From the above model, Iglesias and Pecharroman also obtained, for η = 2/3, the ready-to-use formula: (see also Panaretos and Malesios; Vinkler,; Ionescu and Chopard ).By starting from a continuous probability distribution—a Pareto distribution of the second kind, (Johnson et al., p. 575; Arnold, p. 44), also known as the Lomax distribution (Lomax ), where, represents the probability of observing a number greater than x, x > 0—and estimating its expectation (that exists if ) by the sample mean m, Schubert and Glänzel (see also Glänzel ) derived a slightly more general two-parameter model: here defined as also reported by Bletsas and Sahalos; see their Eq. (4)), as an approximate (and generalized) solution of the equation where. In words, model (16) states that “the h-index can be approximated by a power function of the sample size and the sample mean” (Schubert et al. ). It is important to note that the model is similar to but different from the above model, because in the former the proportionality constant is not merely a function of the power parameter η, while in the latter γ represents a free parameter. This gives rise to a more flexible model. Malesios estimated the parameters of model (16) in a study on 134 journals in the field of ecology and 54 journals in the field of forestry sciences. He obtained the best fit, respectively, with the estimates (0.64, 0.7) and (0.66, 0.78) for the pair (η, γ) (in our parameterization).The above Pareto distribution of the second kind has also recently become known as the Tsallis distribution (Tsallis and de Albuquerque ). More specifically, with reparameterization and, the probability of observing a number greater than x, x > 0, becomes equal to (see Bletsas and Sahalos; Shalizi ). Bletsas and Sahalos suggest obtaining an estimate of the h-index as the numerical solution of the Eq. (17), that is for a pre-specified fixed value of the unknown parameter q. Let us call the (implicit) solution of Eq. (18). It is important to stress that, unlike all the other estimators of h-index considered in the present study, a closed-form expression for h T does not exist. Nevertheless, in an empirical application to a set of electrical engineering journals, Bletsas and Sahalos found a very good fit between measured and estimated values of the h-index, assuming Tsallis distribution with parameter q = 1.5 and q = 1.6. It is interesting to note that these values correspond, respectively, to η = 2/3 and η = 0.625, since.For a special choice of the power parameter (η = 2/3 in the present parameterization) in model (16), Schubert and Glänzel derived the celebrated one-parameter model also known as the Glänzel–Schubert model of the h-index. This model has been widely used (mainly for interpretative purposes—i.e. to provide a better understanding of the “mathematical properties” of the h-index) because several empirical studies suggest the existence of a strong correlation between h-index and. Its drawback (as with model (16)) is obviously that the value of the proportionality constant γ is unknown. Certainly, this parameter can be determined (ex post) empirically, but it is likely to vary from case to case (Prathap; Alguliev et al. ). Then, as a ready-to-use formula for estimating the h-index a priori, the Glänzel–Schubert model is in fact unusable. Sometimes researchers find an ex post least square estimate of the parameter γ, starting from known values of the h-index. In different contexts, and for different datasets, the estimate of the γ parameter has been found to vary appreciably, in that it turns out to range approximately from 0.7 to 0.95. Indeed, for example, Schubert and Glänzel found, for γ, the estimates 0.73 and 0.76, in a study on the h-index for journals, for two different sets of journals, while Csajbók et al. found an estimate of γ of 0.93 in a macro-level analysis of the h-index for countries. Instead, other authors, among them Annibaldi et al., Bouabid et al. and Zhao et al., have found values of around 0.8. In quite different contexts (partnership ability and h-index for networks) Schubert and Schubert et al. have estimated the parameter γ of the model, obtaining values within the range 0.6–0.96.In the absence of a specific value of the proportionality constant γ, researchers sometimes decide to set γ equal to a fixed arbitrary value γ 0, obtaining a ready-to-use formula In the framework of the analysis of the h-index for journals, ready-to-use formulas for estimating the h-index with the formula have been adopted, for example, by Bletsas and Sahalos, with the choice. Instead, for example, Ye and Elango et al. adopted the rule to set for journals and for other sources. Abbas and Vinkler also adopted the choice. It is worth noting that the latter value leads to the formula, which coincides with the so-called p-index defined by Prathap. Finally, note that.As noted above, empirical analyses suggest a “strong linear correlation” between the h-index and the function (Schubert and Glänzel; Glänzel; Schreiber et al.; Malesios ). Strictly speaking, this only means that when h is plotted against, the data fall fairly close to a straight line. In other terms, h is approximately equal to, for suitable choices of the parameters δ and γ. Indeed, the following three-parameter model has been considered in literature (see Bador and Lafouge ) In a comparative analysis of two samples of 50 journals (taken from the ‘‘Pharmacology and Pharmacy’’ and ‘‘Psychiatry’’ sections of the Journal Citation Reports 2006), Bador and Lafouge obtained the LS estimates of the parameters δ and γ for different fixed values of the power parameter η (values of “α close to 2”, in their parameterization, where ). Their best estimates of the proportionality constant γ ranged from 0.7 to 0.8, with an intercept point always very close to 1. Based on these results, and a fortiori, underestimate the h-index.For the particular choice of the power parameter η = 2/3 in the above model, we obtain the two-parameter model This model directly generalizes the above Glänzel–Schubert model by introducing a free intercept parameter, δ. Tahira et al. tested this model in a scientometric analysis of engineering in Malaysian universities. They found the estimates δ = −0.28 and γ = 0.97.Finally, by assuming a linear dependence between the h-index and the function in a double logarithmic axis plot (log–log plot), one may define the following three-parameter model (see Radicchi and Castellano ) Indeed, after taking logs, this corresponds to a regression relationship between log h and the linear model, where. Needless to say, model is similar to but essentially different from the above models (a)–(h). Radicchi and Castellano analyzed the scientific profile of more than 30,000 researchers. They found a good linear correlation, in a log–log plot, between the true h-index and the values given by the model. Using this relationship, they obtained, in particular, the least square estimate of the parameter η:. It is quite puzzling to observe that the solution reached by Radicchi and Castellano is out of the parameter space of all the above models (η > 0.5).

24Iglesias and Pecharroman derived the following one-parameter family of models of the h-index: where (the formula was reported by Iglesias and Pecharroman with parameter ). Glänzel estimated this model in an empirical comparative study of h-index for journals. He found that the estimate of the power parameter depends on the length of the citation window considered. In particular, he found that the formula (α = 2 in his notation, which corresponds to η = 2/3 in ours) is appropriate “for small windows comprising an initial period of about 3 years after publication”.

25From the above model, Iglesias and Pecharroman also obtained, for η = 2/3, the ready-to-use formula: (see also Panaretos and Malesios; Vinkler,; Ionescu and Chopard ).

26By starting from a continuous probability distribution—a Pareto distribution of the second kind, (Johnson et al., p. 575; Arnold, p. 44), also known as the Lomax distribution (Lomax ), where, represents the probability of observing a number greater than x, x > 0—and estimating its expectation (that exists if ) by the sample mean m, Schubert and Glänzel (see also Glänzel ) derived a slightly more general two-parameter model: here defined as also reported by Bletsas and Sahalos; see their Eq. (4)), as an approximate (and generalized) solution of the equation where. In words, model (16) states that “the h-index can be approximated by a power function of the sample size and the sample mean” (Schubert et al. ). It is important to note that the model is similar to but different from the above model, because in the former the proportionality constant is not merely a function of the power parameter η, while in the latter γ represents a free parameter. This gives rise to a more flexible model. Malesios estimated the parameters of model (16) in a study on 134 journals in the field of ecology and 54 journals in the field of forestry sciences. He obtained the best fit, respectively, with the estimates (0.64, 0.7) and (0.66, 0.78) for the pair (η, γ) (in our parameterization).

27The above Pareto distribution of the second kind has also recently become known as the Tsallis distribution (Tsallis and de Albuquerque ). More specifically, with reparameterization and, the probability of observing a number greater than x, x > 0, becomes equal to (see Bletsas and Sahalos; Shalizi ). Bletsas and Sahalos suggest obtaining an estimate of the h-index as the numerical solution of the Eq. (17), that is for a pre-specified fixed value of the unknown parameter q. Let us call the (implicit) solution of Eq. (18). It is important to stress that, unlike all the other estimators of h-index considered in the present study, a closed-form expression for h T does not exist. Nevertheless, in an empirical application to a set of electrical engineering journals, Bletsas and Sahalos found a very good fit between measured and estimated values of the h-index, assuming Tsallis distribution with parameter q = 1.5 and q = 1.6. It is interesting to note that these values correspond, respectively, to η = 2/3 and η = 0.625, since.

28For a special choice of the power parameter (η = 2/3 in the present parameterization) in model (16), Schubert and Glänzel derived the celebrated one-parameter model also known as the Glänzel–Schubert model of the h-index. This model has been widely used (mainly for interpretative purposes—i.e. to provide a better understanding of the “mathematical properties” of the h-index) because several empirical studies suggest the existence of a strong correlation between h-index and. Its drawback (as with model (16)) is obviously that the value of the proportionality constant γ is unknown. Certainly, this parameter can be determined (ex post) empirically, but it is likely to vary from case to case (Prathap; Alguliev et al. ). Then, as a ready-to-use formula for estimating the h-index a priori, the Glänzel–Schubert model is in fact unusable. Sometimes researchers find an ex post least square estimate of the parameter γ, starting from known values of the h-index. In different contexts, and for different datasets, the estimate of the γ parameter has been found to vary appreciably, in that it turns out to range approximately from 0.7 to 0.95. Indeed, for example, Schubert and Glänzel found, for γ, the estimates 0.73 and 0.76, in a study on the h-index for journals, for two different sets of journals, while Csajbók et al. found an estimate of γ of 0.93 in a macro-level analysis of the h-index for countries. Instead, other authors, among them Annibaldi et al., Bouabid et al. and Zhao et al., have found values of around 0.8. In quite different contexts (partnership ability and h-index for networks) Schubert and Schubert et al. have estimated the parameter γ of the model, obtaining values within the range 0.6–0.96.

29In the absence of a specific value of the proportionality constant γ, researchers sometimes decide to set γ equal to a fixed arbitrary value γ 0, obtaining a ready-to-use formula In the framework of the analysis of the h-index for journals, ready-to-use formulas for estimating the h-index with the formula have been adopted, for example, by Bletsas and Sahalos, with the choice. Instead, for example, Ye and Elango et al. adopted the rule to set for journals and for other sources. Abbas and Vinkler also adopted the choice. It is worth noting that the latter value leads to the formula, which coincides with the so-called p-index defined by Prathap. Finally, note that.

30As noted above, empirical analyses suggest a “strong linear correlation” between the h-index and the function (Schubert and Glänzel; Glänzel; Schreiber et al.; Malesios ). Strictly speaking, this only means that when h is plotted against, the data fall fairly close to a straight line. In other terms, h is approximately equal to, for suitable choices of the parameters δ and γ. Indeed, the following three-parameter model has been considered in literature (see Bador and Lafouge ) In a comparative analysis of two samples of 50 journals (taken from the ‘‘Pharmacology and Pharmacy’’ and ‘‘Psychiatry’’ sections of the Journal Citation Reports 2006), Bador and Lafouge obtained the LS estimates of the parameters δ and γ for different fixed values of the power parameter η (values of “α close to 2”, in their parameterization, where ). Their best estimates of the proportionality constant γ ranged from 0.7 to 0.8, with an intercept point always very close to 1. Based on these results, and a fortiori, underestimate the h-index.

31For the particular choice of the power parameter η = 2/3 in the above model, we obtain the two-parameter model This model directly generalizes the above Glänzel–Schubert model by introducing a free intercept parameter, δ. Tahira et al. tested this model in a scientometric analysis of engineering in Malaysian universities. They found the estimates δ = −0.28 and γ = 0.97.

32Finally, by assuming a linear dependence between the h-index and the function in a double logarithmic axis plot (log–log plot), one may define the following three-parameter model (see Radicchi and Castellano ) Indeed, after taking logs, this corresponds to a regression relationship between log h and the linear model, where. Needless to say, model is similar to but essentially different from the above models (a)–(h). Radicchi and Castellano analyzed the scientific profile of more than 30,000 researchers. They found a good linear correlation, in a log–log plot, between the true h-index and the values given by the model. Using this relationship, they obtained, in particular, the least square estimate of the parameter η:. It is quite puzzling to observe that the solution reached by Radicchi and Castellano is out of the parameter space of all the above models (η > 0.5).

33The Research Evaluation Exercise for the period 2011–2014 named “Valutazione della Qualità della Ricerca 2011–2014” (hereinafter VQR) is a national research assessment exercise organized under the aegis of the Italian Ministry of Education, University and Research for evaluating and ranking all Italian scientific institutions (typically, all national universities and research centers), on the basis of the quality of their research outcomes. The results obtained are particularly important because they determine the allocation of government funding to Italian universities. The VQR is carried out under the responsibility of a National Agency for the Evaluation of University and Research, the “Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca” (ANVUR), and is organized with reference to 14 different academic fields, or Areas. The research assessment is actually conducted by Groups of Evaluation Experts (GEV, in the Italian acronym), one for each Area. For our first empirical analysis, we consider the so-called Area 13—Scienze economiche e statistiche—Economics and Statistics. The evaluation of each researcher is based on the quality of his/her research outcomes published during the period 2011–2014. As a general rule, the evaluation of a research product for Area 13 is made at journal-level. This means that journal bibliometric indicators are used as surrogate measures to quantify the quality of each individual research product (published in that journal). For this purpose, a list of “relevant” journals for Area 13 has been compiled by the corresponding GEV (the so-called GEV 13) and suitable journal-based metrics are extracted to this end from three sources, that is: Web of Science (WoS), Scopus, and Google Scholar (GS). The full list of the “relevant” journals for Area 13 includes 2717 journals and may be found on the ANVUR website (www.anvur.org). Each journal on the Area 13 list was individually assigned to one of five sub-areas, among them “Statistics and Mathematical Methods” (S&MM). For the purpose of our case study, we selected a somewhat homogeneous list of journals using the following steps:we considered all and only the journals (568 journals) belonging to the sub-area S&MM;to facilitate possible comparisons between databases, the journals selected were subsequently restricted to only those (253) journals indexed by all three databases: WoS, Scopus and GS;we excluded 15 journals with incomplete issues within the period under investigation, 2010–2014;finally, in order to preserve the homogeneity of the sample, we excluded 6 journals with a “too large” number of published papers (more than 2000) and 1 journal that publishes only online.

34we considered all and only the journals (568 journals) belonging to the sub-area S&MM;

35to facilitate possible comparisons between databases, the journals selected were subsequently restricted to only those (253) journals indexed by all three databases: WoS, Scopus and GS;

36we excluded 15 journals with incomplete issues within the period under investigation, 2010–2014;

37finally, in order to preserve the homogeneity of the sample, we excluded 6 journals with a “too large” number of published papers (more than 2000) and 1 journal that publishes only online.

38Our final sample included 231 journals. According to the Scopus classification, these journals belong to a number of different “Subject Areas”. Table 1 shows the “Subject Areas” in which the 231 journals selected from the S&MM list are placed by Scopus (it should be recalled that Scopus classifies journal titles into 27 major thematic categories and a journal may belong to more than one category).

39After selecting the S&MM list of journals, we retrieved citation data from the Scopus database. According to the VQR time-span, we considered all documents within the publication window of 5 years (2010–2014) (in fact GEV13 considers the 5-year Google Scholar’s h-index, for the period 2010–2014) and the citations that these items received until the time of accessing the database (last week of December 2015). This means a 6-year citation window, 2010–2015, over a 5-year publication window: 2010–2014. Harzing and van der Wal considered similar timeframes in a study on a set of journals in the area of economics and business. Overall, the dataset obtained included 99,409 publications receiving (until December 2015) a total of 485,628 citations. The complete list of the 231 journals in the S&MM dataset is reported in Table 2, where each journal is identified by its ISSN code. For each journal, we manually computed, on the basis of the citations downloaded, the actual value h of the h-index, as: the largest number of papers published in the journal between 2010 and 2014 and which obtained at least h citations each, from the time of publication until December 2015. Table 2 reports, for each journal, the h-index, h, and its estimates, obtained (1) with the Lambert-W formulas for the h-index,,, and, as a comparison, (2) with the Glänzel–Schubert formula,, for different values of the proportionality constant γ 0, namely, 0.63, 0.7, 0.8, 0.9, 1 (note that identifies formula ), and (3) by means of a numerical solution of Eq. (18), for different values of q 0, namely, 1.2, 1.4, 1.6. Table 2 also reports: the total number of citations, C; the total number of publications, T; the total number of publications cited at least once, T 1; the total number of citations of the most cited paper, C 1. To facilitate comparisons, have all been rounded to the nearest integer to produce numbers in the same range of values as the h-index.

40We also analyzed a second dataset, based on the citation data of the top 100 journals, within the Scopus subject area of “Economics, Econometrics and Finance”, ranked according to the Scopus journal impact factor, i.e. the Impact per Publication (IPP) 2014. The list (let us call it the EE&F list) may be found at http://www.journalindicators.com and it consists of journals with a minimum number of 50 publications. We recall that the IPP 2014 of a journal is basically the average number of citations received by papers published in 2014 (registered in the Scopus database), to papers published by the same journal from 2011 until 2013. In particular, Scopus takes account of the following types of citable items and citing sources: articles, reviews, and conference papers. All other documents (e.g. notes, letters, articles in press, erratum, etc.) are excluded from the computation. We downloaded from Scopus the citation data of all 100 journals on the aforementioned list during the last week of April, 2016. The dataset obtained included 19,889 publications receiving a total of 74,096 citations (during 2014). The complete list of these journals is reported in Table 3, where each journal is identified by its ISSN code. Differently from above, we excluded all non-citable items (e.g. notes, etc.) in order to obtain sets of publications as close as possible to those employed for the computation of IPPs by Scopus. Once the set of papers for each journal has been selected, it is possible to request a citation report (“view citation overview”) and download the citations per paper received in the year 2014: that is, all and only the citations needed for the computation of the IPP 2014. In fact, we found some positive differences between the actual values of, with an average value over all 100 journals of 3.8, and the official IPPs 2014, with an average value of 3. These differences may be due to: (1) a delayed update of the database (the IPPs were published by Scopus in June 2015), and (2) a larger set of citing sources and documents (with Scopus, it is not possible to limit the citation report to particular citing sources or documents). Similar differences between official and observed values have been found and discussed, for instance, by Leydesdorff and Opthof, Stern and Seiler and Wohlrabe. Nonetheless, in this case the ACPP should, theoretically, represent a 3-year synchronous impact factor for the year 2014 (Ingwersen et al.; Ingwersen ) in that we considered only citations received during 2014 of papers published within the previous 3 years. For each journal, we manually computed the actual value h of the h-index as the largest number of papers published in the journal between 2011 and 2013 and which obtained at least h citations each in the year 2014. Ultimately, we obtained a synchronous h-index (Bar-Ilan ), for a 1-year citation window.

41In the same way as above, for each journal we manually computed the actual value h of the h-index. Table 3 reports, for each journal, the h-index, h, and the other indicators also considered in Table 2, namely,,, for, the numerical solution of Eq. (18), for different values of q 0, namely, as well as the simple basic metrics C, T, T 1 and C 1.

42The h-index is, today, one of the tools most commonly used to rank journals (Braun et al.; Vanclay,; Schubert and Glänzel; Bornmann et al.; Harzing and van der Wal; Liu et al.; Hodge and Lacasse; Bornmann et al.; Mingers et al.; Xu et al. ). Indeed, its value is currently provided by all the three major citation databases, WoS, Scopus and GS. In an earlier study (Bertoli-Barsotti and Lando ) the Lambert-W formula for the h-index was proved to be a good estimator of the h-index for authors. In this paper, we have extended the empirical study to the case of the h-index for journals. One of the major differences between the case of an individual scientist and that of a journal, for the computation of the h-index, is the role played by publication and citation time windows, and the approach adopted for the analysis and interpretation of the citation process (“prospective” vs “retrospective”; Glänzel ). As stressed by Braun et al.: “The journal h-index would not be calculated for a “life-time contribution”, as suggested by Hirsch for individual scientists, but for a definite period”. In fact, “Hirsch did not limit the period in which the citations were received” (Bar-Ilan ). Unlike the case of individual scientists, and in view of a comparative assessment, calculations of a journal’s h-index must be timed (note that a notion of “timed h-index” has also been recently introduced by Schreiber, for the case of individual scientists), i.e. it must be referred to standardized time periods of journal coverage, for example of 2, 3 or 5 years, as is usually done for the computation of the impact factor, in order to limit the typical size-dependency of the h-index—that is, its dependency on the total number of publications (an indicator is said to be size-dependent if it never decreases when new publications are added, Waltman ). A journal’s “impact factor” is essentially a time-limited version of the average number of citations by papers published in the journal in a given period of time. Several types of “impact factors” may be defined, depending on different time windows considered for publication and citation data and, possibly, different approaches to the analysis of the citation process, leading to synchronous or diachronous impact factors (Ingwersen et al.; Ingwersen ). In its WoS form, the publication window is 2 years (defining the 2-year Impact Factor, IF) or 5 years (defining the 5-year Impact Factor, IF5), while Scopus adopts a 3-year publication window for its IPP. In all these cases, the impact factor is computed in a synchronous mode, i.e. the citations used for the calculation are all received during the same fixed period—1 year, in these cases.

43In this paper, we first presented the Lambert-W formula for the h-index in two versions (differing on the basis of the various citation metrics on which they depend), a basic version and an improved version, respectively and. Then we tested, by means of an empirical study, their efficiency and effectiveness, as well as:that of another popular theoretical model for the h-index that has been successfully applied elsewhere to the same type of application, i.e. the Glänzel–Schubert formula,, for different values of the free parameter γ 0, and secondly,that given by the numerical solution of Eq. (18), for different values of the free parameter q 0.

44that of another popular theoretical model for the h-index that has been successfully applied elsewhere to the same type of application, i.e. the Glänzel–Schubert formula,, for different values of the free parameter γ 0, and secondly,

45that given by the numerical solution of Eq. (18), for different values of the free parameter q 0.

46We compared the performances of these formulas as estimators of the h-index—in particular, in terms of accuracy and robustness—with an empirical study conducted on two different samples of journals. We computed the h-index manually, on the basis of citations downloaded. In our empirical study, in the first dataset (S&MM), the ACPP can be interpreted as a diachronous impact factor (Ingwersen et al.; Ingwersen ), because for each paper the citations are counted from the moment of publication until the time of accessing the database (as in the case of individual scientists). More specifically, we computed an “impact factor” involving a 6-year citation window over a 5-year publication window. As to be expected, due to the larger citation window, we obtained, for all 231 journals, the averages of 4.4 and 1.5 respectively for m and IF5{2014}, the traditional 5-year impact factors 2014, as published by WoS in its Journal Citation Report. Moreover, we also observed a high level of Pearson correlation, ρ, between m and IF5{2014}, that is: (quite similar to that observed between IF5{2014} and IF{2014}, the WoS 2-year and impact factors 2014, that is: ). Instead, in the second dataset (EE&F), m can be interpreted as a 3-year impact factor in its ordinary synchronous version, as computed by Scopus. Hence, following the terminology of Bar-Ilan, we obtained a diachronous and a synchronous h-index, respectively, in the first and second empirical study. To evaluate the measure of fit of an estimate of the h-index, say (rounded to the nearest natural number), with respect to the exact value h j, we computed the absolute relative error and the squared relative error for each journal j, j = 1,…,J. Then, as a criterion with which to assess the overall quality of the various estimators considered in the paper, we computed the mean absolute relative error, and the root mean squared relative error, for each estimator.As expected, the Pearson correlation between the actual value h of the h-index and each of its estimates, and, was very high, for both S&MM and EE&F datasets. In particular, this confirms previous empirical results concerning the formula (see Schubert and Glänzel; Glänzel ). Indeed, ρ always exceeded 0.97. More specifically, we found the following: for the S&MM dataset, and; for the EE&F dataset, and. Nevertheless, as can be seen from Figs. 2 and 4, a high correlation does not specifically identify a “good” estimator for the h-index. Formula yielded similar levels of correlation, but a much lower level of MARE, see Figs. 1 and 3 (be aware that the figures refer to non-rounded values of the estimates). Note that the correlation between the h-index and does not depend on the unknown value of, while, at the same time, the MARE of depends heavily on the choice of. As can be seen from Table 4, at its best (among the values of tested), the error of reached its minimum (in terms of both MARE and RMSRE), for, for the dataset S&MM, while for the EE&F dataset the error of is at its minimum for a slightly different value of γ 0, i.e. γ 0 = 0.8. This confirms that, for fixed values of γ 0, the effectiveness of the formula may depend on the length of the citation window considered (Glänzel ) and, finally, that there is no “universal” optimal value for the constant γ 0 in the formula. Instead, for both datasets, the formula gives similar, and even smaller, levels of error (in terms of both MARE and RMSRE). The approach that consists of obtaining the numerical solution of Eq. (18) was also considered. We tentatively tested this method for nine different values of the free parameter q between 1 and 2, i.e. q 0 = 1.1, 1.2,…,1.9. As expected, the resulting estimates were more or less accurate depending on the set value of q 0. Of the nine values of q 0 tested, the smallest estimation error was obtained for a q 0 value equal to around 1.4 (MARE = 0.065; RMSRE = 0.094), for the S&MM dataset, and for a q 0 value equal to around 1.2 (MARE = 0.058; RMSRE = 0.093) for the EE&F dataset (see Table 4). Ultimately, h T was found to be the most accurate estimator (if one takes q 0 = 1.4), of those included in Table 4, for the S&MM dataset and the third best (if one takes q 0 = 1.2), for the EE&F dataset. Overall, the errors are not dramatically different in the range of q between 1.2 and 1.6, and then a value of q 0 = 1.5, also tested by Bletsas and Sahalos, may be a good compromise solution. The Pearson correlation between the actual value h of the h-index and its estimate varies slightly according to the selected value of q 0, but it is still very high: in particular, for q 0 = 1.5, we obtain for the S&MM dataset and for the EE&F dataset. Hence, overall, the method may lead to a very good fit, but it has two main drawbacks. First, the expression of is not given by any explicit formula. Second, this method continues to suffer from the problem of the conventional choice of an unknown parameter, in that we do not know a priori the value of the parameter q that will yield the “smallest” estimation error.

47As expected, the Pearson correlation between the actual value h of the h-index and each of its estimates, and, was very high, for both S&MM and EE&F datasets. In particular, this confirms previous empirical results concerning the formula (see Schubert and Glänzel; Glänzel ). Indeed, ρ always exceeded 0.97. More specifically, we found the following: for the S&MM dataset, and; for the EE&F dataset, and. Nevertheless, as can be seen from Figs. 2 and 4, a high correlation does not specifically identify a “good” estimator for the h-index. Formula yielded similar levels of correlation, but a much lower level of MARE, see Figs. 1 and 3 (be aware that the figures refer to non-rounded values of the estimates). Note that the correlation between the h-index and does not depend on the unknown value of, while, at the same time, the MARE of depends heavily on the choice of. As can be seen from Table 4, at its best (among the values of tested), the error of reached its minimum (in terms of both MARE and RMSRE), for, for the dataset S&MM, while for the EE&F dataset the error of is at its minimum for a slightly different value of γ 0, i.e. γ 0 = 0.8. This confirms that, for fixed values of γ 0, the effectiveness of the formula may depend on the length of the citation window considered (Glänzel ) and, finally, that there is no “universal” optimal value for the constant γ 0 in the formula. Instead, for both datasets, the formula gives similar, and even smaller, levels of error (in terms of both MARE and RMSRE).

48The approach that consists of obtaining the numerical solution of Eq. (18) was also considered. We tentatively tested this method for nine different values of the free parameter q between 1 and 2, i.e. q 0 = 1.1, 1.2,…,1.9. As expected, the resulting estimates were more or less accurate depending on the set value of q 0. Of the nine values of q 0 tested, the smallest estimation error was obtained for a q 0 value equal to around 1.4 (MARE = 0.065; RMSRE = 0.094), for the S&MM dataset, and for a q 0 value equal to around 1.2 (MARE = 0.058; RMSRE = 0.093) for the EE&F dataset (see Table 4). Ultimately, h T was found to be the most accurate estimator (if one takes q 0 = 1.4), of those included in Table 4, for the S&MM dataset and the third best (if one takes q 0 = 1.2), for the EE&F dataset. Overall, the errors are not dramatically different in the range of q between 1.2 and 1.6, and then a value of q 0 = 1.5, also tested by Bletsas and Sahalos, may be a good compromise solution. The Pearson correlation between the actual value h of the h-index and its estimate varies slightly according to the selected value of q 0, but it is still very high: in particular, for q 0 = 1.5, we obtain for the S&MM dataset and for the EE&F dataset. Hence, overall, the method may lead to a very good fit, but it has two main drawbacks. First, the expression of is not given by any explicit formula. Second, this method continues to suffer from the problem of the conventional choice of an unknown parameter, in that we do not know a priori the value of the parameter q that will yield the “smallest” estimation error.

49In conclusion, basically, the same type of equation (see Eqs. 4, 10), describes the relationship between the h-index and other simple citation metrics. The Lambert-W formula for the h-index works well (also) for estimating the h-index for journals—especially in its improved version (13). As can be deduced from our empirical study, this still holds true for different scientific areas, for different time windows for publication and citation, for different types of “citable” documents, and for different approaches to the analysis of the citation process (“prospective” vs “retrospective”; Glänzel ). At the same time, the Glänzel–Schubert class of models seems to be much less robust and reliable as an estimator of the h-index, because its accuracy closely depends on a conventional choice of one or more unknown parameters. We may accordingly conclude that and are quite effective “universal” (in the sense that they are ready-to-use) informetric functions that work well for estimating the h-index, for a sufficiently wide range of values. Indeed, our empirical analysis, though preliminary, suggests that the fit is very good, at least for the datasets that we studied, and for values of its arguments that are not too large, namely, h < 40, T < 2000 and m < 20, which may be considered standard values for the cases of both and scientists journals within time-spans of 2–5 years.