Skip to content
naulon

No retraction, correction or concern found when last checked on Oct 20, 2026.

Addressing health literacy in patient decision aids

Kirsten J McCaffery, Margaret Holmes-Rovner, Sian K Smith and 6 more

BMC Medical Informatics and Decision Making · 2013 · Systematic review

doi:10.1186/1472-6947-13-s2-s10

Abstract

Background. Effective use of a patient decision aid (PtDA) can be affected by the user’s health literacy and the PtDA’s characteristics. Systematic reviews of the relevant literature can guide PtDA developers to attend to the health literacy needs of patients. The reviews reported here aimed to assess: 1. a) the effects of health literacy / numeracy on selected decision-making outcomes, and b) the effects of interventions designed to mitigate the influence of lower health literacy on decision-making outcomes, and 2. the extent to which existing PtDAs a) account for health literacy, and b) are tested in lower health literacy populations.

Methods. We reviewed literature for evidence relevant to these two aims. When high-quality systematic reviews existed, we summarized their evidence. When reviews were unavailable, we conducted our own systematic reviews.

Results. Aim 1: In an existing systematic review of PtDA trials, lower health literacy was associated with lower patient health knowledge (14 of 16 eligible studies). Fourteen studies reported practical design strategies to improve knowledge for lower health literacy patients. In our own systematic review, no studies reported on values clarity per se, but in 2 lower health literacy was related to higher decisional uncertainty and regret. Lower health literacy was associated with less desire for involvement in 3 studies, less question-asking in 2, and less patient-centered communication in 4 studies; its effects on other measures of patient involvement were mixed. Only one study assessed the effects of a health literacy intervention on outcomes; it showed that using video to improve the salience of health states reduced decisional uncertainty. Aim 2: In our review of 97 trials, only 3 PtDAs overtly addressed the needs of lower health literacy users. In 90% of trials, user health literacy and readability of the PtDA were not reported. However, increases in knowledge and informed choice were reported in those studies in which health literacy needs were addressed.

Conclusion. Lower health literacy affects key decision-making outcomes, but few existing PtDAs have addressed the needs of lower health literacy users. The specific effects of PtDAs designed to mitigate the influence of low health literacy are unknown. More attention to the needs of patients with lower health literacy is indicated, to ensure that PtDAs are appropriate for lower as well as higher health literacy patients.

Full text

From PubMed Central, U.S. National Library of Medicine, as current on Oct 20, 2026. Paragraph numbers are ours, so an answer can point at one.

Background

A person’s health literacy status affects their ability to utilise health information and services, and their health outcomes. It is therefore an important potential consideration in patient decision aid (PtDA) development and shared decision making. There has been no systematic examination of the effects of health literacy on outcomes relevant to PtDA development or of interventions that might mitigate potential adverse effects of low health literacy in the decision-making context.

Health literacy can be conceptualized in different ways. A simple and common definition is “the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions”. However, broader definitions are gaining popularity and encompass a wider range of cognitive and social skills that enable people to feel empowered to take control and improve their health. Nutbeam’s model of health literacy delineates three levels:

1. Functional health literacy – basic reading comprehension and writing skills to understand health information/messages, together with knowledge of health conditions, services, and systems.

2. Communicative/interactive health literacy – higher level communicative and social skills required to extract and discuss information with others.

3. Critical health literacy skills – advanced literacy, cognitive, and social skills to analyze information and make informed decisions.

We propose that these levels correspond with the skills required to effectively use PtDAs and to engage in shared decision making activities: understanding health information, clarifying personal values, and communicating with health care providers.

A high proportion of adults have limited health literacy. In the U.S., estimates suggest that 36% (80 million adults) have “basic” or “below basic” health literacy. Although methods of assessment vary, the picture is very similar in other developed countries. The high levels of limited health literacy throughout the world provide a sharp reminder of the need to provide health information in a form that is appropriate for the literacy and numeracy levels of the majority of adults for whom it is prepared.

A person’s ability to effectively use a PtDA will be determined by both their health literacy skills and the quality and suitability of the PtDA. Designers of PtDAs need to ensure that their tools can be accessed and understood by adults across the health literacy spectrum, including those with lower health literacy, and can support decisions that are both informed and behaviorally implemented.

Theoretical justification for addressing health literacy in patient decision aids

We propose that Nutbeam’s multi-level model of health literacy provides a useful framework to help PtDA developers to address the needs of lower health literacy patients. Each of the levels of health literacy described in the model provides the skills required by patients to effectively use PtDAs and to engage in shared decision making. Individuals can obtain differential benefit from PtDAs depending on the relative differences in their health literacy. Those with the most basic functional health literacy will be able to obtain information from a PtDA and to use this as the basis for a more informed decision about their health. Those with higher level skills will be able to make greater use of available information to consider critically what is best for them, and to interact and communicate successfully with their health care provider in making a shared decision.

Attending to each of the levels of health literacy in PtDA design is important in order to meet the needs of patients with lower health literacy. This means developing PtDA materials to support consumers not only to read and understand evidence-based information (i.e., functional health literacy), but also to have the skills and confidence to communicate with health professionals and negotiate decisions (interactive/ communicative health literacy), and to clarify their values and think critically to make an informed decision (critical health literacy).

Purpose and objectives

This paper systematically reviews the literature to guide PtDA developers as they attend to the health literacy needs of patients. We systematically reviewed empirical evidence relevant to health literacy and PtDAs with two principal aims:

I. To assess a) the effects of health literacy / numeracy on selected decision-making outcomes, and b) the effects of interventions designed to mitigate the influence of lower health literacy on decision-making outcomes, and

II. To assess the extent to which existing PtDAs a) account for health literacy, and b) are tested in lower health literacy populations.

REVIEW I. Health literacy / health numeracy: the effects on decision-making outcomes and the effects of health literacy interventions

To address our first aim, we asked two review questions: 1) What is known about how health literacy affects decision-making outcomes, including knowledge, values clarity, and patient involvement? and 2) What is known about health literacy interventions’ ability to mitigate the effects of lower health literacy on decision-making outcomes?

Review I: overview of methods

We searched two existing systematic reviews conducted for the U.S. Agency for Healthcare Research and Quality (AHRQ) in 2004 and 2011 on Health Literacy Interventions and Outcomes for answers to these questions, and summarized their results if they were available. When questions were not answered (e.g., for outcomes and health literacy interventions related to values clarity or to patient involvement in decision making and communication (see below)), we performed our own systematic review and data synthesis.

Review I: data sources and selection

To identify relevant articles, we searched articles included in the 2004 and 2011 systematic reviews performed for the AHRQ. These reviews had examined 1) the effects of health literacy (including numeracy) on health outcomes, and 2) the effects of interventions designed to mitigate the effects of low health literacy on health outcomes. These reviews had focused on English-language articles with an objective measure of health literacy that were published from 1966 to February 2011 and indexed in MEDLINE®, CINAHL, PsycINFO, ERIC, or the Cochrane Library database, and used the keywords literacy, numeracy, or terms or phrases related to measures thereof.

For our review, we also searched the titles and abstracts of articles that had been excluded from the 2011 AHRQ review for the following reasons: 1) they didn’t use an objective measure of health literacy, or 2) they used an objective measure of health literacy, but reported only on outcomes that were not of interest to the AHRQ review (including outcomes related to clarifying values and participating in decision making). We did not search articles that had been excluded from the 2004 review because they pre-dated landmark work in health literacy and the first set of International Patient Decision Aid Standards (IPDAS) that called for values clarification and coaching as components of decision making.

To determine article inclusion, first one reviewer adhered to the inclusion criteria listed in Table 1, and excluded articles that were clearly unrelated to the questions guiding our review. Two reviewers then reviewed the remaining abstracts, and when necessary, reviewed full text articles to determine inclusion. Reviewers also reviewed the reference lists of included studies, prior narrative reviews on health literacy and decision making or communication, and the bibliographies of systematic reviews on question-asking and patient-centered care.

The inclusion criteria for this review mirrored the criteria used in the AHRQ reviews. However, our criteria were extended to include articles that included individual-level subjective assessments of health literacy, as well as outcomes related to clarifying values (e.g., decision uncertainty, decision regret, decision confidence, values clarity) and outcomes related to participating in decision making during the clinical encounter (e.g., patient activation, desire to participate, actual participation, communication quality). Our selection of these decision outcomes was guided by the general functional goals of PtDAs, which are: 1) To support users to understand health information relevant to their decision; 2) To support users to clarify their values; and 3) To support users to be actively involved in decision making and to communicate with others. Thus, these functional goals served as an overall conceptual structure for our review, as outlined in Table 2.

Review I: data extraction and quality assessment

After inclusion was determined, a single reviewer entered information about studies into evidence tables, and a second reviewer checked abstractions for accuracy and consistency in presentation. Two reviewers independently rated study quality (good, fair, poor), using quality forms developed for the AHRQ reviews. These forms assessed selection bias, measurement bias, confounding, and inadequate power. We excluded poor quality studies from our analysis and resolved disagreements about inclusion by consensus. Figure 1 is a flow diagram summarizing the full process of article exclusion / inclusion.

Review I: data synthesis

Given the diversity in study outcomes and measurements, we synthesized data qualitatively and not by meta-analysis. We divided studies into those addressing values clarity and those addressing patient involvement and communication.

Review I: results

Below, the results for the first review are structured according to three general functional goals of PtDAs. Detailed summary tables for this first review appear in Appendices 1 to 4 (see Additional files 1, 2, 3, 4).

PtDA Goal 1: To support users to understand health information relevant to their decision

The two AHRQ reviews addressed our questions related to health literacy / numeracy, knowledge outcomes, and health literacy interventions. Their findings are summarized below, grouped according to our two review questions.

Relationship between health literacy / numeracy and the outcomes of knowledge, comprehension, and risk perception

The 2004 AHRQ review found that low health literacy (measured predominantly by the Rapid Estimate of Adult Literacy in Medicine (REALM) or Test of Functional Health Literacy in Adults (TOFHLA)) was related to patient knowledge in 14 of 16 studies dating from 1980 through 2003. Of the two studies that did not show a relationship, one was underpowered. Investigators concluded that the relationship was so clear that additional examination of this relationship was not necessary during a 2011 update.

The 2011 AHRQ review assessed the effect of numeracy on knowledge and on accuracy of risk perception from 1966 to February 2011. In four studies with a quantitative measure of numeracy, investigators found mixed effects of numeracy on general and disease-specific knowledge. Additionally, in five studies with a quantitative measure of numeracy, investigators found mixed effects of numeracy on the accuracy of perception of absolute risk and treatment benefit (expressed in multiple formats, including relative risk reduction (RRR), absolute risk reduction (ARR), or number needed to treat (NNT), with notable variations by task and measured outcomes (e.g., disparities in accuracy were even greater for participants when stating exact benefit rather than comparing the benefit of two treatments).

Effect of health literacy interventions designed to help low literacy individuals’ knowledge, comprehension, or accuracy of risk perception

A paper published from the 2011 AHRQ review that included an updated search identified 38 studies published between 1966 and February 2011 that met the following criteria: they examined the effect of single or multiple literacy-directed strategies on knowledge or comprehension; they quantitatively assessed participants’ health literacy or numeracy; and they stratified analyses by health literacy level.

Fourteen of these studies (13 RCTs and 1 non-randomized controlled trial) specifically examined the effects of single strategies that might be useful in PtDA design. Among these 14 studies (which were reported in 12 articles), investigators found multiple discrete design features that improved comprehension for low health literacy individuals in at least one study. These are summarized in Table 3. Of the remaining 24 studies that reported on interventions using a mixture of literacy-directed strategies, only one reported on a health literacy intervention in a PtDA context. While this pre-post study of a prostate cancer PtDA reported improved knowledge among individuals in all health literacy subgroups (adequate literacy: + 1.27 points on a 10-point scale, adjusted p < 0.01; inadequate literacy +2.05 points, adjusted p < 0.01; p for interaction not reported), it did not describe its literacy-directed strategies in sufficient detail to allow recommendations to be derived.

PtDA Goal 2: To support users to clarify their values

In our own systematic review, we addressed the relationship between health literacy / numeracy and the outcome of values clarity, and the effect of health literacy interventions on values clarity.

Relationship between health literacy / numeracy and the outcome of values clarity

We found no studies that examined the relationship between health literacy and values clarity per se. Four cross-sectional studies investigated the relationships between health literacy and decision uncertainty, decisional regret, and decision confidence. In the two studies that performed adjusted analyses, lower subjective health literacy increased decision uncertainty, and lower objective health literacy increased decision regret. The effect of health literacy on decision confidence is less clear, with two unadjusted analyses showing mixed results. See Table 4.

Effect of health literacy interventions designed to help low literacy individuals clarify values

In the single intervention study in this group, Volandes et al. found that a PtDA using videos of patients to help elucidate the salience of various health states reduced decision uncertainty among all patients, with the greatest reduction found among patients with lower objective health literacy. See Table 4.

PtDA Goal 3: To support users to be actively involved in decision making and to communicate with others

In our own systematic review, we addressed the relationship between health literacy / numeracy and the outcomes of patient involvement and communication, and the effect of health literacy interventions on patient involvement and communication. See Table 5.

Relationship between health literacy / numeracy and the outcomes of patient involvement and communication

Thirteen studies reviewed the relationship between health literacy level and various aspects of the decision-making encounter, such as patient activation and desire for participation, question-asking, broader participation, and communication quality. Three studies, including two that adjusted for confounders, reported that adults with lower health literacy were less likely to want to be involved in decision making compared to those with higher literacy. A fourth showed that patient activation was associated with numeracy, but not health literacy in unadjusted correlational analysis.

Two papers addressed the relationship between literacy and question-asking. One found that both level and type of subjective health literacy (functional, communicative, or critical) influenced question-asking, with lower levels of communicative health literacy associated with less question-asking. The other found that lower objective health literacy patients were less likely to ask questions overall (although this just failed to reach statistical significance). Of the questions asked, lower health literacy patients asked significantly fewer medical and lifestyle questions and more clarification questions suggesting lack of understanding.

Two papers addressed patient involvement more broadly. One study reported patients with lower subjective “communicative” health literacy perceived they had less involvement in clinical consultations (although this just failed to achieve statistical significance, likely due to the small sample size). Arthur et al. reported a trend toward less “mutuality” (where both the physician and patient displayed a shared role in deciding the patient’s healthcare plan) among U.S. diabetes patients with limited objective health literacy.

Of the nine studies examining communication quality, four large high-quality studies with adjusted analyses reported lower patient-centered communication across several communication outcomes among those with lower objective health literacy (n = 2), lower subjective health literacy (n = 1), and lower subjective numeracy (n = 1). Communication was not specific to decision making, and the effect varied by outcome. A fifth study, in which the sample overlaps with one of the previous four, found that ratings of patient-centered communication varied by measurement of numeracy. Patients with low objective numeracy reported more favorable communication and those with low subjective numeracy reported less favorable communication. One unadjusted analysis found no difference between lower and higher health literacy patients’ perceptions that doctors facilitated their involvement in diabetes care.

Results of the remaining three studies varied by outcome. One study found that parents with lower objective health literacy reported more favorable interactions with their child’s healthcare provider. Another found poorer decision satisfaction about breast cancer treatment among those with lower subjective health literacy. Finally, one study found that those with lower critical health literacy were more likely to report they had received adequate information when more information was given (whereas those with higher critical literacy did not), offering a potential explanation for the contrasting findings reported above.

Effect of health literacy interventions designed to help low literacy individuals with communication

We identified no papers that tested health literacy interventions to assist the communication in the encounter, although one reported on the design of a cluster randomized trial to aid communication.

Review II. Attending to health literacy among lower literacy populations in PtDA trials

To address our second aim, we asked the question: what level of attention is paid to health literacy in trials of PtDAs to date?

Review II: overview of methods

To assess how health literacy has been addressed in PtDA trials, we systematically examined the PtDAs included in the published Cochrane Collaboration review of randomized controlled trials of PtDAs (including trials published to the end of 2009). We additionally updated their review with studies published to the end of 2010 (identified using the same search strategy).

Review II: data sources

To identify articles, we used a set of 102 references maintained by the Cochrane Decision Aid group that included reports of 97 separate decision aids. This list is an up-to-date listing complete through to the end of 2010. We note that 16/102 papers published in 2010 and reporting on 11 decision aids were identified with the assistance of the Cochrane Group and added to the current review. These have not been included in published reports of the main Cochrane Collaboration’s systematic review of decision aids.

Review II: data selection

After discussion among the 3 members of the review team, a set of review criteria was developed to indicate reading level accessibility to lower literacy audience and attention to literacy in the research design. The evaluative criteria for this review were based on those published in the 2006 IPDAS for literacy standards in PtDAs (see Table 6), with five new review criteria added to assess the level of attention paid to health literacy in the trials design (items 4-8 in Table 6).

Each study was graded using these eight review criteria. Criteria were coded as present, absent, or unknown. Lack of indication of meeting any of the original IPDAS literacy criteria and lack of mention of health literacy at all were coded as “absent”. If articles made a global reference to the IPDAS literacy criteria, but made no specific mention of the particular IPDAS literacy criteria addressed, they were scored as “unknown”. See Table 7 for the working definitions of our full set of eight criteria and an explanation of how coding was performed. Additional detail about Review II’s criteria is available in Additional file 5 Appendix 5.

We found that the original IPDAS literacy criterion #1 (“decision aid written at a level that can be understood by a majority of the sample”) to be very hard to determine. The intent of the criterion was that the PtDA developers should state the literacy requirements of the population for whom the PtDA was intended. The criterion was meant to encourage matching of population literacy requirements to the tool developed. However, in practice, the population literacy requirements were never stated, though sometimes the literacy level of the specific study sample was evaluated. To credit the attention paid by some investigators to the literacy levels of the sample, without having to attribute attention to population literacy requirements, we modified the wording of this criterion to code if literacy requirements of the subjects of the research were stated.

To test the functionality of our set of criteria, a 13% random sample was drawn from the pre-2010 set of 85 PtDAs, and assigned to 3 reviewers in groups of two in a blinded manner. One of us compared ratings and found that there was disagreement in 5.6% of the ratings. There were no obvious differences among the 3 reviewers in accuracy. We then discussed the differences and better specified the definitions.

Review II: results

The total number of PtDA trials in the latest Cochrane update was 86. Eleven more trials were identified in the updated search through 2010, so that the total set of PtDA trials evaluated was 97. Table 8 lists the percentages and absolute numbers of individual studies that were evaluated according to our set of literacy criteria.

Five of the 97 RCTs reported the PtDA reading level as grade 8 or below. Two trial reports stated that IPDAS literacy criteria were followed, but made no reference to which specific criteria. Many PtDAs used media other than pure text, including interactive video and audio tape, although it was not clear whether these modalities were used specifically to address issues of literacy. There were eight PtDAs available on the web, though in many the presence or absence of content other than straight text could not be determined.

Seven studies recruited patients presumed to have lower health literacy (based on their educational status) and reported conclusions about health literacy. Three randomized trials compared lower health literacy or education groups to higher health literacy groups. Two Australian studies developed PtDAs for bowel cancer screening, one specifically designed for a lower health literacy audience. One U.S. study additionally addressed prostate cancer screening through a comparison “edutainment” PtDA and a paper PtDA. All three of these PtDAs appear to have adhered to the IPDAS literacy criteria and included quantitative outcome data and explained technical terms. One PtDA was paper-based; the remaining two used either audio or multi-media.

Two of three studies showed that PtDAs increased knowledge in samples presumed to have lower health literacy based on their educational status. Trevena et al. found a significant increase in knowledge in both lower- and highly-educated participants in the PtDA arm. Smith et al. reported a 38% increase in knowledge and a 22% increase in informed choice (adequate knowledge and consistent attitudes and behavior) in lower education adults who received the literacy-sensitive PtDA compared with those who received standard information. The third study—on prostate cancer screening—found no difference in knowledge between the “edutainment” and an audio-booklet presentation containing the same information. However, the entertainment-based PtDA was associated with lower decisional conflict and greater self-advocacy when compared to the audio booklet among “low health literacy” patients. No differences between were observed for “high health literacy” patients.

Discussion

Overall, our reviews suggest that patients with lower health literacy may be less able to use PtDAs effectively and to engage in shared decision making unless special attention has been paid to low health literacy in the PtDA development process. The reviews also indicate that patients with lower health literacy may be at greater need of support in decision making, given their higher levels of uncertainty and decisional regret, less involvement, and less patient-centered care. These findings suggest that attention to the needs of lower health literacy patients is required in PtDAs. Our reviews of the PtDA literature shows that health literacy has rarely been considered in the literature to date. However, in the small number of studies where the literacy needs of patients have been attended to, the results are encouraging.

Our reviews indicate that developers need to ensure they attend not only to issues of comprehension (functional health literacy), but also to values clarification and to the processes involved in the decision-making encounter itself (communicative and critical health literacy). If PtDAs are to be helpful in structuring and guiding decision making, they must address health literacy issues related to all of these processes. Because current evidence best supports improving the design and content of PtDAs—as summarized in Table 3—this should be a priority and will facilitate comprehension among lower health literacy patients.

The use of these strategies is recommended for the development of PtDAs for lower health literacy patients. However, intervention research relating to values clarification and the decision-making encounter are extremely limited, with almost no intervention studies available to guide PtDA developers. More research here is especially needed to understand potential differential effects of interventions by literacy level.

With regards to patient involvement, we note the consistent observation that patients with lower health literacy desire less involvement in decision making. We suggest that this may in part be a consequence of a lack of awareness that they can be involved and a lack of confidence in sharing the decision process with health care providers. In patients with higher health literacy, desire for involvement has been found to increase when patients are shown the PtDA tools that are available. Entwistle and colleagues emphasize the complexity of patients’ desire and capacity for involvement in health decisions and suggest that using a trusted source, such as a health care provider, to make decisions (i.e., “intellectual outsourcing”) can be an autonomy-enhancing strategy for some patients, including patients who do not feel they have the capacity to make a shared decision. Entwistle highlights relational approaches to understanding health decision making that offer broader understandings of how patient autonomy in health decision making and in shared decision making can be achieved. Here, more research is needed to understanding the meaning and implications of involvement with lower literacy groups.

From our reviews of the PtDA literature, only 3 out of 97 PtDA trials included adults with lower education or health literacy, or used PtDAs designed explicitly to address lower health literacy audiences. In 90% or more of the trials, the reading needs of the participants and the reading level of the PtDAs were not reported. A clear bottom line is that PtDAs are rarely developed with lower literacy populations in mind, despite the observations that in many developed countries only half the population has more than basic reading skills, and that health information developed for adults at a grade 8 reading level is widely accepted by more educated as well as less educated users. However, in PtDA studies in which education and health literacy were addressed, results across literacy groups are positive, with increases in knowledge and in informed choice reported.

In addition to the principles identified in these reviews, others’ reviews suggest strategies that may be useful when applied to PtDA development. For instance, we believe that particular principles based on the broader health literacy literature can be tentatively applied to the design, development, testing and implementation of PtDAs for lower health literacy patients. See Table 9.

Until further evidence is available, PtDA developers may also turn to best practices materials for development and testing that have been outlined by literacy experts and several national organizations (see, for example,).

Principles for PtDA development and areas for further research

The field must continue to push ahead with new research among lower health literacy populations. There are important deficits in the PtDA literature; much work is still needed to develop and test strategies to help adults with lower health literacy to gain key literacy skills, to engage in the values clarification process, and to be involved in doctor-patient communication. We propose that the most useful information in the field will result if PtDA developers do the following:

1. Use a measurable strategy to ensure that the language of the PtDA is written at a level that is understood by the majority of the target audience (e.g., Flesch-Kincaid, Simple Measure Of Gobbledegook (SMOG), Fry Readability Formula (or Fry Readability Graph), or other accepted approaches; see http://www.nlm.nih.gov/medlineplus/etr.html, Suitability Assessment of Materials (SAM) or Systemic Functional Linguistics.

2. State how the PtDA accommodated health literacy or numeracy requirements identified in the population and whether good health literacy principles were followed [e.g].

3. Recruit, where possible, adequate numbers of low health literacy / numeracy individuals to evaluate effectiveness in this population. If full inclusion is not possible, report results of pilot studies among lower health literacy patients.

4. In conjunction with point 3, assess literacy / health literacy and/or numeracy levels in study samples by directly measuring functional health literacy and/or numeracy among a representative sample of the target sample of patients, using a recognized measure such as TOFHLA, REALM, or Newest Vital Signs (NVS).

5. Assess outcomes that have been shown to be or are postulated to be of specific concern for lower literacy patients.

Important research gaps in the field include a) the role of health literacy in the process of values clarification, b) the need to identify the characteristics of PtDAs that are universally acceptable and helpful, and c) the need to identify those characteristics that are particularly effective at enhancing shared decision making among people with lower health literacy/numeracy. Ultimately, this future research will contribute in important ways towards understanding the effects of PtDAs on health inequalities and towards ensuring that lower literacy/numeracy groups are not disadvantaged.

Limitations

There are a number of limitations to our reviews of the decision-making literature. The decision for inclusion/exclusion was based on only a single assessment of each article. Although the articles closest in subject to our reviews’ purposes were assessed by two reviewers, it is possible that some relevant studies were missed. Other limitations include the inherent potential for publication bias, the diversity of measures used for similar outcomes, and the small number of physicians (despite adequate numbers of patients) that were included in studies of health literacy and communication. Indeed, many of the reviewed studies had small samples and larger studies are needed in future research. We also acknowledge there may be confounding and lack of control of relevant variables in some of the papers included in our reviews; however, we have presented the quality rating of our reviewed studies and indicated those with adjusted analyses. Although not a specific limitation of our reviews, one additional issue is that we excluded studies stratifying analyses by education. We made this choice because our search was not designed to pick up education as a proxy for literacy and we likely would have missed many relevant studies. It should be noted, however, that important information may be learned from such studies. Furthermore, this field is growing quickly and we are already aware of new studies published since the study period. While these studies support the conclusions in our reviews, it is possible that other new studies would provide new or different insights. Future updates will be important.

In our reviews of the PtDA literature, health literacy was not directly measured in any study. Proxy measures were used in three studies that reported having recruited individuals with unspecified deficits in education and health literacy. A limitation of our reviews is that we have not included the background articles that describe the development of PtDAs tested in the trials. Adding these may provide greater insight into how literacy was addressed. It does not, however, address the lack of attention to literacy in the trials and in identification of research questions. A related limitation is that we included only RCTs. Other studies of PtDAs that used different research designs may have included PtDAs that addressed low health literacy. We also note that there is now mixed evidence regarding the benefit of alternate media on improving outcomes for lower literacy patients, and so review criteria may need to be revised for future reviews.

Conclusion

Because lower health literacy groups have lower levels of knowledge and involvement in health care, and some of the poorest health outcomes, they may be considered a priority group for better support in health decision making. Yet little attention has been given to this group in the PtDA literature. To enable more equitable access to PtDA resources and shared decision making, developers of PtDAs need to ensure that tools are accessible to lower as well as higher health literacy patients, and that lower health literacy groups are better equipped and supported to utilize PtDAs. Evidence is available to improve written information for lower health literacy patients; this evidence needs to be used more widely in PtDA development. However, there is little evidence to guide values clarification and increasing patient involvement in the consultation for this group of patients. More research is needed in these domains, in particular, to facilitate shared decision making among patients from varying health literacy backgrounds.

List of abbreviations used

NVS: Newest Vital Signs; PtDA: patient decision aid; RCTs: randomized controlled trials; REALM: Rapid Estimate of Adult Literacy in Medicine; SAM: Suitability Assessment of Materials; SMOG: Simple Measure Of Gobbledegook; TOFHLA: Test of Functional Health Literacy in Adults

Competing interests

Kirsten McCaffery and Marla Clayman have received research funding from the Informed Medical Decisions Foundation, a not-for-profit (501 (c)3) private foundation (http://www.informedmedicaldecisions.org). The Foundation develops content for patient education programs. The Foundation has an arrangement with a for-profit company, Health Dialog, to co-produce these programs. The programs are used as part of the decision support and disease management services Health Dialog provides to consumers through health care organizations and employers.

All other authors have no competing interests to declare.

Authors’ contributions

KM and SLS drafted the manuscript with comments from MHR, DR, SKS and DN. All authors contributed to the drafting of the chapter on which the paper was based. SLS led the decision making review which involved MC, KM, SKS and MW. MHR and DR led the DA RCT review with input from KKB. DN, SKS and KM contributed to the theoretical framework for the paper. All authors read and approved the final manuscript.

References 88

  1. 1.

    Berkman ND Sheridan SL Donahue KE Halpern DJ Crotty K Low health literacy and health outcomes: an updated systematic reviewAnn Intern Med2011 1397107

    Not in our library10.7326/0003-4819-155-2-201107190-00005

  2. 2.

    McCaffery KJ Smith S Wolf MS The challenge of shared decision making among adults with low literacy: a framework for research and developmentMed Decis Making2010 133544

    Not in our library10.1177/0272989x09342279

  3. 3.

    Elwyn G O'Connor A Stacey D Volk R Edwards A Coulter A the International Patient Decision Aids Standards CDeveloping a quality criteria framework for patient decision aids: online international Delphi consensus processBMJ2006 13417

    Not in our library10.1136/bmj.38926.629329.ae

  4. 4.

    Institute of MedicineHealth literacy: a prescription to end confusion2004 Washington, DC: National Academies Press

    Not in our library

  5. 5.

    US Department of Health and Human ServicesHealthy people 2010: understanding and improving health2000 Washington DC

    Not in our library

  6. 6.

    Nutbeam D Health Promotion Glossary1998 World Health Organization. Geneva: WHO/HPR/98.1

    Not in our library

  7. 7.

    Nutbeam D Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st centuryHealth Promot Int2000 13259267http://heapro.oxfordjournals.org/content/15/3/259.full

    Not in our library10.1093/heapro/15.3.259

  8. 8.

    Nutbeam D Defining and measuring health literacy: what can we learn from literacy studies?Int J Public Health2009 13303305

    Not in our library10.1007/s00038-009-0050-x

  9. 9.

    Kutner M Greenberg E J B National Assessment of Adult Literacy (NAAL). A first look at the literacy of America's adults in the 21st centuryhttp://nces.ed.gov/naal

    Not in our library

  10. 10.

    OECDLearning a living: first results of the adult literacy and life skills surveyCatalogue no. 89-603-XWE: Statistics CanadaOttawa and Paris. Organisation for Economic Co-operation and Developmenthttp://www.statcan.gc.ca/

    Not in our library

  11. 11.

    DeWalt DA Berkman ND Sheridan S Lohr KN Pignone MP Literacy and health outcomes: a systematic review of the literatureJ Gen Intern Med2004 1312281239

    Not in our library10.1111/j.1525-1497.2004.40153.x

  12. 12.

    Pignone M DeWalt DA Sheridan S Berkman N Lohr KN Interventions to improve health outcomes for patients with low literacyJ Gen Intern Med2005 13185192

    Not in our library10.1111/j.1525-1497.2005.40208.x

  13. 13.

    Sheridan SL Halpern DJ Viera AJ Donahue K Berkman ND Krotty KA Interventions for individuals with low health literacy: a systematic reviewJ Health Commun2011 13Suppl 33054

    Not in our library10.1080/10810730.2011.604391

  14. 14.

    Williams MV Davis T Parker RM Weiss BD The role of health literacy in patient-physician communicationFam Med2002 13383389

    Not in our library

  15. 15.

    Kinnersley P Edwards A Hood K Ryan R Prout H Cadbury N MacBeth F Butow P Butler C Interventions before consultations to help patients address their information needs by encouraging question asking: systematic reviewBMJ2008 13a485

    Not in our library10.1136/bmj.a485

  16. 16.

    McCormack LA Treiman K Rupert D Williams-Piehota P Nadler E Arora NK Lawrence W Street RL Jr. Measuring patient-centered communication in cancer care: a literature review and the development of a systematic approachSoc Sci Med2011 1310851095

    Not in our library10.1016/j.socscimed.2011.01.020

  17. 17.
  18. 18.

    Bryant MD Schoenberg ED Johnson TV Goodman M Owen-Smith A Master VA Multimedia version of a standard medical questionnaire improves patient understanding across all literacy levelsJ Urol2009 1311201125

    Not in our library10.1016/j.juro.2009.05.027

  19. 19.

    Galesic M Garcia-Retamero R Gigerenzer G Using icon arrays to communicate medical risks: overcoming low numeracyHealth Psychol2009 13210216

    Not in our library10.1037/a0014474

  20. 20.

    Galesic M Gigerenzer G Straubinger N Natural frequencies help older adults and people with low numeracy to evaluate medical screening testsMed Decis Making2009 13368371

    Not in our library10.1177/0272989x08329463

  21. 21.

    Garcia-Retamero R Galesic M Communicating treatment risk reduction to people with low numeracy skills: a cross-cultural comparisonAm J Public Health2009 1321962202

    Not in our library10.2105/ajph.2009.160234

  22. 22.

    Greene J Peters E Mertz C Hibbard J Comprehension and choice of a consumer-directed health plan: an experimental studyAm J Manag Care2008 13369376

    Not in our library

  23. 23.

    Meade CD McKinney WP Barnas GP Educating patients with limited literacy skills: the effectiveness of printed and videotaped materials about colon cancerAm J Public Health1994 13119121

    Not in our library10.2105/ajph.84.1.119

  24. 24.

    Murphy PW Chesson AL Walker L Arnold CL Chesson LM Comparing the effectiveness of video and written material for improving knowledge among sleep disorders clinic patients with limited literacy skillsSouth Med J2000 13297304

    Not in our library

  25. 25.

    Peters E Dieckmann N Dixon A Hibbard JH Mertz CK Less is more in presenting quality information to consumersMed Care Res Rev2007 13169190

    Not in our library10.1177/10775587070640020301

  26. 26.

    Tait AR Voepel-Lewis T Zikmund-Fisher BJ Fagerlin A Presenting research risks and benefits to parents: does format matter?Anesth Analg2010 13718723

    Not in our library10.1213/ane.0b013e3181e8570a

  27. 27.

    Volandes AE Paasche-Orlow MK Barry MJ Gillick MR Minaker KL Chang Y Cook EF Abbo ED El-Jawahri A Mitchell SL Video decision support tool for advance care planning in dementia: randomised controlled trialBMJ2009 13b2159

    Not in our library10.1136/bmj.b2159

  28. 28.

    Wilson EAH Park DC Curtis LM Cameron KA Clayman ML Makoul G vom Eigen K Wolf MS Media and memory: the efficacy of video and print materials for promoting patient education about asthmaPatient Educ Couns2010 13393398

    Not in our library10.1016/j.pec.2010.07.011

  29. 29.

    Wright A Whitwell S Takeichi C Hankins M Marteau T The impact of numeracy on reactions to different graphic risk presentation formats: an experimental analogue studyBr J Health Psychol2009 13107125

    Not in our library10.1348/135910708x304432

  30. 30.

    Couper MP Singer E Levin CA Fowler FJ Jr. Fagerlin A Zikmund-Fisher BJ Use of the internet and ratings of information sources for medical decisions: results from the DECISIONS surveyMed Decis Making2010 135 Suppl106S114S

    Not in our library10.1177/0272989x10377661

  31. 31.

    Hawley ST Janz NK Hamilton A Griggs JJ Alderman AK Mujahid M Katz SJ Latina patient perspectives about informed treatment decision making for breast cancerPatient Educ Couns2008 13363370

    Not in our library10.1016/j.pec.2008.07.036

  32. 32.

    Sudore RL Schillinger D Knight SJ Fried TR Uncertainty about advance care planning treatment preferences among diverse older adultsJ Health Commun2010 13Suppl 2159171

    Not in our library10.1080/10810730.2010.499982

  33. 33.

    Martin RW Head AJ Rene J Swartz TJ Fiechtner JJ McIntosh BA Holmes-Rovner M Patient decision-making related to antirheumatic drugs in rheumatoid arthritis: the importance of patient trust of physicianJ Rheumatol2008 13618624

    Not in our library

  34. 34.

    Torres RY Marks R Relationships among health literacy, knowledge about hormone therapy, self-efficacy, and decision-making among postmenopausal healthJ Health Commun2009 134355

    Not in our library10.1080/10810730802592247

  35. 35.

    Volandes AE Barry MJ Chang YC Paasche-Orlow MK Improving decision making at the end of life with video imagesMed Decis Making2010 132934

    Not in our library10.1177/0272989x09341587

  36. 36.

    DeWalt DA Boone RS Pignone MP Literacy and its relationship with self-efficacy, trust, and participation in medical decision makingAm J Health Behav2007 13S27S35

    Not in our library10.5993/ajhb.31.s1.5

  37. 37.

    Lillie SE Brewer NT O'Neill SC Morrill EF Dees EC Carey LA Rimer BK Retention and use of breast cancer recurrence risk information from genomic tests: the role of health literacyCancer Epidemiol Biomarkers Prev2007 13249255

    Not in our library10.1158/1055-9965.epi-06-0525

  38. 38.

    Mancuso CA Rincon M Asthma patients' assessments of health care and medical decision making: the role of health literacyJ Asthma2006 134144

    Not in our library10.1080/02770900500447052

  39. 39.

    Hibbard JH Peters E Dixon A Tusler M Consumer competencies and the use of comparative quality information: it isn't just about literacyMed Care Res Rev2007 13379394

    Not in our library10.1177/1077558707301630

  40. 40.

    Ishikawa H Yano E Fujimori S Kinoshita M Yamanouchi T Yoshikawa M Yamazaki Y Teramoto T Patient health literacy and patient–physician information exchange during a visitFam Pract2009 13517523

    Not in our library10.1093/fampra/cmp060

  41. 41.

    Katz M Jacobson T Veledar E Kripalani S Patient literacy and question-asking behavior during the medical encounter: a mixed-methods analysisJ Gen Intern Med2007 13782786

    Not in our library10.1007/s11606-007-0184-6

  42. 42.

    Arthur SA Geiser HR Arriola KRJ Kripalani S Health literacy and control in the medical encounter: a mixed-methods analysisJ Natl Med Assoc2009 13677683

    Not in our library10.1016/s0027-9684(15)30976-7

  43. 43.

    Schillinger D Bindman A Wang F Stewart A Piette J Functional health literacy and the quality of physician-patient communication among diabetes patientsPatient Educ Couns2004 13315323

    Not in our library10.1016/s0738-3991(03)00107-1

  44. 44.

    Sudore RL Landefeld CS Perez-Stable EJ Bibbins-Domingo K Williams BA Schillinger D Unraveling the relationship between literacy, language proficiency, and patient-physician communicationPatient Educ Couns2009 13398402

    Not in our library10.1016/j.pec.2009.02.019

  45. 45.

    Wynia MK Osborn CY Health literacy and communication quality in health care organizationsJ Health Commun2010 13102115

    Not in our library10.1080/10810730.2010.499981

  46. 46.

    Smith SG Wolf MS von Wagner C Socioeconomic status, statistical confidence, and patient-provider communication: an analysis of the Health Information National Trends Survey (HINTS 2007)J Health Commun2010 13Suppl 3169185

    Not in our library10.1080/10810730.2010.522690

  47. 47.

    Ciampa PJ Osborn CY Peterson NB Rothman RL Patient numeracy, perceptions of provider communication, and colorectal cancer screening utilizationJ Health Commun2010 13Suppl 3157168

    Not in our library10.1080/10810730.2010.522699

  48. 48.

    Shone LP Conn KM Sanders L Halterman JS The role of parent health literacy among urban children with persistent asthmaPatient Educ Couns2009 13368375

    Not in our library10.1016/j.pec.2009.01.004

  49. 49.

    Price-Haywood EG Roth KG Shelby K Cooper LA Cancer risk communication with low health literacy patients: a continuing medical education programJ Gen Intern Med2010 13S126S129

    Not in our library10.1007/s11606-009-1211-6

  50. 50.

    Stacey D Bennett CL Barry MJ Col NF Eden KB Holmes-Rovner M Llewellyn-Thomas H Lyddiatt A Legare F Thomson R Decision aids for people facing health treatment or screening decisionsCochrane Database Syst Rev2011 10CD001431

    Not in our library10.1002/14651858.cd001431.pub3

  51. 51.

    Holmes-Rovner M Kroll J Rovner DR Schmitt N Rothert M Padonu G Talarczyk G Patient decision support intervention: increased consistency with decision analytic modelsMed Care1999 13270284

    Not in our library10.1097/00005650-199903000-00007

  52. 52.

    Oakley S Walley T A pilot study assessing the effectiveness of a decision aid on patient adherence with oral bisphosphonate medicationPharm J2006 13536538

    Not in our library

  53. 53.

    Rothert ML Holmes-Rovner M Rovner D Kroll J Breer L Talarczyk G Schmitt N Padonu G Wills C An educational intervention as decision support for menopausal womenRes Nurs Health1997 13377387

    Not in our library10.1002/(sici)1098-240x(199710)20:5<377::aid-nur2>3.0.co;2-l

  54. 54.

    Schapira MM Gilligan MA McAuliffe T Garmon G Carnes M Nattinger AB Decision-making at menopause: A randomized controlled trial of a computer-based hormone therapy decision-aidPatient Educ Couns2007 13100107

    Not in our library10.1016/j.pec.2007.02.007

  55. 55.

    Ratanawongsa N Zikmund-Fisher BJ Couper MP Van Hoewyk J Powe NR Race, ethnicity, and shared decision making for hyperlipidemia and hypertension treatment: the DECISIONS surveyMed Decis Making2010 135 Suppl65S76S

    Not in our library10.1177/0272989x10378699

  56. 56.

    Smith SK Trevena L Simpson JM Barratt A Nutbeam D McCaffery KJ A decision aid to support informed choices about bowel cancer screening among adults with low education: randomised controlled trialBMJ2010 13c5370

    Not in our library10.1136/bmj.c5370

  57. 57.

    van Peperstraten A Nelen W Grol R Zielhuis G Adang E Stalmeler P Hermens R Kremer J The effect of a multifaceted empowerment strategy on decision making about the number of embryos transferred in in vitro fertilisation: randomised controlled trialBMJ2010 13c2501

    Not in our library10.1136/bmj.c2501

  58. 58.
  59. 59.

    Fraenkel L Rabidou N Wittink D Fried T Improving informed decision-making for patients with knee painJ Rheumatol2007 1318941898

    Not in our library

  60. 60.

    Frosch DL Bhatnagar V Tally S Hamori CJ Kaplan RM Internet patient decision support: a randomized controlled trial comparing alternative approaches for men considering prostate cancer screeningArch Intern Med2008 13363369

    Not in our library10.1001/archinternmed.2007.111

  61. 61.

    Frosch DL Kaplan RM Felitti VJ A randomized controlled trial comparing internet and video to facilitate patient education for men considering the prostate specific antigen testJ Gen Intern Med2003 13781787

    Not in our library10.1046/j.1525-1497.2003.20911.x

  62. 62.

    Krist AH Woolf SH Johnson RE Kerns JW Patient education on prostate cancer screening and involvement in decision makingAnn Fam Med2007 13112119

    Not in our library10.1370/afm.623

  63. 63.

    Hoffman RM Lewis CL Pignone MP Couper MP Barry MJ Elmore JG Levin CA Van Hoewyk J Zikmund-Fisher BJ Decision-making processes for breast, colorectal, and prostate cancer screening: the DECISIONS surveyMed Decis Making2010 13Suppl 553S64S

    Not in our library10.1177/0272989x10378701

  64. 64.

    Mathieu E Barratt AL McGeechan K Davey HM Howard K Houssami N Helping women make choices about mammography screening: an online randomized trial of a decision aid for 40-year-old womenPatient Educ Couns2010 136372

    Not in our library10.1016/j.pec.2010.01.001

  65. 65.

    Ruffin MT Fetters MD Jimbo M Preference-based electronic decision aid to promote colorectal cancer screening: results of a randomized controlled trialPrev Med2007 13267273

    Not in our library10.1016/j.ypmed.2007.07.003

  66. 66.

    Vandemheen K O'Connor A Bell S Freitag A Bye P Jeanneret A Berthiaume Y Brown N Wilcox P Ryan G Randomized trial of a decision aid for patients with cystic fibrosis considering lung transplantationAm J Respir Crit Care Med2009 13761768

    Not in our library10.1164/rccm.200903-0421oc

  67. 67.

    Volk RJ Jibaja-Weiss ML Hawley ST Kneuper S Spann SJ Miles BJ Hyman DJ Entertainment education for prostate cancer screening: a randomized trial among primary care patients with low health literacyPatient Educ Couns2008 13482489

    Not in our library10.1016/j.pec.2008.07.033

  68. 68.

    Rovner DR Wills CE Bonham V Williams G Lillie J Kelly-Blake K Williams MV Holmes-Rovner M Decision aids for benign prostatic hyperplasia: applicability across race and educationMed Decis Making2004 13359366

    Not in our library10.1177/0272989x04267010

  69. 69.

    Solberg LI Asche SE Sepucha K Thygeson NM Madden JE Morrissey L Kraemer KK Anderson LH Informed choice assistance for women making uterine fibroid treatment decisions: a practical clinical trialMed Decis Making2010 13444452

    Not in our library10.1177/0272989x09353947

  70. 70.

    Street RL Voigt B Geyer C Manning T Swanson GP Increasing patient involvement in choosing treatment for early breast cancerCancer1995 1322752285

    Not in our library10.1002/1097-0142(19951201)76:11<2275::aid-cncr2820761115>3.0.co;2-s

  71. 71.

    Trevena LJ Irwig L Barratt A Randomized trial of a self-administered decision aid for colorectal cancer screeningJ Med Screen2008 137682

    Not in our library10.1258/jms.2008.007110

  72. 72.

    Smith SK Trevena L Nutbeam D Barratt A McCaffery KJ Information needs and preferences of low and high literacy consumers for decisions about colorectal cancer screening: utilizing a linguistic modelHealth Expect2008 13123136

    Not in our library10.1111/j.1369-7625.2008.00489.x

  73. 73.

    van Tol-Geerdink JJ Leer JW van Lin EN Schimmel EC Huizenga H van Daal WA Stalmeier PF Offering a treatment choice in the irradiation of prostate cancer leads to better informed and more active patients, without harm to well-beingInt J Radiat Oncol Biol Phys2008 13442448

    Not in our library10.1016/j.ijrobp.2007.06.058

  74. 74.

    Entwistle VA Carter SM Trevena L Flitcroft K Irwig L McCaffery K Salkeld G Communicating about screeningBMJ2008 13a1591

    Not in our library10.1136/bmj.a1591

  75. 75.

    Entwistle VA Carter SM Cribb A McCaffery K Supporting patient autonomy: the importance of clinician-patient relationshipsJ Gen Intern Med2010 13741745

    Not in our library10.1007/s11606-010-1292-2

  76. 76.

    Davis TC Parent comprehension of polio vaccine information pamphletsPediatrics1996 13804810

    Not in our library

  77. 77.

    Kleimann S Enlow B Is plain language appropriate for well-educated and politically important people? Results of research with congressional correspondenceClarity2003 13410

    Not in our library

  78. 78.

    Develop Materialshttp://www.cdc.gov/healthliteracy/DevelopMaterials/

    Not in our library

  79. 79.

    Health Literacy Universal Precautions Toolkit. AHRQ Publication No. 10-0046-EFhttp://www.ahrq.gov/qual/literacy/healthliteracytoolkit.pdf

    Not in our library

  80. 80.

    DeWalt D Callahan L Hawk V Broucksou K Hink A Rudd R Brach C Health Literacy Universal Precautions Toolkit. (Prepared by North Carolina Network Consortium, The Cecil G. Sheps Center for Health Services Research, The University of North Carolina at Chapel Hill, under Contract No. HHSA290200710014. AHRQ Publication No. 10-0046-EF)2010 Rockville, MD: Agency for Healthcare Research and Qualityhttp://www.cdc.gov/healthliteracy/pdf/Simply_Put.pdf

    Not in our library

  81. 81.

    Federal plain language guidelineshttp://www.plainlanguage.gov/howto/guidelines/bigdoc/fullbigdoc.pdf

    Not in our library

  82. 82.

    Home webpagehttp://www.cancer.gov/

    Not in our library

  83. 83.

    Health literacy studies: assessing and developing health materialshttp://www.hsph.harvard.edu/healthliteracy/practice/innovative-actions/index.html

    Not in our library

  84. 84.

    Doak CC Doak LG Root JH Teaching patients with low literacy skills1996 2Philadelphia: J. B. Lippincott

    Not in our library

  85. 85.

    Clerehan R Buchbinder R Moodie J A linguistic framework for assessing the quality of written patient information: its use in assessing methotrexate information for rheumatoid arthritisHealth Educ Res2005 13334344

    Not in our library10.1093/her/cyg123

  86. 86.

    Smith RA Cokkinides V Brawley OW Cancer screening in the United States, 2008: a review of current American Cancer Society guidelines and cancer screening issuesCA Cancer J Clin2008 13161179

    Not in our library10.3322/ca.2007.0017

  87. 87.

    Wilson EA Makoul G Bojarski EA Bailey SC Waite KR Rapp DN Baker DW Wolf MS Comparative analysis of print and multimedia health materials: a review of the literaturePatient Educ Couns2012 13714

    Not in our library10.1016/j.pec.2012.06.007

  88. 88.

    Manganello JA Clayman ML The association of understanding of medical statistics with health information seeking and health provider interaction in a national sample of young adultsJ Health Commun2011 13Suppl 3163176

    Not in our library10.1080/10810730.2011.604704

Cited by papers we hold 4

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.