Measurement of Digital Literacy Among Older Adults: Systematic Review

1Adopting digital technology is becoming imperative for all areas of service and business including health care. In the era of global aging, digital technology is viewed as a new opportunity to overcome various challenges associated with aging, such as reduced physical and cognitive function, multiple chronic conditions, and altered social networking. Consistent with this trend, the proportion of older populations using digital technology has increased exponentially, although this proportion is still smaller than that of younger generations. According to the latest Digital Economy Outlook Report from the Organization for Economic Cooperation and Development (OECD), 62.8% of 55–74-year-olds are now connected to the internet, as are 96.5% of 16–24-year-olds.

2Improving the inclusion and engagement of older adults in digital technology is becoming increasingly important for the promotion of their health and function. While numerous studies have measured the digital literacy of younger generations, few have examined the inclusion of older adults in the research and design of digital technologies. Moreover, existing measures of digital literacy for older adults are generally focused on acceptance models and barriers to adoption, which fail to consider heterogeneity in user ability. As emphasized by Mannheim et al, designs that focus heavily on barriers may be marginalizing older adults by assuming that they are less capable of utilizing digital technologies than their younger counterparts.

3For health care professionals, the rapid digitalization of social and health care services has various implications for providing older adults with improved access, knowledge, and behavior. Telehealth platforms are a solution for frailer, older adults to receive medical support remotely, while GPS can be used to mine personalized data to locate older patients and track or predict their needs. Internet use is associated with reduced likelihood of depression among the retired, and social networking sites represent an opportunity for older adults to reduce feelings of loneliness through online interactions with family and friends. The increasing number of Alzheimer’s disease forums on the microblogging system, Twitter, for example, shows how social networking systems serve as a platform for older individuals to share the latest health-related information with others.

4Quantifying the digital literacy of older adults is the first step to assist older adults to take advantage of this trend of digitalization in health care. However, when measuring digital literacy among older adults, measures must consider how basic competencies among one age cohort can be harder to achieve for another cohort with fewer information-and-communication-technology experiences and opportunities. In the case of older adults, other age-related factors including life transitions, personal health, attitudes, and economic incentives must also be considered during instrument research and design.

5To our knowledge, few systematic reviews to date have evaluated instruments of digital literacy for older adults in general, although 1 systematic review of digitally underserved populations attributed poor eHealth literacy to age, as well as language, educational attainment, residential area, and race. Furthermore, the compatibility between these instruments and older adults has not been measured according to a validated framework.

6Therefore, this systematic review aimed to (1) identify and critically appraise studies that involved the assessment of digital literacy among older adults and (2) evaluate how digital literacy instruments used in existing studies address the elements of age-appropriate digital literacy using the European Commission’s Digital Competence (DigComp) Framework. According to DigComp, digital literacy is defined in 5 areas: (1) information and data literacy, (2) communication and collaboration, (3) digital content creation, (4) safety, and (5) problem solving. For this review, we chose the DigComp over other frameworks, such as the International Computer and Information Literacy Study and OECD’s Program for the International Assessment of Adult Competencies because the DigComp Framework is the most generalizable across different regions and age groups.

7This systematic review was conducted by searching multiple electronic databases according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Electronic databases and search engines employed in the initial screening period included PubMed, CINAHL, Embase, and MeSH. The combination of search keywords for each database was summarized in a table (See Multimedia Appendix 1). Keywords were matched to database-specific indexing terms, and searches were not limited to a specific region or study design. However, we limited the year of study to those that were conducted after 2009 for a more recent conceptualization of digital literacy.

8The reference lists of identified studies were manually reviewed by a team of academics to prevent relevant studies from being excluded in our search for relevant articles. EndNote X9 was used for database management.

9We included studies that (1) were published in English, (2) targeted older adults, and (3) measured the use of a validated instrument to assess digital literacy. However, publications were excluded if older adults were not the study’s main target population. To elaborate, publications targeting the general population, for example, were excluded from our list of eligible articles as older adults were not the main target population examined.

10Exceptions to this rule were studies that compared older populations to younger populations with the aim of addressing the age-related digital divide, like the study by Schneider and colleagues comparing the digital literacy of “baby boomers” (50-65 years old) to that of millennials (18-35 years old).

11Using these eligibility criteria, 3 independent investigators (SO, MK, and JO) examined all studies reporting the use of a digital literacy instrument in the databases and search engines. All studies were screened according to their title and excluded if the main target population did not consist of older adults.

12Subsequently, abstracts were screened so that non-English studies and studies not assessing digital literacy through a validated instrument could be excluded from our investigation. During this process, any studies that were incapable of providing information on the required general characteristics were excluded.

13Last, full-text reviews were performed to ensure that all articles measured the digital literacy of older adults through validated instruments. In this process, investigator-developed questionnaires were included only if authors mentioned that they had been evaluated by experts for face validity. The instruments mentioned in each article were checked to ensure that they were accessible for our quality assessment. All processes were supervised by 2 independent reviewers (SC and JC), and any disagreement was resolved through discussions.

14Data on the general characteristics of the included studies included a summary of the year of publication, study design, region where the study was conducted, age of older adults studied, and main literacy instrument used. Regarding the region where the study was conducted, 2 studies were international collaborations, including 1 study between Italy and Sweden and another study between the United States, United Kingdom, and New Zealand. For these 2 studies, the first author’s region of study was used in our general characteristics summary.

15Three independent reviewers (SO, JC, and KK) assessed the quality of each included study using the Crowe Critical Appraisal Tool (CCAT). The CCAT is a validated quality assessment tool developed to rate research papers in systematic reviews based on a number of criteria relative to research design, variables and analysis, sampling methods, and data collection (Multimedia Appendix 2). Many systematic reviews targeting older adults have used this tool for quality appraisal.

16Instruments were also assessed to the DigComp’s definition of the 5 areas of digital literacy: (1) information and data literacy (browsing, searching, filtering data), (2) communication and collaboration (interacting, sharing, engaging in citizenship, collaborating), (3) digital content creation (developing, integrating, and re-elaborating digital content; copyright; licenses; programming), (4) safety (protecting devices, protecting personal data and privacy, protecting health and well-being), and (5) problem solving (solving technical problems, identifying needs and technological responses, creatively using digital technologies, identifying digital competence gaps).

17The PRISMA flow diagram in Figure 1 summarizes the search results and selection process of all studies included in our synthesis. Overall, the number of records identified in our database was 1561 (PubMed: 931; CINAHL: 147; Embase: 483). The number of additional records identified through other sources was 435 (MeSH: 434, hand search: 1). Of these records, 1412 remained after duplicates were electronically removed. An additional 1026 articles were removed after title screening, and 308 articles were removed after abstract screening.

18Of the 78 articles assessed for eligibility, 50 were excluded for the following reasons: (1) no report of an instrument for digital literacy despite the title or abstract of the paper alluding to measures of digital literacy (n=23); (2) the instrument presented was not validated (n=13); (3) studies were mainly on population assessments and measured digital literacy only as part of a wider assessment of multiple factors (n=10); (3) instruments were not available in English or in a publicly accessible format (n=4); and (4) the study did not specifically target older adults (n=1). Ultimately, 27 articles were included in our review.

19Table 1 provides a general summary of the included studies. While publication years ranged from 2009 to 2020, most articles reviewed were conducted between 2015 and 2020. The majority (17/27, 63%) of included studies were cross-sectional, but 2 studies were pre- and post-test studies, 2 were randomized controlled trials (RCTs), 1 was longitudinal, and 1 was a mixed-method study with both surveys and focus group interviews. Most studies were conducted in the United States (18/27), but some studies were also conducted in Europe (Germany, 3/27; Italy, 1/27; Sweden, 1/27). Studies mostly defined older adults as aged ≥50 years (10/27) or ≥60 years (8/27).

20Table 2 presents the detailed characteristics of all 27 included studies. Overall, the eHealth Literacy Scale (eHEALS) was the most frequently used instrument to measure digital literacy among older adults (16/27, 59%). The Unified Theory of Acceptance and Usage of Technology (UTAUT) was also used by 2 studies from Germany and 1 study from Bangladesh. Loyd-Gressard’s Computer Attitude Scale (CAS) was used in 2 studies that focused heavily on computer anxiety and confidence. There was not wide variation in the quality of studies assessed via the CCAT, with scores ranging from 34 (34/40, 85%) to 40 (40/40, 100%) of a total of 40 points.

21As seen in Table 3, all instruments were analyzed for quality assessment to assess which DigComp elements of digital literacy were met. Studies mostly satisfied 1 or 2 aspects of the information and data literacy criteria, but the Mobile Device Proficiency Questionnaire (MDPQ) satisfied all 5 elements, including those related to safety and data creation.

22In this systematic review, we highlighted the importance of digital literacy among older adults and provided a comprehensive overview of the instruments that are being employed to measure their digital literacy. We also illustrated the various strengths and limitations of each instrument, relative to age-appropriateness and suitability for older adults, in accordance with the components of a validated, digital competency framework. Our review is timely because, to the best of our knowledge, few systematic reviews to date have evaluated measurements of digital literacy for older adults specifically.

23In the digital era, providing education for patients regarding management of their physical or mental illness or injury, explaining posttreatment home care needs, and managing their diet, nutrition, and exercise are all duties that are beginning to be “digitalized”. Moreover, digital technologies are providing practitioners with more effective and user-centered ways to educate, inform, and treat older patients. For example, in a systematic review of “virtual visits” in home care for older patients, both service users and providers found online visits to be more flexible, easy to arrange, and personal than offline visits. In another study of an internet-based videoconferencing system for frail elderly people in Nordic countries, telehealth was associated with reduced loneliness among 88% of users, while simultaneously reducing the odds of catching a cold during winter months due to leaving the house.

24Overall, we discovered that while the eHEALS is most frequently used to measure digital literacy among older adults, the MDPQ may be more appropriate for measuring the literacy of older adults. Unlike the eHEALS, the MDPQ attempts to measure older adults’ digital content creation capacity (developing, integrating, and re-elaborating digital content; copyright; licenses; programming), which according to the European Commission, can give valuable information regarding an individual’s ability to add value to new media for self-expression and knowledge creation.

25Also, the MDPQ contains numerous items related to data protection and privacy such as “passwords can be created to block/unblock a mobile device” or “search history and temporary files can be deleted” despite the fact that security was the least measured element of the DigComp Framework among the instruments in our study. Only the CAS, eHEALS, and MDPQ provide items related to data protection and privacy, which is concerning given that older adults comprise a significant proportion of the target population for internet scams or email attacks.

26In our review of 27 selected articles, more than half (16/27, 59%) used the eHEALS to measure the digital literacy of older adults. Several reasons can be speculated; this instrument is short (8 items), and the questions are simple to understand (eg, “I know how to use the Internet to answer my health questions”). Scholars claim that it is easy to administer to older adults. It should be noted that because of its simplicity, there has been some debate regarding the validity of the eHEALS. As described by Jordan and colleagues, the eHEALS has a “lack of an explicit definition of the concept that many health literacy indices were developed to measure limit......its ability to make fully informed judgments......about a person’s ability to seek, understand, and use health information.”

27Studies focusing on similar research aims also employ similar instruments. For example, the CAS was used in 2 studies that focused on computer anxiety and confidence. In the existing body of literature, the CAS has often been used for studies targeting individuals in highly stressful environments such as business graduate students, psychiatric inpatients, and students studying at a 2-year technical college experiencing “technostress”. As explained by Kelley and Charness, older adults “commit more errors in post-training evaluations” than the general population, which may result in greater stress and anxiety. This may demonstrate the suitability of the CAS for older adult populations.

28Regarding the overall quality of the included studies evaluated using the CCAT, some variation existed among the studies reviewed. Studies that were cross-sectional or lacked acquisition of written informed consent and used alternate approaches, such as telephone or self-reported, web-based or email surveys, scored poorly in the “design” and “ethical matters” category. Studies also lost marks if there was no flow diagram, there was no mention of design methods in the title of their manuscript, or they had biased sampling methods (convenience sampling, pertaining only to 1 or 2 ethnic groups).

29Contrastingly, 2 RCTs in our review received a score of 100% on the CCAT, as they had excellent preliminaries, introductions, study design, sampling methods, data collection methods, ethical matters, results, and discussions. These studies employed performance-based measures like the Attitudes Toward Computer/Internet Questionnaire (ATC-IQ; semistructured interview) and UTAUT model (face-to-face interview), which are more reliable data collection methods than self-administered questionnaires. Performance-based measures like these may be suitable for studies targeting older adults, but it should be noted that clinical environments and personal fitness can greatly influence outcomes, especially if environments contain learners of mixed ability, rapid progression, and the possibility for embarrassment or discomfort. Positive clinical settings are associated with improved performance, as observed in 1 of the RCTs in our review, where “a combination of patience, perseverance, and peer-to-peer or instructor encouragement, whether with words or a pat on the shoulder” were successful in reducing older adults’ stress and anxiety during digital learning.

30As aforementioned, for older adults, it is important that the research and design of digital technologies encompass the heterogeneity of their capacity. While we believe that instructions should be “clear and understandable” to study participants, we also believe that literacy elements that are generalizable to the rest of the population (relative to communication, safety, problem solving, and competence) should be measured for this population as well. As described by Hänninen et al, the digital capacity of older adults lies on a continuum or spectrum and can range from actively independent to limited.

31Previous studies recommend that, instead of employing the full MDPQ or technology acceptance model (TAM), the shorter, 16-question version, or senior version of the TAM (Senior Technology Acceptance & Adoption Model), may be more appropriate for relatively older and frailer populations. User-centeredness in instrument development and measurement is crucial for this population, as the functional status of older adults varies immensely. Furthermore, scales and scoring methods are encouraged to be as inclusive as possible, so that they encompass the diversity in functionality that exists among study subjects.

32Ultimately, many limitations exist in our review. First, it is important to mention that the association between age and digital capacity is controversial among certain scholars who argue that age-based divisions are too simplistic and unclear to explain the digital divide. In the Netherlands, for example, “digital natives” do not appear to exist, and other factors like life stages and socialization are considered to be more relevant proxies of digital literacy than age. Also, in a German study, perceptions of threat due to technologization were perceived as the main predictors of digital capacity, rather than age itself. Older adults with lower perceptions of threat could be digitally fluent, just as younger adults with higher perceptions of threat could be digitally illiterate. Future questionnaires should consider measuring this factor in depth and the possible interaction that it has with age in predicting digital capacity outcomes.

33Likewise, digital literacy is a process-oriented skill, and measuring it in isolation may be inaccurate for quantifying an individual’s skillset. In the Lily Model, Norman and Skinner posit that there are 6 core skills of literacy: traditional, media, information, computer, scientific, and health. Not only are these skills heavily interconnected with one another but also only an in-depth analysis of all 6 can fully contextualize an individual’s personal, social, and environmental contexts. For example, computer literacy may be heavily influenced by an individual’s ability to understand and read passages (traditional literacy) as well as their ability to find information on a topic (information literacy) and understand certain scientific terms (science literacy). Because these literacy types are interconnected, only an in-depth analysis of all 6 may accurately measure an individual’s knowledge.

34Also, as observed in our review, many of the investigated instruments, including the Attitudes Toward Technologies Questionnaire, TAM, ATC-IQ, APOI, and CAS, measured attitudes or perceptions toward technology rather than digital aptitude itself. While studies on attitude are important, the lack of measures examining older adults’ abilities to use information and communications technology was an unexpected limitation of the reviews studied.

35Last, even though previous studies have argued that the DigComp Framework is one of the broadest and most generalizable frameworks for assessing digital literacy measures, it is undeniable that certain types of survey error are more likely to occur among older populations relative to memory loss, health problems, sensory and cognitive impairments, and personal or motivational factors that influence their ability to participate in an investigation. The author and editors of this framework specifically mention in their proposal that, because they adopted a “general” rather than “individual” approach, their framework should be considered only as a starting point in interpreting digital competence among different age groups.

36In conclusion, more studies are required so that the measurement of digital literacy among older adults can become more elaborate and specific. Digital literacy evidently has strong associations with the utility of information and communications technologies that promote physical and mental well-being among older adults. Further assessments and studies of digital literacy among older adults that overcome the limitations of existing research and measurement designs would allow for better allocation of support and resources to address the diverse health care needs of this growing but vulnerable population.