Technology Acceptance in Mobile Health: Scoping Review of Definitions, Models, and Measurement

1Technology acceptance is a major challenge faced by designers of new technologies. In health care, patients are a vulnerable population, and their data are considered sensitive, especially in the case of stigmatized conditions such as those concerning mental health. As mobile technology becomes increasingly pervasive in health care, the expanding use of potentially invasive technologies such as sensing and machine learning is likely to lead to greater concerns among users, exacerbating existing problems with attrition, and willingness to use new technologies. In addition, health care providers’ perception of a technology is likely to affect treatment delivery, especially if it is not considered sufficiently acceptable.

2The last decade has seen an increasing number of studies addressing this issue in the mobile health (mHealth) domain. A recent systematic review by Wozney et al revealed that acceptability was the most frequently measured outcome in studies on mental health technologies for anxiety and depression. Designing for acceptance is not straightforward, as the user journey is complex and often nonlinear. Patients who decide to try an application will not necessarily use it in the long run, and similarly, someone who has stopped using a system might go back to it later. Conditions such as depression might also impact patients’ perception of their experience with technology and thus affect their acceptance. Different stages punctuate the users’ journey with technology where they, consciously or not, repeatedly make the decision to keep using or to abandon it. In addition, the extent to which users have appropriated and integrated technology into their lives may impact their decision to maintain use.

3An evolving terminology and range of definitions can be found in the literature discussing technology acceptance. Terms such as acceptability, acceptance, and adoption are often employed, sometimes interchangeably. For instance, Al-Youssef refers to acceptability as users’ willingness to use a system while citing the definition of acceptance given in Dillon and Morris. These terms are sometimes equated to other human-computer interaction (HCI) concepts such as user satisfaction. Yet, a part of the literature tries to differentiate the process of technology acceptance from existing concepts. For instance, Schade and Schlag describe acceptability as “a prospective judgement of measures to be introduced in the future,” which they detail as “the target group will not yet have experienced the new measures.” Close to this interpretation, Adell equates acceptance to “the degree to which an individual intends to use a system and, when available, incorporates the system in his/her [driving].”

4The Cambridge Dictionary defines acceptability as “the quality of being satisfactory and able to be agreed to or approved of” and acceptance as a “general agreement that something is satisfactory or right”. This suggests that acceptability is a quality of an object from the perspective of a stakeholder, whereas acceptance is more of a process relating to a stakeholder’s interaction with this object. In the context of HCI, it translates into a user-system dyad.

5In the same line, Proctor et al define acceptability as “the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory.”

6With the literature highlighting the importance of temporality in user experience research, some authors attempt to integrate a temporal dimension into the process of technology acceptance. In that respect, a group of authors supported the idea of a multistage process. Martin et al and Distler et al define technology acceptability as one's perception of a system before use. They distinguish acceptability from acceptance by equating acceptance to the users’ perception of the system after use. Terrade et al and Garces et al go a step further, stating that acceptance refers to the initial use of a system in controlled settings. Reerink-Boulanger introduces the continuum acceptability-acceptance- appropriation, describing acceptability as the subjective perception of the use of a system, acceptance as the first interactions with the system and appropriation as the use of the system by an individual in interaction with other individuals (translated from the original paper in French). Other definitions of appropriation include “a process of social construction in which the actions and thoughts of the user are shaped by the technology, while the meaning and effects of the technology are shaped through the users’ actions” and “the process through which technology users go beyond mere adoption to make technology their own and to embed it within their social, economic, and political practices”.

7Whereas a consistent framework seems to emerge from these definitions, another term, adoption, comes into the picture. Some authors such as Carroll et al equate the process of adoption with the entire user journey: “a multi-phase process starting with deciding to adopt (selecting, purchasing, or committing to use it) and then achieving persistent use.” In contrast, Karahanna et al describe “preadoption” and “postadoption (continued use)” stages and present adoption as an event allowing progress from one stage to the other. Similarly, Rogers defines adoption as the user’s decision to “make full use of an innovation as the best course of action available.”

8Another strand of work has attempted to identify the factors influencing technology acceptance, the most well-known of which is the technology acceptance model (TAM) of Davis. In the health domain, Kim and Park have proposed a health information technology acceptance model (HITAM). This model integrates the TAM by Davis, along with antecedents and health-related constructs (health status, health beliefs and concerns, behavioral beliefs, and perceived health threat). Other models, such as the pervasive technology acceptance model, integrate the influence of demographics and trust on user acceptance. Dou et al introduced the constructs relationship with doctor and resistance to change. Finally, Cheung et al introduced constructs related to privacy and consumer innovativeness.

9However, the temporal dimension is missing from these models. This raises the question of whether acceptance is motivated by the same factors and in the same manner, regardless of how long the system has been used. As stated in a review on user engagement, describing a concept as a process, rather than a discrete state, “enables the analysis of change over time”. In line with this, some authors propose models that incorporate temporality. Karapanos et al showed that, in their study, qualities that satisfied participants at initial use did not necessarily motivate prolonged use. Building on this, Karapanos et al explored factors influencing user experience over time; they proposed a temporal framework that identifies 4 stages: anticipation (formation of expectations before any experience of use), orientation (users’ initial experiences), incorporation (how the product becomes meaningful in the user’s life), and identification (how the product participates in users’ social interactions). Although the work addresses user experience rather than acceptance, the authors claim that different qualities contribute to user experience over time, and thus, time alters the way users experience technology. As the extent to which users accept a technology is undeniably linked to their usage experience—or lack of—it is relevant to ask how user acceptance evolves over time and how the influencing factors vary accordingly.

10Further exploring the temporal dimension of acceptance, Greenhalgh et al proposed a framework for nonadoption, abandonment, scale-up, spread, and sustainability (NASSS). Directed at health care technologies, the NASSS framework describes different domains influencing technology adoption. It adds a temporal dimension targeting technology’s “continuous embedding and adaptation over time.” The framework is designed to be used at different points in time (at early design, after deployment, and after abandonment), which distinguishes it from the other models.

11Both definitions and more detailed models of acceptance may find expression in the form of approaches to measurement. Examples of acceptance studies at different stages of the technology lifecycle can be found in the literature, in some cases employing measurement tools based on theoretical models. This includes gathering of qualitative data via focus groups and interviews based on the TAM, a survey based on the unified theory of acceptance and use of technology, and interviews based on the fast form TAM.

12Proctor et al defined eight outcomes for the measurement of implementation of health interventions, two of which were acceptability and adoption. In a recent paper, Hermes et al proposed to adapt the characterization of these outcomes for behavioral intervention technologies. The authors explicitly link the adoption outcome to the use (or intention to use) of a system. In addition, they state that “usability clearly overlaps with acceptability”, and thus, usability measurement tools (such as the system usability scale) could be used to assess acceptability. It would be interesting to see whether researchers chose to measure a system’s usability to make inferences regarding its acceptability.

13Although there seems to be an effort in the research community to ground assessment methods in existing theoretical frameworks, custom measurement tools such as ad hoc surveys are also used to investigate technology acceptance (eg, Allen and Seaman). The use of ad hoc tools allows explorations of acceptance to be tailored to a particular context but makes it more difficult to compare results across projects.

14A number of definitions and models of technology acceptance are available in the literature, several of which make important distinctions between the different stages of the process. Omitting that distinction in the terminology, and using the terms acceptability, acceptance, and adoption interchangeably can create ambiguities about what is actually being measured, making the replication of interventions and comparison of results difficult. Such ambiguity is highlighted in a review that showed the confusion around the concept of technology acceptability in the health care literature. Furthermore, a review of definitions for electronic health stressed the importance of common terminology for interdisciplinary collaboration. Technology acceptance is a particularly significant challenge for the design of mHealth care technologies; hence we focus on that particular context. In this paper, we present the results of a scoping review of the mHealth literature addressing the following questions:

15How do researchers define technology acceptance?

16What terminology is used to refer to technology acceptance?

17How do researchers measure technology acceptance?

18How do researchers make use of existing models of acceptance?

19In addition to outlining research practices to evaluate acceptance, this review reveals the potential limitations and areas for improvement of existing models. A better understanding of these elements contributes toward the development (or improvement) of methodologies and measurement tools for addressing technology acceptance within the development of mHealth applications. Informed by this analysis, a further contribution is made by integrating and disambiguating existing definitions. To this end, we present a lifecycle of the process of acceptance, providing researchers with a common terminology to report results, and help them to measure the evolution of user acceptance over time.

20We performed a scoping review to map relevant literature in the field of mHealth. In contrast to systematic and narrative reviews, scoping reviews allow for a broad but structured exploration that permitted us to examine the range of definitions and measurements of technology acceptance, and to identify gaps and inconsistencies in the existing literature. We followed the framework developed by Arksey and O'Malley, which consists of the stages outlined below.

21We were interested in how technology acceptance was understood, measured, and reported in mHealth studies.

22We searched the PubMed database for papers addressing technology acceptance. Following preliminary searches to assess the relevance of search criteria, a search of the Medical Subject Headings (MeSH) terms “Patient Acceptance of Health Care” and “Mobile Applications,” without time restriction, resulted in a corpus of 287 articles starting from 2013.

23Inclusion criteria were subject to discussion between the 3 authors; 2 random samples of 10 papers were independently assessed for inclusion by 2 pairs of authors (first and second, first and third). Disagreements concerning the definitions of inclusion criteria were resolved by explicitly stating these criteria as following: a publication was considered relevant if (1) it contained at least one of the terms “acceptability,” “acceptance,” “adoption,” “accept,” or “adopt”; and (2) defined the concept in question with (1) a full definition; (2) a synonym; or (3) an operationalized definition (ie, means used to measure the concept). Some articles were associated with the specified MeSH terms but did not directly discuss technology acceptance. Articles were excluded if they did not contain any of the 5 terms above, did not provide a definition, or if their web version was not accessible. Among those, 40 were excluded at the screening stage (Figure 1) because they did not address acceptance in the sense of technology acceptance (eg, Acceptance and Commitment Therapy). A total of 68 relevant publications were included in the review.

24The codes used for analysis aimed to expose how technology acceptance was referred to, defined, and measured, depending on the context (Table 1). Each pair of authors coded 2 samples of 10 papers independently. This resulted in no disagreement, the codes were straightforward to interpret.

25Definitions from the literature mentioned earlier distinguish between the different stages of the process of acceptance. We assessed whether the definitions found in our corpus also differentiate between the stages of acceptance and the different terms acceptability, acceptance, and adoption. We carried out 2 classifications of the papers, one with respect to the terminology used (ie, acceptability, acceptance, or adoption) and one following the measurement stage (ie, preuse, initial use, and sustained use). We then grouped papers with similar interpretations of technology acceptance and extracted the measurements used. Finally, we identified possible gaps and suggested ways to improve the exploration of acceptance in mHealth.

26All 3 terms (acceptance, acceptability, and adoption) were used in the mHealth literature sample and were associated with various meanings and measurements.

27Classification of the papers reveals that one-fifth of them cited definitions from the literature. Two-third of the articles reported on a mental health intervention, which shows that the question of technology acceptance is particularly studied in this context. Figure 2 compares the terminology used in the papers with a classification following the measurement stage. A significant difference can be seen between the 2 distributions, marking the distinction between the 3 terms, but also their use in relation to the measurement timeline.

28The majority of publications (n=51) referred to technology acceptability. Among these, a small number (n=8) addressed the preuse stage, sometimes equating acceptability to users' interest or willingness to use a system. The other 34 papers referred to acceptability in the context of initial or sustained use.

29Papers in the second group (n=20) referred to technology acceptance. Among these, 2 papers refer to it at preuse, and 13 in the context of sustained use.

30The last part of the corpus (n=11) refers to technology adoption. Half of these papers focused on the sustained use stage and the other half on the preuse stage. Papers reported as unclear were those for which classification was not possible as the context of the study was ambiguous. Although the terminology used showed a focus on acceptability, looking at the corpus through the lens of the measurement timeline highlighted that more than two-third of the papers (n=47) explored the sustained use stage. This distribution suggested a trend in the corpus to employ the term acceptability, regardless of the context and stage of the user journey. Table 2 presents the range of interpretations for acceptability, acceptance, and adoption extracted from the corpus.

31The distribution shown in Table 2 reveals that acceptability was mainly understood as perceived usefulness (a concept from TAM) or user satisfaction. System usability, user feedback, and the other TAM constructs (perceived ease of use, attitude toward using, intention to use, and actual usage) were often mentioned. It also emerged that researchers may consider a perceived reduced stigma and high similarity between the behavior of the technology and traditional health care as markers for acceptability. Furthermore, one study explored sociocultural aspects of technology acceptability in developing regions, explaining that a system needs to take into account “the preferences and aspirations of individual service users and the cultures of their communities”. A striking finding was that, out of 6 years of research, only 2 works explored mHealth for children, with Farooqui et al alone studying children’s acceptance.

32Fewer studies employed the term acceptance, associating it with constructs from the TAM—perhaps because the model itself uses the term acceptance.

33TAM constructs were also found in the interpretations of adoption. Some of this work seeks to adapt or extend the TAM, such as Khatun et al, who developed the concept of readiness to adopt mHealth in developing countries. The authors argued that the TAM “does not consider the influence of human factors, the internal resources of the user or the external environmental and ecological factors” and proposed a model integrating TAM constructs and others more specific to rural and developing areas (access, sociodemographics, awareness of mHealth services, and trust).

34Among the literature reviews present in the corpus, some associated acceptability or acceptance with feasibility. Feasibility, as defined by the National Institute for Health Research glossary, relates to whether a study can be carried out. A feasibility study explores not only technical parameters but also human factors likely to be important for the conduct of the main study. Although a feasibility study may include the assessment of participants’ acceptance, it is not the case that all feasibility studies will do so.

35A small number of studies (n=13) cited conceptual definitions from the literature. The TAM was cited in reference to the 3 concepts: acceptability, acceptance, and adoption.

36However, some studies have highlighted the limitations of existing models. Zhu et al argue that conceptual models should not regard mobile services as a generic concept but specifically address particular use cases (eg, mobile services for health monitoring). They present their own model for technology acceptance, combining TAM constructs with health-related concepts from the health belief model.

37In the same vein, Povey et al stated that existing models were not suitable for their study. The authors attempted to build a model for the acceptability of e-mental health for an indigenous population and acknowledged that the resulting model is similar to the HITAM.

38Ammenwerth et al argued that existing models such as the TAM and task technology fit failed to address the interaction between user and task. To bridge this gap, the authors developed and validated the fit between individuals, task, and technology (FITT) framework for information technology adoption in clinical environments. Only one study then employed FITT to measure acceptance of the use of mobile apps among physical therapists. Finally, another study cited Agarwal and Prasad, supporting the idea of 2 stages: initial adoption and long-term engagement.

39These references to models and definitions within the literature show researchers’ willingness to reuse existing theories. However, the adaptations of these models and their inconsistent interpretation also reveal their limitations and the lack of a common framework to study technology acceptance.

40We extracted measurements employed in the corpus to assess technology acceptance and classified them by type and temporality (Table 3).

41Table 3 illustrates a strong preference for the use of surveys (73.0%, 49/67), independently of the timeline, with a prevalence of custom questionnaires (55.0%, 37/67). Almost all studies assessing system usability (which is a frequent interpretation of acceptability, see Table 2), made use of standardized surveys, which reflects the range of reliable tools available to evaluate this concept.

42Other studies used existing surveys incorporating TAM constructs or developed their own with elements from different models to fit their research better. Owing to the need to assess technology acceptance in developing countries and remote areas, Chen et al suggested that models found in the literature may need to be validated on a larger variety of populations.

43The rest of the corpus measured technology acceptance through participant feedback (via custom surveys, focus groups, and interviews), adherence, usage, and rate of completion of tasks. Although this wide set of metrics contributes to expanding the number of assessment tools, it also impedes consistency within technology acceptance research.

44The low number of measures applied at the pre and initial use stages also reveals that technology acceptance was rarely investigated at the design stage (Table 4).

45Interestingly, although existing acceptance models provide sets of measurable constructs, researchers’ efforts have focused on perceived usefulness. Only the studies in Table 5 measured the full set of constructs contained in the technology acceptance models which they cite.

46Table 5 shows that almost all studies that relied on the TAM added constructs to capture the influence of context-related factors on acceptance. Indeed, Zhu et al followed the added variables approach described by Holden and Karsh to evaluate the acceptance of their technology within the specific Chinese health care context. In addition to the TAM constructs, they used 4 context-related constructs taken from other studies. Similarly, Chen et al used a questionnaire based on the TAM with additional constructs from other studies to fit the context of their application. Finally, Wang et al showed the influence of certain demographic factors (age, position at work, usage time of the personal digital assistant [PDA], and skill level of using a PDA) on the TAM constructs (perceived usefulness and ease of use).

47On the other hand, Ammenwerth et al used a survey based on the information system success model to evaluate acceptance postuse at 2 different points in time. The first survey assessed the 7 constructs from the model, whereas the second only assessed intention to use and net benefit. This could suggest that these constructs are seen as more stable or important, or that the existing constructs are not suitable for all study designs.

48As expected from the analysis of definitions, these studies measured a wide variety of constructs. New questionnaires were developed, and researchers stressed the limitations of existing measurement frameworks. Without the validation of these new surveys, the comparison and replication of studies may be difficult. Thus, we argue that standardized ways to measure technology acceptance are needed to support objective and coherent assessments at the different stages of the user journey.

49This scoping review analyzed the terminology used to refer to technology acceptance and extracted the different interpretations given and measurement methods employed.

50The findings reveal that technology acceptance was mostly referred to as acceptability and, to a lesser extent, acceptance or adoption. A small part of the corpus converges toward the differentiation of these terms as distinct concepts, corresponding to stages (ie, preuse acceptability, initial and sustained use acceptance) or events (ie, point of adoption) in the user journey of technology acceptance. However, the rest of the corpus does not distinguish between these terms. One reason could be that the existing definitions arguing for that distinction were published separately; hence, providing a coherent and more precise terminology and set of definitions can aid researchers in communicating which concepts they are referring to.

51The corpus provided a variety of interpretations of the concept of technology acceptance. Among these, some were based on constructs present in literature models such as the TAM, indicating that existing models do influence acceptance studies. Other definitions should be associated with the appropriate model constructs; for example, user satisfaction and feedback should refer to the TAM’s construct attitude toward use. Similarly, factors related to the sensitive nature of health care technologies (such as perceived stigma) should be linked to the appropriate constructs’ subjective norm (in the HITAM), social influence (in the pervasive technology acceptance model), or sharing (in the model by Cheung et al). Thus, it is likely that the extent to which an individual is concerned by these risks will impact their acceptance of the technology. Following previous research, we argue that acceptance models should take into account the cultural and health context of end users.

52Although part of the corpus employed standardized tools, the majority of the studies used custom surveys. This aligns with the findings of Wozney et al that the major part of their corpus used nonvalidated measures of acceptability. This could be because of a lack of validated tools to assess technology acceptance in the context of mHealth. Many researchers felt the need to design their own survey to have a measurement instrument adapted to the specific issues of their target population (eg, technology access and cost). Similarly, researchers who used existing acceptance models felt the need to add context-specific constructs. It would be interesting to see a community effort to validate new tools, create adapted tools for important contexts such as mHealth, and adapt existing models and questionnaires so that they embrace the changes in users’ acceptance as they use the technology. Finally, technology acceptance was rarely investigated at the design stage. This is unexpected as exploring acceptance issues could greatly inform design work and reduce risks that an implemented technology is rejected or abandoned. On the use of machine learning in clinical contexts, Thieme et al argue that collaborating with health care users at the design stage may increase the chances of acceptance of the technology. Existing acceptance models were only used once to measure acceptance at the preuse stage (Table 5). A reason for this might be that existing models do not target the design stage and are not adapted to measurement at the preuse stage.

53In line with the existing literature definitions and informed by our analysis of the corpus, we argue for clearly distinguishing between the different stages of technology acceptance. We have seen in our analysis both a wide variety of interpretations of these concepts, and a range of measurement approaches applied across the lifecycle, mostly based on ad hoc tools. This variety creates ambiguities in the reporting and understanding of results and makes it difficult to draw conclusions on the acceptance of the systems studied. We believe that a better understanding of the process of technology acceptance would greatly benefit the community in terms of researchers articulating their findings with regard to the entire process. To contribute toward clarification of the measurement of technology acceptance, we propose the Technology Acceptance Lifecycle (TAL). The TAL consists of a timeline to anchor the definitions of technology acceptance within the overall process (Figure 3).

54Our motivation is to highlight the evolving nature of technology acceptance across the different stages of the user journey with a technology. The two main stages of the TAL follow Karahanna et al and Roger's interpretations in favor of a distinction between the pre- and postadoption stages. With regard to the definitions of Distler et al, Martin et al, Garces et al, and Terrade et al, we argue for a distinction between the stages of acceptability (preuse) and acceptance (initial use) and propose the continuum acceptability-acceptance-sustained use. The TAL proposes a more explicit terminology, embedding temporality in the name of the different stages. The continuum then becomes preuse acceptability—initial use acceptance—sustained use acceptance. Finally, sometime during the sustained use stage, the user would reach the point of adoption of the technology. According to Rogers, a system can be considered adopted when users make full use of it. However, the literature does not specify the conditions for full use to be achieved. On the basis of the existing literature definitions, the TAL articulates the process of technology acceptance and its different stages across the user journey. We argue that acceptance at the initial and sustained use stages is likely to be impacted by factors related to the actual use of the technology and user engagement. Therefore, we suggest that the research community considers acceptance as a process (rather than a discrete measure) and adopts assessment approaches that take into account the temporal dimension and possible evolution of acceptance. We believe that the proposed TAL could help align the research field and provide researchers with a timeline that they can follow to assess technology acceptance and terminology to communicate their research clearly. Further research is needed to establish the influencing factors at the different stages of the process and to develop and validate measurement methods adapted to these stages.

55Our search focused on the PubMed database and relied on the MeSH classification. We did not apply any time restriction and obtained a reasonably sized and highly relevant sample. However, a more exhaustive sample could be obtained by expanding the search to other terms.

56This review has identified the common interpretations and measurement approaches that have been used to assess the acceptance of mHealth technologies. To our knowledge, this is the first study to look at the terminology employed and examine the basis and consistency of the definitions employed in mHealth research. In addition, this review has described how researchers made use of existing models to measure mHealth acceptance and the lack of readily available assessment tools that are appropriate for specific study contexts and use at the design stage. This paper has uncovered the need for a common definition and measurement framework to address technology acceptance, particularly in the domains of health and mental health. A common set of definitions and more consistent approaches to measurement would support both developers of mHealth applications in addressing user acceptance of their systems and the communication and reporting of results in interdisciplinary studies of mHealth interventions. With the TAL, we propose a more explicit terminology and a representation of the process of technology acceptance throughout the user journey.

57Our findings highlight the importance of better articulating the specific concepts highlighted in the TAL and developing appropriate measurement tools, ideally standardized, for each of these concepts. This perspective encourages developers to consider acceptance across the user journey and allows researchers to be more explicit about what they are investigating. Finally, efforts from the mHealth community are needed to adapt existing acceptance models for use in sensitive contexts such as mental health interventions.