Health workers’ perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis

1Mobile health (mHealth) refers to medical and public healthcare practices supported by mobile devices, such as mobile and smartphones, client‐monitoring devices, personal digital assistants (PDAs), and tablets (WHO 2011). It also refers to these devices' capabilities to create, store, retrieve, and transmit information between users (Akter 2010). mHealth relies mainly on the mobile phone's utility of voice, short message services (SMS) and multimedia message services (MMS), but also includes more complex applications, such as global positioning systems, Bluetooth technology, and third and fourth generation mobile telecommunications (3G and 4G systems) (WHO 2011).

2These devices leverage the reach and speed of mobile networks and mobile computing power to improve the reach of healthcare delivery (Leon 2012; West 2014), including the capturing, processing, and exchange of information (Gagnon 2009), holding the potential to transform aspects of health service delivery and health systems management (Qiang 2011; Tomlinson 2013). In pursuit of universal health coverage, mobile health has the potential to extend the scope, accessibility and quality of health services, to increase the accountability mechanisms, to expand the population base accessing health services, and to increase capacity of the healthcare workforce (Agarwal 2016; Labrique 2013a; Labrique 2013b; Mehl 2014).

3The growing interest in mHealth as a research topic is reflected in the 25 effectiveness reviews published in the Cochrane Library (Appendix 1). Two overviews of reviews have also identified 29 systematic reviews (Marcolino 2018; Mbuagbaw 2015), of which 17 were non‐Cochrane Reviews. These Cochrane and non‐Cochrane reviews cover mobile health technologies that vary in their type and purpose, from the use of email for clinical communication between healthcare professionals (Pappas 2012), to the use of mobile phones for healthcare appointment reminders (Gurol‐Urganci 2013). The evidence on the effectiveness of mHealth cited in these reviews also varies. The overview of reviews from Marcolino 2018 shows mixed results and a lack of long‐term studies, although some evidence suggests an effect on some health outcomes. The growing importance and interest in mHealth is also reflected in the launch of two new journals, one of which is within the Lancet group of journals, namely The Lancet Digital Health (www.thelancet.com/journals), and mHealth (mhealth.amegroups.com).

4This review synthesises evidence of how health workers perceive and experience their use of mHealth devices to provide and support primary healthcare services, defined in this review as either the first contact point of health care (Awofeso 2004), any rehabilitative, therapeutic, preventive and promotive health care (Global Health Watch 2011), being delivered at an individual or community level, or both (Muldoon 2006); or bringing healthcare services to where people work and live, which in particular applies to low‐income settings (Muldoon 2006).

5These devices refer to mobile devices, such as mobile and smartphones, client‐monitoring devices, PDAs, and digital tablets, and particularly refers to these devices' capabilities to create, store, retrieve, and transmit information between users (Blaya 2010; Braun 2013; Catalani 2013; Hall 2014). Examples of how mHealth supports primary healthcare services, include: (i) clinical decision support during client consultation (Ginsburg 2016; Ilozumba 2018); (ii) information management to organise or deliver services, or both (Lodhia 2016; Ramirez 2017); (iii) health promotion messages to health workers and clients alike (Cherrington 2015); and (iv) communication between health workers, as well as between health workers and clients (Barnabee 2014; Chang 2011; Hampshire 2016). We use the terms 'mHealth' and 'mobile health' interchangeably in this review.

6The release in April 2019, of the World Health Organization (WHO) guideline on digital interventions for health system strengthening (WHO 2019), attests to recognition at the highest level of global health, that mHealth is now a significant component in the delivery and support of healthcare policy, guideline and decision‐making processes. Processes, such as the development of this guideline, should be supported by “… social scientific studies explicating processes of technology adoption …” (Chib 2015). Identifying, appraising and synthesising the qualitative evidence of health workers' perceptions and experiences of mHealth programmes, complement the reviews of mHealth effectiveness and help improve our understanding of the barriers to, and facilitators of, its successful implementation (Chang 2013; Grimsbø 2012; Medhanyie 2015), as well as helping us to understand the outcomes, implementation, and feasibility of these programmes. This is particularly important as decision makers move from assessing the options to implementing the intervention, and thus need to consider more than whether an intervention works or not, but also the extent to which it may be acceptable in different contexts (Langlois 2018). This review is one of two qualitative evidence syntheses, that have been used alongside a suite of reviews of effectiveness, to inform the recently published WHO guidelines (WHO 2019); the other Cochrane Review focuses on clients' and peoples' perceptions and experiences of targeted digital communication, accessible via mobile devices for reproductive, maternal, newborn, child and adolescent health (Ames 2019).

7The Cochrane and non‐Cochrane effectiveness reviews (Agarwal 2018a; Agarwal 2018b; Agarwal 2018c; Braun 2013; Gonçalves‐Bradley 2018a; Gonçalves‐Bradley 2018b; Vasudevan 2018; Vervloet 2012), showed mixed or inconclusive results. In order to understand this heterogeneity, we need to go beyond the numbers and explore the context in which the interventions are delivered, and the experiences of the people involved in the delivery (Langlois 2018). This may lead to a better understanding of possible reasons why mHealth interventions have worked differently in different contexts. It is therefore, important to supplement the evidence of effectiveness by exploring the barriers and facilitators to the successful implementation of mHealth interventions, through qualitative studies that take contextualised experience into account (Glenton 2013). This would support the call by some of the effectiveness reviews that "… clients’ and healthcare providers' evaluation and perceptions of the safety of the interventions, potential harms, and adverse effects … should be assessed" (Gurol‐Urganci 2013), and "… barriers to trial development and implementation should also be tackled [in future studies]" (Atherton 2012). This qualitative evidence synthesis intends to be both complementary to the effectiveness reviews, as well as providing robust evidence in its own right.

8To synthesise qualitative research evidence on health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services, and to develop hypotheses about why some technologies are more effective than others.

9We included primary studies that used qualitative methods for data collection (e.g. interviews, focus group discussions, document analysis, and observations), and qualitative methods for data analysis (for instance, thematic analysis, and grounded theory). We excluded primary studies that collected data using qualitative methods but did not perform a qualitative analysis (e.g. open‐ended survey questions where the responses were analysed using descriptive statistics). We included mixed‐methods studies when it was possible to extract data that resulted from the qualitative methods. We included studies regardless of whether they had or had not been carried out alongside studies of effectiveness of mHealth.

10We included studies that focus on the perceptions and experiences of the following.

11All categories of health workers (i.e. professionals, paraprofessionals and lay health workers) who were involved in providing primary healthcare services to clients. We defined a paraprofessional health worker as someone with some form of secondary education and subsequent informal and/or formal training, lasting a few months to more than a year (Olaniran 2017). We defined a lay health worker as any health worker who performs functions related to healthcare delivery, is trained in some way to provide these functions, but has received no formal professional or paraprofessional certificate or tertiary education degree (Lewin 2005). Where appropriate, we distinguished between different categories of health workers, for example, health professionals and lay health workers.Any other individuals or groups involved in delivering and managing mHealth programmes which aimed to provide or support primary healthcare services to clients. These individuals or groups included administrative staff, information technology staff, managerial and supervisory staff, they may have been based in a primary healthcare facility or in the community, but could also have been employed at a district or national level. The criterion was that they were to be involved in supporting the delivery of primary healthcare services or the mHealth programmes or both, irrespective of their placement.

12All categories of health workers (i.e. professionals, paraprofessionals and lay health workers) who were involved in providing primary healthcare services to clients. We defined a paraprofessional health worker as someone with some form of secondary education and subsequent informal and/or formal training, lasting a few months to more than a year (Olaniran 2017). We defined a lay health worker as any health worker who performs functions related to healthcare delivery, is trained in some way to provide these functions, but has received no formal professional or paraprofessional certificate or tertiary education degree (Lewin 2005). Where appropriate, we distinguished between different categories of health workers, for example, health professionals and lay health workers.

13Any other individuals or groups involved in delivering and managing mHealth programmes which aimed to provide or support primary healthcare services to clients. These individuals or groups included administrative staff, information technology staff, managerial and supervisory staff, they may have been based in a primary healthcare facility or in the community, but could also have been employed at a district or national level. The criterion was that they were to be involved in supporting the delivery of primary healthcare services or the mHealth programmes or both, irrespective of their placement.

14Given the review's focus, i.e. health workers' use of mHealth to deliver primary healthcare services, we excluded the perceptions and experience of clients in this review. We also excluded participants identified as technical staff who developed and maintained the mHealth architecture used, for example, those involved in writing the software programmes or who provided technical support to the end users.

15We included studies of mHealth programmes that were part of primary healthcare services delivery. For the purposes of this review, we defined 'primary healthcare services' as one or any combination of the following.

16The first contact point of health care (Muldoon 2006).All rehabilitative, therapeutic, preventive, and promotive health care (Global Health Watch 2011).Being delivered at an individual or community level, or both (Muldoon 2006).Bringing healthcare services to where people work and live, which in particular applies to low‐income settings (Muldoon 2006).

17The first contact point of health care (Muldoon 2006).

18All rehabilitative, therapeutic, preventive, and promotive health care (Global Health Watch 2011).

19Being delivered at an individual or community level, or both (Muldoon 2006).

20Bringing healthcare services to where people work and live, which in particular applies to low‐income settings (Muldoon 2006).

21These services could be implemented in public or private primary healthcare facilities, in the community and workplace, or the homes of clients. We included studies conducted in any country.

22While our review focuses on primary healthcare services as a micro‐level health system, we understand and acknowledge that these services are embedded within broader, meso‐level, i.e. district health systems, which deliver health care at secondary and tertiary levels (Gilson 2012; Langlois 2018). These district‐level systems are in turn, shaped by the socioeconomic, political, and health system contexts at a macro level, i.e. national and global levels (Langlois 2018). It is therefore, to be anticipated that the barriers and facilitators to the successful implementation of mHealth programmes will be found across the three tiers, ranging from micro‐level issues, such as protecting the confidentiality of primary healthcare client information (Labrique 2013), to meso‐ and macro‐level issues, such as reliable network coverage (Aranda‐Jan 2014), and the integration of mHealth platforms into higher‐level existing electronic health systems (Aranda‐Jan 2014), and that this will be reflected in the experience of participants.

23This review focused on health workers' perceptions and experiences of their use of mHealth devices to provide and support primary healthcare services.

24In this review, mHealth devices were defined as mobile devices that are used to create, store, retrieve, and transmit data in real time between users (see Appendix 2 for more technical definitions related to these devices).

25We included interventions in which health workers used mobile devices to provide and support any type of primary healthcare service, which revolved around uni‐ and bi‐directional communication between health workers and clients, between health workers themselves, and between health workers and programme staff, other than health workers. In some instances, there was no direct interpersonal communication per se, but only health workers interacting with digital information available on the devices. Examples of these communications and interactions with data, included client registration and tracking, disease surveillance, various forms of decision support during consultations, for instance algorithms loaded on the devices, automated messaging to health workers, and stock notifications. We accessed the World Health Organization (WHO) taxonomy for digital health interventions (WHO 2018), and added their classification to our description of the technologies used in the included studies (Characteristics of included studies). According to this classification, digital health interventions are categorised according to targeted primary users, identified as: clients, health workers, health system or resource managers, and interventions for data services.

26In our search for synthesised evidence, we searched PDQ‐Evidence (www.pdq‐evidence.org) and the Cochrane Library (www.cochranelibrary.com) for related reviews on 21 February 2018. We scanned any identified reviews to assess if any of the studies included or cited in the reviews could potentially also be included in our review.

27We searched the following databases for primary studies without any language, date, or geographic restrictions.

28MEDLINE Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, MEDLINE Daily and MEDLINE 1946 to present, Ovid (searched 12 January 2018)Embase 1974 to 11 January 2018, Ovid (searched 12 January 2018)CINAHL 1981 to present, EbscoHost (searched 11 January 2018)Science Citation Index and Social Sciences Citation Index 1987 to present, and Emerging Sources Citation Index 2015 to present, ISI Web of Science (searched 12 January 2018) (topic search)Global Health 1973 to 2015 Week 48, Ovid (searched 8 December 2015)

29MEDLINE Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, MEDLINE Daily and MEDLINE 1946 to present, Ovid (searched 12 January 2018)

30Embase 1974 to 11 January 2018, Ovid (searched 12 January 2018)

31CINAHL 1981 to present, EbscoHost (searched 11 January 2018)

32Science Citation Index and Social Sciences Citation Index 1987 to present, and Emerging Sources Citation Index 2015 to present, ISI Web of Science (searched 12 January 2018) (topic search)

33Global Health 1973 to 2015 Week 48, Ovid (searched 8 December 2015)

34We did not search Global Health in 2018 as we had no access to this database.

35We screened the reference lists of all the included studies and key references (i.e. relevant systematic reviews).

36We conducted a grey literature search in the following sources to identify studies not indexed in the databases listed above.

37Eldis: www.eldis.org (searched 21 February 2018)Google Scholar: scholar.google.co.za (searched 21 February 2018)mHealth Database: www.africanstrategies4health.org/mhealth (searched 05 March 2018)mHealth Evidence: www.mhealthevidence.org (searched 21 February 2018)mHealth Knowledge: mhealthknowledge.org (searched 05 March 2018 )mPowering: partnerships.usaid.gov/partnership/mpowering‐frontline‐health‐workers (searched 05 March 2018)OpenGrey: www.opengrey.eu (searched 16 February 2018)The Grey Literature Report: www.greylit.org (searched 21 February 2018)

38Eldis: www.eldis.org (searched 21 February 2018)

39Google Scholar: scholar.google.co.za (searched 21 February 2018)

40mHealth Database: www.africanstrategies4health.org/mhealth (searched 05 March 2018)

41mHealth Evidence: www.mhealthevidence.org (searched 21 February 2018)

42mHealth Knowledge: mhealthknowledge.org (searched 05 March 2018 )

43mPowering: partnerships.usaid.gov/partnership/mpowering‐frontline‐health‐workers (searched 05 March 2018)

44OpenGrey: www.opengrey.eu (searched 16 February 2018)

45The Grey Literature Report: www.greylit.org (searched 21 February 2018)

46The search strategies for the main databases can be found in Appendix 3.

47We re‐ran the search strategies in February 2020. We screened these records and potentially relevant studies are awaiting classification; we will assess these studies at the next update.

48We collated all titles and abstracts identified through the search strategy into one reference management database Covidence. After removing duplicate records, each record was independently assessed by the first review author and any one of the other review authors, for its potential inclusion eligibility. We excluded records that were not relevant to the topic of this review. Thereafter, we retrieved the full text of all of the abstracts and titles that have been assessed as potentially eligible. Using the same process as for the abstracts, each full text was independently assessed by the first review author and any one of the other review authors, based on the review's inclusion criteria. To minimise bias, a review author was not permitted to assess a full text to which (s)he was an author. Given the high number of full texts we had to assess, we recruited an additional researcher and trained her to assist us with these assessments. We resolved disagreements between review authors through email correspondence and face‐to‐face discussions. When the two review authors could not reach consensus, we reverted to a team decision through email correspondence. In one instance, these email discussions resulted in a refinement of our inclusion criteria: though we included mobile health communication in our protocol, we did not specify the equipment used for emailing, and during a team discussion we agreed to exclude papers in which email was sent from stationery devices, such as a laptop used by a general practitioner in his/her consultation room. We contacted several study authors for more study information, when the information in the full text was insufficient to determine inclusion or exclusion of the study.

49Translation of languages other than English

50Abstracts of three studies required translation. Two of these were in Spanish and one in French. We translated the abstracts of these studies, using open source software (Google Translate: translate.google.com), and excluded the studies based on the translated version of the abstracts. No full‐text studies required translation.

51We identified 23 studies from our 2015 search. We included all 23 studies in our analysis. In 2018, we repeated our search, and identified an additional 30 studies. While small sample sizes can lead us to have less confidence in a finding, large sample sizes can also threaten our ability to carry out a thorough qualitative analysis (Glenton 2018; Sandelowski 1995). We therefore decided to select a sample of these 30 studies. Several of the studies we had identified in our 2015 search had a number of methodological limitations. This had led us to downgrade our certainty in several of the findings we had developed during our first analysis. We therefore decided to sample studies from the 2018 search based on our assessment of their methodological limitations. While we had included all studies identified from our 2015 search regardless of their methodological limitations, we only included studies from the 2018 search that we assessed as having no to moderate concerns regarding their methodological limitations. This led us to sample 20 of the 30 studies from the 2018 search (see Table 2 for the exclusion reasons of the 10 studies we appraised as having serious methodological limitations). In the main, studies that were not sampled because of serious methodological limitations had poor descriptions of participant selection, data collection and analysis methods; thin data; and little information on author reflexivity.

52It should be noted that our data coding, extraction, synthesis, and writing of findings, were conducted in two stages. We had already completed these steps for the 23 included studies from the first search (done in 2015), by the time we began the same process for the 20 included studies from the second search (done in 2018). The details of both stages are presented below.

53For both the 2015 and 2018 search studies, we extracted study information, such as country, the health worker category, the healthcare issue addressed, and the specific mobile health technology used, into an Excel spreadsheet. This served as a tool to refer to the study details during the data extraction and coding.

54The data coding, extraction and synthesis process was an iterative process, aligned with the thematic synthesis process outlined by Thomas 2008. For the 23 included studies from the first search, two review authors (WO, KD) independently read each study as a whole, including the background, methods, results, discussion, and conclusions sections, to get a sense of their meaning and their contribution to answering the review question. Each review author thereafter conducted a line‐by‐line coding of the data of the first study. They then met and agreed on the codes and supporting data. They used this code list to code the second paper, thus beginning the process of translating the data from one study into the next. New codes that emerged from the second, and subsequent studies, were added to the list, and we returned to the already coded studies, to determine if these codes applied to that data also. As the code list was amended, the authors began the process of organising the codes into broad themes, which in some cases had subthemes attached to it. Using the thematically coded data, the same two review authors jointly wrote up discreet findings. Since many of the extracts did not neatly fit within any theme, we continued the iterative process of trying to make sense of the extracts, by regrouping them with other extracts from which similar underlying issues had emerged, and eventually synthesised all the extracted data into findings.

55The same two review authors (WO and KD) that led the analysis for the first 23 studies, did so for the 20 new studies from the 2018 search. By the time we started coding these 20 new studies, we already had an existing list of themes and subthemes to use as a deductive coding framework. However, we were cognisant that the new set of studies might yield data not yet captured in our framework. We therefore approached the analysis both deductively and inductively, reading the data to determine if and where it fit within the existing framework, and for what new insights it yielded. Data extracts were therefore grouped by WO and KD, both into existing categories, as well as into new categories that emerged from the data. Upon completing this for all the new studies, one review author (WO) amended the texts of the existing findings to reflect the additional data. The rest of the author team verified that all the supporting data were reflected in the amended and new findings. Upon completing this for all the new studies, one review author (WO) amended the texts of the existing findings to reflect the additional data. We also constantly evaluated each extract against our inclusion criteria and review objectives, deciding up until the very end, whether or not it was an appropriate fit. The findings thus represent the final translation of the coded data across all of the 43 included sample studies.

56The aim of the data synthesis was to develop a set of findings we believe represent a trustworthy, coherent, and detailed understanding of the perceptions and experiences of those who deliver and support primary healthcare services through using mobile devices. As detailed above, we synthesised the coded data into a set of 42 discreet findings. Thereafter, the one review author (WO) involved in drafting the findings, thematically analysed these findings and grouped them into four overarching themes. These themes provide a coherent overview of our findings.

57At a minimum, all included studies had to have used qualitative data collection and analysis methods. Prior to the data coding, extraction, synthesising, and writing the findings from both searches, two review authors (WO, KD) independently assessed the methodological limitations of the included studies using an adapted Critical Appraisals Skills Programme (CASP) tool (Atkins 2008). We assessed each study on the following nine criteria.

58Adequately described setting and contextA well described sampling strategy that is appropriateA well described data collection strategy that is appropriateAn adequately described data analysis method that is appropriateSufficient evidence to support the claims made/findingsAdequate evidence of researcher reflexivityDemonstrated sensitivity to ethical concernsAdequately described study limitationsAny other concerns raised by the review authors

59Adequately described setting and context

60A well described sampling strategy that is appropriate

61A well described data collection strategy that is appropriate

62An adequately described data analysis method that is appropriate

63Sufficient evidence to support the claims made/findings

64Adequate evidence of researcher reflexivity

65Demonstrated sensitivity to ethical concerns

66Adequately described study limitations

67Any other concerns raised by the review authors

68Based on their assessment, the two review authors (WO, KD) independently graded each study as having no, or very minor, minor, moderate, or serious methodological limitations. Thereafter, they met and reached consensus on their respective assessments.

69Three review authors (WO, JAW, KD) used the GRADE‐CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to summarise our confidence in each finding (Lewin 2018).

70GRADE‐CERQual assesses confidence in the evidence, based on the following four key components.

71Methodological limitations of included studies: the extent to which there are concerns about the design or conduct of the primary studies that contributed evidence to an individual review finding.Coherence of the review finding: an assessment of how clear and cogent the fit is between the data from the primary studies and a review finding that synthesises those data. By cogent, we mean well supported or compelling.Adequacy of the data contributing to a review finding: an overall determination of the degree of richness and quantity of data supporting a review finding.Relevance of the included studies to the review question: the extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context (perspective or population, phenomenon of interest, setting) specified in the review question.

72Methodological limitations of included studies: the extent to which there are concerns about the design or conduct of the primary studies that contributed evidence to an individual review finding.

73Coherence of the review finding: an assessment of how clear and cogent the fit is between the data from the primary studies and a review finding that synthesises those data. By cogent, we mean well supported or compelling.

74Adequacy of the data contributing to a review finding: an overall determination of the degree of richness and quantity of data supporting a review finding.

75Relevance of the included studies to the review question: the extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context (perspective or population, phenomenon of interest, setting) specified in the review question.

76After assessing each of the four components, we made a judgement about the overall confidence in the evidence supporting the review finding. We judged confidence as high, moderate, low, or very low. A sample (40%) of the final assessment was peer reviewed by a fourth review author (NL), and we adjusted some of the assessments after reaching consensus with the fourth review author. We started with high confidence in all findings, and then downgraded any findings where we had important concerns regarding any of the GRADE‐CERQual components.

77We presented summaries of the findings and our assessments of our confidence in these findings in Table 1. We presented detailed descriptions of our confidence assessment in Appendix 4.

78We sought to understand how our findings were related to, and could help to inform, the findings of six of the Cochrane Reviews of effectiveness that were used to inform the WHO guideline on digital interventions for health system strengthening (WHO 2019). These reviews assessed the effectiveness of the following mHealth interventions.

79Birth and death notification via mobile devices (Vasudevan 2018)Stock notification and commodity management via mobile devices (Agarwal 2018a)Client to provider telemedicine (Gonçalves‐Bradley 2018a)Tracking of client's health status and services received (Agarwal 2018b)Health provider decision support via mobile devices (Agarwal 2018c)Health provider to health provider telemedicine (Gonçalves‐Bradley 2018b)

80Birth and death notification via mobile devices (Vasudevan 2018)

81Stock notification and commodity management via mobile devices (Agarwal 2018a)

82Client to provider telemedicine (Gonçalves‐Bradley 2018a)

83Tracking of client's health status and services received (Agarwal 2018b)

84Health provider decision support via mobile devices (Agarwal 2018c)

85Health provider to health provider telemedicine (Gonçalves‐Bradley 2018b)

86Each of these interventions was also the topic of a recommendation in the WHO guideline (WHO 2019).

87As part of the WHO's guideline process, our qualitative evidence was used as a source of information about intervention acceptability and feasibility. The WHO technical team prepared GRADE evidence‐to‐decision tables for each recommendation. Each table included evidence from the relevant Cochrane Review of effectiveness. In addition, each table included evidence from this qualitative evidence synthesis regarding the acceptability and feasibility of each intervention. The WHO's technical team prepared these tables, with input from the review authors. The technical team and review authors of this synthesis also collaborated on a supplementary document presenting evidence about the acceptability and feasibility of all these interventions. The guideline panel used these tables and supplementary documents as the basis for their recommendations.

88While our review was not directly linked to the effectiveness reviews, the findings from our review may be used to shed light on the outcomes observed in the effectiveness reviews, by offering insight into contextual factors, including health worker preferences, that may have influenced outcomes, either positively or negatively. Furthermore, the findings from our review may be used to develop hypothesis for subsequent consideration and assessment in future effectiveness reviews, seeking to understand why some mHealth technologies are more effective than others.

89The review author team represents diverse professional backgrounds, with a range of research experiences and expertise that could have influenced their input in conducting this review. All of them are experienced qualitative researchers. Except for one review author (KD), everybody has had previous experience in conducting primary mHealth research in the context of primary healthcare services in low‐income settings in South Africa, and have published on this (Coetzee 2017; Leon 2012; Neupane 2014; Watkins 2018). FG has also experience in conducting telemedicine research in high‐income contexts (Griffiths 2017). Our experiences in conducting effectiveness studies and process evaluations of mobile health programmes, included positive, negative, and mixed results. This provided us with a good platform for engaging and understanding the complexities and nuances of qualitative research of mobile health interventions.

90The review authors reflected on the influence our perspectives might have on the conduct of the review, and in some cases tried to moderate this influence, in a number of ways. During the screening of abstracts/titles and full texts, the team constantly referred to each other to resolve conflicts, and in many instances a team decision was called upon. As is standard practice within qualitative research, the two review authors (KD, WO) who did the data coding, extraction, and synthesising, and wrote the findings, constantly discussed with each other how their own background and position, may have affected their analysis and writing of the findings.

91WO realised that at times his research experiences resonated strongly with some of the included studies, and was aware that this could lead him to give these data more importance than was due. Conversely, he was aware that he could be more dismissive towards data which contradicted his experiences. KD questioned the weight he attributed to certain data, ensuring that all data were equally represented in the final set of findings. WO and KD repeatedly questioned each other's interpretation of the data and how it fitted with the existing findings. They also called upon other members of the author team to verify that the findings were reasonable reflections of the supporting data. JAW, KD, and WO also used the same process of constant discussion and being aware of their personal perspectives when appraising their confidence in the findings. Finally, the contact editor of this review read each finding and its supporting data closely. She pointed to any mismatch between the supporting data and a finding, and critically engaged with our interpretation of the data, which led to a refinement of our analysis and writing of the findings.

92We screened 7225 records. Fifty‐three studies met our inclusion criteria. We purposively sampled 43 of these studies for inclusion in our analysis (Figure 1). All of the sampled studies were published between 2005 and 2018; see Methods section ‐ 'Sampling from the included studies' for a description of how we sampled these studies. In February 2020, we re‐ran the search strategies. We screened those records and 85 studies that we identified as potentially relevant are listed under Studies awaiting classification; we will assess these studies at the next update.

93In this section, we describe the studies that we sampled for analysis. For a more detailed description of each included and sampled study, see Characteristics of included studies. For an overview of the studies that we included but did not sample, see Table 2.

94In 17 (40%) of the sampled studies, the participants included both lay health workers and a range of professional health workers, such as nurses, paramedics, doctors, midwives, pharmacists, and laboratory staff. Non‐medical professional participants, found in four studies, referred to programme managers, administrators, decision makers, and social workers. A further 10 (23%) and 12 (27%) studies only included lay health workers or health professionals, respectively. In four (9%) of the studies, the participants were only described as ‘healthcare providers’.

95Thirty‐two (74%) of the sampled studies were from low‐ or middle‐income countries: Bangladesh (3), Brazil (1), India (4), Ethiopia (1), Ghana + Malawi (1), Ghana (2), Kenya (6), Lesotho (1), Rwanda (1), South Africa (4), Swaziland (1), Tanzania (3), Timor‐Leste and Myanmar (1 from each country), and Uganda (2). The remaining 11 studies (26%) were from high‐income countries: Australia (1), Canada (2), Ireland (1), Norway (1), Scotland (1), and USA (5). In 12 (28%) of the studies, the mobile devices were used to provide health care in clients' homes, and in nine (21%) of the studies, they were used in clinics. A further nine (21%) studies implemented in clinics plus another setting. Only two (4%) of the studies took place in a hospital, one (2%) in a general practice, and one (2%) in a school. The type of healthcare setting was unclear in the remaining nine (21%) studies. Most of the studies (14, 32%) were implemented in a rural setting, 11 (26%) in urban settings, and four (9%) in a rural and urban setting. Another four (9%) of the studies were conducted in rural and per‐urban communities, and in 10 (23%) of the studies, the setting was not detailed enough to classify.

96Mobile phones, also described in the studies as 'iPhones', or 'smartphones', were used in 31 (72%) of the sampled included studies, with either personal digital assistants (PDAs) and tablets in 11 (26%) studies. A combination of mobile phones and tablets was used in one study. Most of the mobile health interventions, in total 18 (42%) of the studies, comprised of software loaded onto the mobile device which guided the consultation process, for example, a screening algorithm that allowed health workers to diagnose clients. In the remainder of studies, the intervention comprised of in‐person and/or text message communication, collecting of surveillance data, and a range of other interventions, such as health promotion materials in the form of videos on the devices, accessing the Internet, and in one instance, allowing low‐level workers to send wound care images to higher‐level workers for treatment advice. The healthcare issue addressed through the mobile health programmes was in the main, maternal, neonatal and child health, with 17 studies (40%) reporting on this. Other healthcare issues included communicable and non‐communicable diseases, cardiovascular diseases, and intimate partner violence.

97Details of our assessments of the methodological limitations of the individual sampled studies can be found in Table 3. We assessed 10 studies as having no or very minor methodological limitations, 13 as having minor methodological limitations, and 14 and six studies respectively, as having moderate and serious methodological limitations. Of the 43 studies, 41 were published in peer‐reviewed journals, and two as full Masters theses (Barnabee 2014; van der Wal 2016). Most peer‐reviewed journals set a word limitation, which is not suitable for the comprehensive reporting of qualitative research. Our concerns regarding the general lack of rich data and thick description, as well as often poor descriptions of how participants were sampled and recruited, and of researcher reflexivity, may be attributable to these word limitations. Many studies provided poor descriptions of ethical considerations, apart from mentioning that ethical approval was obtained. All studies provided at a minimum a brief description of the study context, who the participants were, the mHealth programme, and to some extent, the study limitations.

98Out of 42 review findings, we had high confidence in 13 findings, moderate confidence in 18, low confidence in six, very low confidence in five (Table 1). Our explanation for each GRADE‐CERQual assessment is shown in the evidence profile in Appendix 4.

99From the synthesised data, we drafted 42 individual findings (Table 1), which we organised into four overarching themes. Theme 1 deals with how mHealth changed how health workers worked with each other, in particular through connecting lower‐level health workers with higher‐level health workers, and peers with each other. Theme 2 describes how mHealth changed how health workers delivered care, and includes health workers' perceptions and experiences about issues, such as accessing information from the Internet, providing care over distance, and using treatment algorithms. In Theme 3, we present how mHealth led to new forms of engagement and relationships with clients and communities, mainly because mHealth allows direct, and often bi‐directional communication between health worker and client. This theme also covers issues such as elevated health worker status that comes from health workers using modern technology and needing to protect client information on their devices. Theme 4 details how health workers' use and perceptions of mHealth can be influenced by factors tied to costs, the health worker, the technology, the health system and society, and how poor network access and poor access to electricity could make mHealth difficult.

100Finding 1: Through being connected to other health workers and across various healthcare services, health workers appreciated that mobile devices allowed them to better co‐ordinate the delivery of care (moderate confidence in the evidence)

101Mobile devices enabled real‐time communication between health workers (Henry 2016; Huq 2014; Lodhia 2016; Madon 2014; Messinger 2017; Murray 2015; Quinn 2013; Ramirez 2017; Rothstein 2016; Schoen 2017; van der Wal 2016; Watkins 2018), and across various healthcare services (Khan 2015; Quinn 2013; Toda 2017; Watkins 2018), for example between home‐based care and emergency services (Barnabee 2014; Chang 2011; Hampshire 2016; Huq 2014; Mwendwa 2016), using the short message service (SMS) on a mobile phone to send prescriptions on behalf of clients (Khan 2015), and ordering supplies using a mobile device (Chang 2011; Hampshire 2016): "When there is shortage of drugs, we put it on WhatsApp so that colleagues will inform us if they have excess for us to borrow. Some indicate which drugs they have run out of so that we will avoid referring patients there" (Hampshire 2016). Health workers found mobile devices particularly useful in emergency situations, such as calling an ambulance to transport a distressed client to a health facility (Barnabee 2014; Hampshire 2016; Mwendwa 2016), and reporting disease outbreaks to healthcare facilities (Henry 2016; Toda 2017): "… while I was attending a patient in [village A], I received a notification from a patient in [village B]. I cannot leave the patient...I used Liga Inan [mHealth application on the mobile phone] to contact the health center in [town C] and they came with an ambulance and took the patient (P2)." (Barnabee 2014). Improved co‐ordination also resulted from mobile devices, which allowed easier screening, diagnosing, and prioritising of clients (Lodhia 2016; Rothstein 2016; Schoen 2017; van der Wal 2016; Watkins 2018), provided that its accuracy was established (Lodhia 2016). Mobile devices also allowed collaborative care when members of a care team shared clinical data and treatment plans of clients, (Ramirez 2017; Rothstein 2016; Watkins 2018), which resulted in tailoring services to clients' needs (Murray 2015).

102Finding 2: Lower‐level health workers valued being able to reach higher‐level health workers via mobile devices, and perceived the advice and support they received as improving their care and as satisfying to clients. When higher‐level professionals responded in anger, it made lower‐level health workers reluctant to call them (moderate confidence in the evidence)

103Apart from facilitating contact between same‐level health workers (Finding 1), lower‐level health workers particularly valued being able to reach higher‐level health workers through mobile devices (Ayiasi 2015; Chang 2011; Cherrington 2015; Hampshire 2016; Huq 2014; Khan 2015; Lodhia 2016; Messinger 2017; Mwendwa 2016; Quinn 2013; Toda 2017; van der Wal 2016; Watkins 2018). They received advice and support from the higher‐level health workers, such as when the latter was asked to intervene with clients refusing treatment (Ayiasi 2015), or had to advise treatment (Cherrington 2015; Huq 2014; van der Wal 2016): "… senior staff nurse, [higher‐level worker] commonly wants to know the detail of patient's condition and then advises [us lower‐level workers] over phone" (Huq 2014). Lower‐level health workers appreciated the immediacy of this support and advice (Chang 2011; Cherrington 2015; Madon 2014; Watkins 2018). These workers perceived this as improving the quality of their care and health outcomes (Ayiasi 2015; Chang 2011), as well as being satisfying to clients (Ayiasi 2015; Cherrington 2015): "They [clients] would feel like, "Oh they care", because if it was something that I didn't know the answer to and then, (someone) would get back with them" (Cherrington 2015). The exchange broke the hierarchy between health worker categories, when previously unreachable professionals could more easily be reached (Chang 2011; Huq 2014; Khan 2015). In one study, health workers perceived direct contact as improving relationships between lower‐ and higher‐level workers (Chang 2011). In contrast, when higher‐level professionals responded in anger, it made lower‐level health workers reluctant to call them (Ayiasi 2015).

104Finding 3: When higher‐level health workers failed to respond and support lower‐level workers through mobile devices, lower‐level staff had negative perceptions of these devices. One study emphasised the importance of having health professionals' buy‐in with mobile health to ensure that mobile devices were optimally used to support lay health workers (moderate confidence in the evidence)

105The data suggests that those participants who expressed an opinion desired good communication and co‐ordination with their seniors and others in their health systems context. Lack of anticipated co‐ordination, support and responsiveness, in particular in emergency cases (Mwendwa 2016), through mobile technology's connectedness with higher‐level staff or emergency services, led to a negative experience of the intervention amongst lower‐level health workers (Cherrington 2015; Huq 2014; Mwendwa 2016;Quinn 2013; Toda 2017): [Lower‐level health worker]: "The technology is good but let the higher levels take it seriously otherwise there is no need of sending the instant reports. The last time is [sic] sent a suspected tetanus baby but he died without anyone coming to see the child at the facility I had referred to" (Toda 2017). This lack of responsiveness was described as limiting the effectiveness of the intervention (Mwendwa 2016). In contrast, direct communication was expressed as having a positive impact on the facility and in turn on client experience. In one instance, a lack of managerial interest in mHealth‐facilitated disease surveillance demotivated the lower‐level workers from consistently using mHealth for their work (Toda 2017). In one study (van der Wal 2016), matters were complicated when the supervising doctor was not given a smartphone, as he could not supervise what the lower‐level health workers were doing with the phones. His lack of a phone led to his lack of supporting these workers and encouraging them not to use the intervention application either. In turn, these lower‐level workers wanted him to be given a phone, not only for his buy‐in, but also because they believed that this would enhance how they worked together.

106Finding 4: The use of mobile devices allowed some health workers to feel connected to their peers within their own organisations. However, others preferred face‐to‐face communication with their peers (moderate confidence in the evidence)

107Some health workers reported positively on being connected with their peers within their respective organisations through mobile devices (Barnabee 2014; Hampshire 2016; Watkins 2018), which some perceived as supportive (Henry 2016; Jennings 2013; Madon 2014). This was particularly the case when they were seeking advice to deal with clients experiencing complications (Barnabee 2014; Henry 2016): "In case you are stuck, you don't know what to do here, you don't know what to do anyway, you just communicate immediately [with other public health officers] and you will get the information immediately. So it has really made our work to be easy" (Henry 2016). However, others preferred face‐to‐face connection with their peers (Valaitis 2005; van der Wal 2016): "If people weren't coming into the office as regularly, then we would have to meet as a team more often just to do that informal [peer connection]"..."I've always come in once a week, because I'm dying to see everybody" (Valaitis 2005).

108Finding 5: Some health workers relayed that mobile devices improved their reporting to supervisors and encouraged them to report more truthfully. Others compared mobile device‐facilitated supervision to "big brother watching". Some supervisors thought that mobile devices allowed them to better identify staff who needed support (moderate confidence in the evidence)

109Some health workers felt that mobile devices improved reporting to their supervisors (Barnabee 2014; Medhanyie 2015), their relationship with their supervisors (van der Wal 2016), and encouraged truthful reporting (Chang 2011). On the other hand, there were supervisors who did not think that mobile devices were a safeguard against false reporting (Jennings 2013). Mobile health‐facilitated supervision left some supervised workers with a sense of "big brother [is] watching" (Valaitis 2005). Mobile health resulted in work being more visible to the supervisors (Henry 2016). In instances where clients complained about health workers who did not visit them, workers reverted to having clients signing a paper record for such proof, and suggested electronic signatures be made available on their devices (Schoen 2017). In the context of conducting disease surveillance, health workers expressed a need for face‐to‐face interactions with those overseeing the surveillance (Toda 2017). Some supervisors expressed that mobile technology‐facilitated supervision allowed them to be more aware of their staff's work, in particular when the latter experienced problems. These supervisors perceived this increased awareness of staff's performance as positive because they could address the problems that came to their attention as a consequence (Henry 2016; Madon 2014; Mwendwa 2016): "[WhatsApp] has made me learn a thing or two, it has made me get to know characters as far as community health volunteers are concerned … I can even gauge performance when it comes to community health volunteers" (Henry 2016).

110Finding 6: Health workers had positive experiences with using instant messaging through WhatsApp. This application was seen as cheap and suitable for a range of activities, such as communicating with peers and posting photos as evidence of work done (very low confidence in the evidence)

111Some health workers valued instant messaging through WhatsApp (Hampshire 2016; Henry 2016; Schoen 2017), because it was perceived to be a cheaper way to communicate, compared to using short message services (SMS) (Henry 2016). They used WhatsApp for a range of activities, including communicating with peers, notifications when drugs were out of stock, supervision, and posting photos as evidence of work done (Hampshire 2016, Henry 2016): "It [WhatsApp] has been the best evidence. If I assess a child, I take a photo, or if I have attended any sick person in the community and I post on WhatsApp. If we [in Kibera] don't post, they will say in Makueni we don’t work" (Henry 2016). Some created groups on WhatsApp to serve different projects and interests (Henry 2016).

112Finding 7: Even when health workers received messages that were automated, rather than sent directly from a manager or supervisor, this was still experienced and responded to, as a kind of supervision. Some lower‐level health workers experienced it as supportive to their work, while others felt guilty for not providing correct care as per these messages (low confidence in the evidence)

113Automated text messages about illness and client management, and in one study, motivational messages (Jones 2012), sent to lower‐level health workers' mobile phones, were perceived by some of them as supervision (Jones 2012), and they felt it improved their care and knowledge (Cherrington 2015; Ilozumba 2018; Jones 2012; Mwendwa 2016). Workers valued the messages' conciseness, and saw it as providing up‐to‐date information and as being a useful reminder to provide correct treatment (Jones 2012): "It kept me on task...if I forgot, I would turn it on and it would pop up, "You're late"...it was wonderful" (Cherrington 2015). Some felt motivated by receiving automated treatment messages, but others felt guilty for not providing correct care as recommended by these messages (Jones 2012). There were concerns that the text messages were too repetitive in the information they carried, and that motivational messages on their own, without treatment guidance, were less meaningful than those with treatment guidance (Jones 2012).

114Finding 8: The task optimisation enabled through mHealth interventions was widely valued by health workers (moderate confidence in the evidence)

115Through the use of mobile devices, health workers were able to expand their current range of tasks, at their own level (Chang 2011; Ilozumba 2018; Khan 2015), as well as take on tasks previously assigned to higher‐level workers (Barnabee 2014; Kolltveit 2017; Lodhia 2016; Praveen 2014): "In this setting ASHAs [community health workers] became proficient in not only performing risk factor measurements, but also in interpretation of the results: Earlier, I just used to go and measure BP [blood pressure], but with this tablet, I came to know what was a normal reading and how the actual reading differs from normal readings" (Praveen 2014), and conducted these tasks independently (Chang 2011). They experienced this as beneficial in improving their skills (Barnabee 2014; Ilozumba 2018; Khan 2015; Praveen 2014), and perceived it as addressing staff shortages (Lodhia 2016). Some health professionals reported that clients with mobile devices first contacted lay health workers before contacting them, which allowed the professionals to focus on clients more in need of their support: "The use of mobile phones typically led to more Patient–PHW [peer health worker, working at lower level than professional staff] communication and less Patient‐Staff." One staff member noted: "Instead of calling us, they first call the PHW … If the PHW can solve it, they don’t bother to call us [higher‐level facility staff]. If they cannot solve it, then the PHW calls us" (Chang 2011). The new tasks that mHealth allowed health workers to perform, were reported to facilitate improved interaction with clients (Barnabee 2014).

116Finding 9: At times, health workers used their mobile devices to access the Internet for health information, and found it useful when they were with clients who needed the information. This interaction also included health workers providing clients with additional information beyond the healthcare intervention. But, if the only way that health workers could access online information, required them to use their own money to purchase data, then this could be prohibitive to them accessing such information (low confidence in the evidence)

117Some health workers used their mobiles devices to gain access to the Internet where they accessed health information, clinical guidelines, health promotion material, and other information thought to be needed by clients (Bacchus 2016; Hampshire 2016; Schoen 2017; Watkins 2018): "Doctors and nurses who used the Internet for work reported using search engines on their phones such as Google, to access clinical information on diseases or prescription drugs. A few nurses reported accessing the digital versions of government clinical guidelines" (Watkins 2018). They found the quick access to such information useful, particularly when they were with clients who needed more information about a certain condition and its treatment (Hampshire 2016). The cost of data, when borne personally, sometimes prevented health workers from searching for information (Watkins 2018). In one study, health workers used their tablets to give non‐health information to clients who wanted to further their education, and thus needed access to information to enable reaching this goal: "We don’t have tablets usually so I used the tablet to do some personality tests of my clients who wanted to be in school” (Bacchus 2016).

118Finding 10: mHealth held the promise of increasing service efficiency for many health workers, but the experience of whether this promise was borne out in practice, varied in the accounts of health workers. It was experienced as efficient if it improved feedback, speed and workflow, but inefficient when the technology was slow and time consuming. Some were concerned that if mHealth was too efficient, making work faster, that this may justify staff cutbacks (high confidence in the evidence)

119Health workers' experiences of the efficiency of mobile devices varied across and within studies. In general, efficiency was related to work being done more quickly, whilst inefficiency was related to the extra time and work it took when using these devices. Health worker accounts of speed efficiency related to enhanced communication and information flow (Ayiasi 2015; Cherrington 2015; Hao 2015; Ramirez 2017; Rothstein 2016; Schoen 2017; Tewari 2017; Valaitis 2005), because of being able to send information quicker (Chang 2011; Madon 2014), and getting immediate or faster feedback and support from peers (Barnabee 2014; Chang 2011; Huq 2014; Lodhia 2016; Madon 2014; Schoen 2017), higher‐level health workers (Ayiasi 2015; Medhanyie 2015), and laboratories, programme managers, and facility staff (Barnabee 2014; Chang 2011; Huq 2014; Madon 2014; Schoen 2017): "Now for me, a PHW with a phone, I can't be like that PHW [peer health workers] without a phone. His/her information cannot [travel] as fast as mine who has a phone" (Chang 2011). Speed efficiency also related to being able to quickly contact clients or vice versa (Cherrington 2015; Coetzee 2017; Hao 2015; Jennings 2013; Messinger 2017), particularly when this meant not having to travel (Ayiasi 2015; Barnabee 2014; Cherrington 2015; Hampshire 2016). Mobile health was also perceived to be more efficient because it improved workflow (Garg 2016; Praveen 2014; Rothstein 2016; Surka 2014), provided follow‐up reminders to health workers (Jones 2012; Rothstein 2016; Schoen 2017), and allowed them to provide immediate feedback to screened clients (Lodhia 2016; Watkins 2018). In one study, health workers were concerned that the ability to work faster because of mobile health was used by management to support staff cutbacks (Valaitis 2005). There were also health workers who experienced mobile health overall, or in part, as inefficient. This included perceptions of the technology as slow, time consuming, and increasing workload: "It complicates and increases our work. One has to stop everything she is doing and concentrate when sending reports" (Mwendwa 2016). The perceived increase in workload was in part because it was experienced as more cumbersome and taking longer to complete work (van der Wal 2016), compared to standard practise (Ginsburg 2016; Hao 2015; Kolltveit 2017; Mwendwa 2016), and sometimes because the application was slow (Schoen 2017). Mobile health also increased workloads when better screening procedures resulted in detecting more clients who needed treatment (Lodhia 2016).

120Finding 11: Health workers frequently reported mobile devices as overcoming the difficulties of rural and geographically challenging contexts when it made it possible for them to provide health care without having to travel. Some reported that reducing travel time allowed them more time with their clients (high confidence in the evidence)

121Health workers in rural and geographically challenging contexts appreciated the efficiency of mobile devices in allowing them to offer a service despite these circumstances, because it saved them travelling to clients and health facilities (Chang 2011; Hampshire 2016; Henry 2016; Lodhia 2016; Messinger 2017; Mwendwa 2016; Quinn 2013; Toda 2017): "...those who are in the most remote areas who have the highest prevalence for blindness will now be linked to the health system and so people will be able to find them and treat them" (Lodhia 2016). It was also used to schedule visits in advance which avoided wasteful travelling (Hampshire 2016). Some workers pointed out that finding clients in these contexts still required being provided with detailed client information on mobile devices (Rothstein 2016). Using these devices saved travelling time in an urban setting too, which allowed health workers to spend more time with their clients (Valaitis 2005).

122Finding 12: Health workers appreciated the portability and work schedule flexibility of mobile devices (moderate confidence in the evidence)

123Health workers had positive views and experiences about the portability of mobile devices. This allowed them flexible working hours, and allowed them to be less office‐bound (Hampshire 2016; Murray 2011; Nguyen 2015; Orchard 2014; Ramirez 2017; Schoen 2017; Toda 2017; Valaitis 2005; van der Wal 2016). Being able to carry mobile phones allowed them to access clients' records when visiting them (Murray 2011; Ramirez 2017; Schoen 2017; van der Wal 2016): "Another example, let's say a community member has a question about an appointment from a few months ago, like when it was. I can just pull it up right away!" (Ramirez 2017). In another study, school nurses expressed their appreciation of not being office‐bound as follows: "I was able to forward an email that had valuable information to my student, right from my school, and right to their home and office email. Otherwise, who knows when I could have sent [it]"; and "[mobile computing ‐ MC] does allow me freedom in my work. It helps to balance [the] workload, since I have access to files more frequently, therefore allowing me to work at times that in the past I was unable to... in particular while sitting in schools during a down period" (Valaitis 2005).

124Finding 13: Through mHealth, health workers were able to use treatment and screening algorithms that were loaded onto mobile devices. Their perceptions of using these electronic algorithms ranged from finding it easy and useful, to threatening their clinical competency, and an information overload. There were also some concerns that erroneous data entry may lead to wrong treatment guidance (high confidence in the evidence)

125Several mobile health programmes comprised of treatment and screening algorithms loaded onto the devices (Ginsburg 2016; Lodhia 2016; Orchard 2014; Ramirez 2017; Rothstein 2016; Shao 2015; Surka 2014; van der Wal 2016). In other instances, mobile devices were preloaded with the information that health workers had to share during client consultations (Ilozumba 2018; Mitchell 2012). Health workers often found it easy to integrate this into routine care (Ginsburg 2016; Lodhia 2016; Tewari 2017), and that it reduced inaccurate consultation and data capturing procedures (Ginsburg 2016; Ilozumba 2018; Mitchell 2012; Mwendwa 2016; Nguyen 2015; Orchard 2014; Shao 2015; Surka 2014; van der Wal 2016): "They [health workers] trusted the results of the mPneumonia algorithm, reporting that "the machine does not tell lies, it will rather tell you the right thing to do"" (Ginsburg 2016), and useful because it guided and simplified providing care (Ginsburg 2016; Mitchell 2012; Rothstein 2016; Shao 2015; van der Wal 2016), which workers experienced as reassuring and improving their knowledge (Ginsburg 2016; Ilozumba 2018; Lodhia 2016; Mitchell 2012; Orchard 2014; Rothstein 2016; Shao 2015; van der Wal 2016). Contrary to these experiences, other health workers held negative perceptions of using algorithms, as they felt it too prescriptive, and were concerned that they may lose their clinical competencies by blindly following it (Mitchell 2012; Surka 2014): “I enjoyed using the chart [paper‐based system] because you could check by yourself and see the status of the participant but when using the phone, it works out everything, it does not tell you what is wrong with the participant" (Surka 2014), and that it was too comprehensive and time consuming (Ginsburg 2016). In one study, health workers were concerned that caregivers would not understand, and by implication be dissatisfied, when the directive of the algorithm was counter to their treatment expectations (Mitchell 2012). There were also concerns that erroneous data entry may lead to wrong service reminders (Nguyen 2015).

126Finding 14: Using mobile devices to record routine client or surveillance data was mostly perceived by health workers and their managers as helpful for decision making, and increasing community and health worker appreciation of these data (moderate confidence in the evidence)

127The use of mobile devices to record routine client (Khan 2015; Rothstein 2016; Schoen 2017), and/or surveillance data (Lodhia 2016), was perceived by health workers and their managers as helpful for the continuity of care (Khan 2015; Madon 2014; Ramirez 2017), decision making and resource planning (Lodhia 2016; Madon 2014; Rothstein 2016), information sharing (Murray 2011; Nguyen 2015), and responding to disease outbreaks (Toda 2017): "At the district level, the majority of officers we interviewed were also pleased with the NTD MIS [software on the mobile phone to collect data at point of source] as reflected in the following comment made by one officer, "I can sit in my office and make a decision based on what is in the database"" (Madon 2014). In one study, the data sharing between different services was regarded as achieving a policy goal (Murray 2011). Workers reported specific advantages to data generated through mobile devices compared to paper‐based reports, which included that it was easy to format the reports to their needs, such as viewing individual or aggregated data (Nguyen 2015; Rothstein 2016; Schoen 2017), and having visual presentations of trends (Rothstein 2016). Mobile devices also offered continued access to data, which allowed timely adjustments of services in response to data trends (Lodhia 2016; Rothstein 2016; Schoen 2017; Toda 2017). Managers described automated visual presentation of data as important where workers struggled with analytical capacity, and the "ready cooked", graphical presentation then made for easier visualisation of what they needed to know (Rothstein 2016). A further advantage of using mobile technology for data collection was that it raised awareness of the value of data for decision making amongst community members (Nguyen 2015), and lower‐level health workers (Rothstein 2016): "Geohealth [mHealth intervention] … helps me make lists of community members in my area. I can choose, for example, to see only the pregnant women in my area, or only the 2 year olds, and then I have a complete list right away" (Rothstein 2016). One programme set automatic thresholds for accuracy and completeness (Rothstein 2016). If these were not met, then reports could not be generated, and this lack of generation of reports was described as frustrating (Rothstein 2016). Some workers suggested additional feedback mechanisms to counter delays in receiving data caused by disrupted connectivity, and recommended having dedicated staff for data capturing (Rothstein 2016).

128Finding 15: In most cases, health workers perceived mobile health as more advantageous than paper. However, some continued to prefer paper (high confidence in the evidence)

129The majority of health workers mentioned advantages to their use of mobile devices, compared to using paper‐based systems (Coetzee 2017; Madon 2014; Mitchell 2012; Nguyen 2015; Surka 2014; Toda 2017; Valaitis 2005; van der Wal 2016). The advantages included convenience (Nguyen 2015; Rothstein 2016; van der Wal 2016; Vedanthan 2015), quicker recording of their work (Ginsburg 2016; Mitchell 2012; Rothstein 2016; Valaitis 2005), easier access to client data (Nguyen 2015; Schoen 2017; Surka 2014; Vedanthan 2015), reducing and easy correction of recording mistakes (Madon 2014; Schoen 2017), faster error alerts (Nguyen 2015), not having to carry heavy paper stationery (Madon 2014; Schoen 2017): "In our work we walk a lot, and our backpacks are heavy, and we end up taking off our backpacks in community members' homes and forgetting them there. Biggest difference between Geohealth [mHealth programme] and paper? My bag is lighter with Geohealth! I don’t have to carry as much paper" (Schoen 2017), ease of moving through algorithms (Mitchell 2012), ease of information transmission and sharing (Mwendwa 2016; Rothstein 2016; Toda 2017; van der Wal 2016; Watkins 2018), immediate recording of data and therefore reducing of errors (Nguyen 2015), quicker access to summaries of reported data (Nguyen 2015; Rothstein 2016; Schoen 2017; Surka 2014), and a more durable platform compared to paper‐based systems, for example with papers that can be damaged by rain (Nguyen 2015; van der Wal 2016). For some workers, these benefits of mobile devices over paper‐based systems, as well as the perceived improvement of care, led to a commitment to use these devices (van der Wal 2016). In one study, being issued with mobile devices made health workers feel more professional compared to when they used a paper‐based system (Valaitis 2005). Yet, in one study, it was found that despite the mentioned advantages, most of the health workers did not use these devices during their visits to clients (Schoen 2017). Some workers preferred paper‐based systems, as they perceived these as safer to store information, more flexible, and had concerns about malfunctioning technology (Bacchus 2016; Schoen 2017, Valaitis 2005; van der Wal 2016): "One participant looked on it as "a totally negative thing, because I’m a paper person and I trust the paper in front of me. I do not lose paper"" (Valaitis 2005). These concerns resonated with those of health workers who complained that mobile health applications were slower than their paper‐based systems (Schoen 2017), and that it was easier to correct errors on paper forms because this did not require technical knowledge (Nguyen 2015).

130Finding 16: mHealth interventions sometimes required health workers to perform tasks that were peripheral to regular service delivery, such as registering clients onto the system. These more menial tasks were sometimes regarded as undermining to professional staff (very low confidence in the evidence)

131In some instances, the use of mobile devices to deliver or support healthcare services resulted in tasks that were additional to health workers' routine care practices. These tasks included registering clients onto the mobile health system (Hirsch‐Moverman 2017; Medhanyie 2015), and being called upon by clients to respond to technical problems or assist them with service plans for their devices (Murray 2015). Some workers perceived the additional tasks as menial and not appropriate for their job level: One respondent [health worker] commented that MomConnect registration [mHealth programme] was not seen as in line with the scope of professional work of a nurse: "The way I take it, it's undermining. Professionals don't do this, really." (Wolff‐Piggott 2018), and recommended outsourcing technical support for clients with technical and service plan problems (Murray 2015).

132Finding 17: Some health workers experienced the use of mHealth as generating an extra workload when, for instance, it resulted in reaching more clients needing care, or having to maintain both a mobile health ‐ and paper system. Some workers disliked this, particularly when their superiors did not perceive their mobile health work as part of their job description. Others did not object to the additional work, yet others wanted to be remunerated (high confidence in the evidence)

133There were several ways in which health workers perceived that mobile health programmes added to their routine healthcare services (Kolltveit 2017; Murray 2015). This included adding more steps to an existing mobile intervention (Chang 2011), inputting data (Mwendwa 2016), and reaching clients who may previously have been missed, but now because of being found created more demand on themselves and the health services (Rothstein 2016; Lodhia 2016). In one instance, the underlying values of the facility of building relationships with clients, created a greater demand on health workers' time, as they felt a need to immediately respond to text messages from clients (Murray 2015). Health workers held mixed views about this increased workload. Some held negative feelings when it complicated their work by maintaining two systems (Kolltveit 2017; Mwendwa 2016; Rothstein 2016; Shao 2015). They also disliked it when the addition of the mobile health interventions to current work was not understood and appreciated by supervisors: "… there were some accounts of hostility from the ANM [supervisor]:...she (the ANM) scolded me and said that I am giving too much importance to this work...I said I am doing both jobs...and it is my problem...but I made 1 mistake...I regretted not showing this tablet to her" (Praveen 2014), or when they themselves perceived the intervention as peripheral to their work (Wolff‐Piggott 2018). Other health workers were neutral about maintaining paper‐based and electronic records (Mwendwa 2016; Schoen 2017): "… besides having to use the electronic devices, the registers had to be filled too "…yeah, it adds more work to us, because we have to enter the data in the phone, and then write the same data in the file, or patient's card/notebook." (Shao 2015). Some health workers did not object to the additional work (Hao 2015; Murray 2015), whilst others expected to be remunerated for the additional work (Hao 2015).

134Finding 18: Through mobile devices, health workers and clients could communicate directly with each other, which health workers reported as improving care and their relationship with clients. When clients initiated the contact, health workers felt that clients took ownership of their health. Health workers felt that some clients still warrant face‐to‐face contact (moderate confidence in the evidence)

135Mobile devices facilitated two‐way communication between all categories of health workers and clients (Chang 2011; Garg 2016; Hampshire 2016; Hirsch‐Moverman 2017), and kept health workers informed about the clients' conditions (Barnabee 2014; Huq 2014). Some workers perceived this to lead to immediacy of care (Barnabee 2014; van der Wal 2016), enabling follow‐up of missing clients (Hampshire 2016), informing care options (Barnabee 2014; Huq 2014; Messinger 2017), and advice and emotional support to clients (Cherrington 2015; Hampshire 2016), when physical contact was not possible. Health workers perceived direct communication with clients as facilitating a trusting relationship with clients (Murray 2015). It also allowed clients to initiate contact with health workers (Hampshire 2016; Jennings 2013; Messinger 2017; Murray 2015): "One woman, who came for [contraceptive implant] took my number. When she does not understand anything, she calls me for clarification" (Hampshire 2016). Workers perceived this as empowering clients to take responsibility for their health (Murray 2015). Some workers allowed the clients to lead the conversation during their communication, which enabled workers to tailor their services to clients' needs (Murray 2015). Mobile devices also allowed workers to contact clients’ relatives when needed (Jennings 2013; Lodhia 2016; Schoen 2017). However, it was felt that some instances still warrant face‐to‐face contact between worker and client (Jennings 2013; Messinger 2017): "Sometimes [counselling] can be taken through phone. But there are problems [with doing so]. Will it be right to give treatment without seeing the client?" (Messinger 2017).

136Finding 19: Health workers were aware of the importance of protecting confidential client information when using mobile devices, and the confidentiality risks in cases of stolen phones and using their SIM cards in colleagues' phones. Health workers were alert to clients' concerns when they shared personal information concerning stigmatised issues, such as HIV/AIDS and interpersonal violence, and suggested ways to keep the information confidential. They emphasised building a trusting relationship with clients prior to using the devices (high confidence in the evidence)

137Health workers were conscious of protecting clients' confidential information (Garg 2016; Hirsch‐Moverman 2017; Lodhia 2016; Murray 2015; Rothstein 2016; Valaitis 2005), complying with legislation regarding the protection of client information (Garg 2016), during consultations in which mobile devices were used. Workers were also aware of clients’ anxieties as to how their information will be used during mHealth‐facilitated consultations (Coetzee 2017), particularly when the information they provided concerned stigmatised and sensitive issues, such as HIV/AIDS and interpersonal violence (Bacchus 2016; Hirsch‐Moverman 2017; Wolff‐Piggott 2018), or showed clients in a bad light because of their behaviour: "Several CHWs reported that some clients worried that the devices were being used as voice or video recorders. "I thought that they [clients] were going to think that you will record them and take pictures of them. I thought they were going to say that as we [clients] drink this way […], you are now going to record us and take pictures of us"" (Coetzee 2017). A lack of legislative clarity on data protection caused some uncertainty for health workers in how to proceed with sharing information via text messages with clients (Garg 2016). Health workers were uncertain about probing clients on sensitive issues that these clients received information about through the device, but may not have wanted to talk about further with the health worker (Bacchus 2016). Their narratives suggest that this was a dilemma to their interpersonal interaction with the client (Bacchus 2016). In this study, mobile devices were used to prompt clients about their experiences of intimate partner violence, with some health workers perceiving these devices as a barrier to engage with clients, whilst others found it helpful (Bacchus 2016). Health workers perceived that some clients did not want sensitive information sent to their personal devices, as they were concerned about the transmission of the information and having it on the device where others may accidentally see it (Hirsch‐Moverman 2017). Measures to protect client information ranged from encryption (Lodhia 2016), using shared secret code words in text messages to clients (Hirsch‐Moverman 2017; Murray 2015): “I text him [client] and say, "It is your time now. Have you remembered your food?" He already knows. I will have taught him that when I say that, I mean it’s time to take his pills" (Hirsch‐Moverman 2017), and an explanation to clients on how their information will be protected (Lodhia 2016). Some health workers preferred sending automated text messaging to clients from a non‐traceable number, rather than using their personal phones to send these messages (Murray 2015). Other health workers pointed to the importance of building a trusting relationship with clients prior to using these devices, perceiving this as a way to mitigate clients' concerns about confidentiality (Coetzee 2017). Health workers were also concerned about risks to client information in cases of stolen phones, and when using their SIM card, with client information, in colleagues' phones (Lodhia 2016; Mwendwa 2016).

138Finding 20: Health workers were concerned that concentrating too much on the mobile technology during client consultations could be to the detriment of their service and interaction with clients (low confidence in the evidence)

139Some health workers were concerned that concentrating too much on the devices during client consultations could distract them to the detriment of their service and interaction with clients, particularly when it resulted in loosing eye contact and missing non‐verbal cues from clients (Bacchus 2016; Schoen 2017; Vedanthan 2015): "The patient likes to talk to you directly but you, you just concentrate in the gadget" (Vedanthan 2015). Some experienced clients as being dissatisfied with the loss of interaction and eye contact due to the workers concentrating too much on their devices (Schoen 2017; Vedanthan 2015): "The community members, especially the older ones, complain. They say, look at me! I'm telling you a story, pay attention!" (Schoen 2017).

140Finding 21: Health workers had differing reactions to being contactable via mobile devices during and outside of working hours: some felt it was useful, some were ambivalent about it, and others objected to it. Workers suggested setting boundaries to protect themselves from this (moderate confidence in the evidence)

141Clients with access to health workers contact numbers, were able to contact them at all hours (Cherrington 2015; Hampshire 2016). Some health workers perceived this as useful in emergency cases (Chang 2011; Hampshire 2016), some were ambivalent about it (Huq 2014), and others disliked it (Hampshire 2016): "Last year a patient called me at 5 am. I was deeply asleep when she called and said she was having menstrual pains and did not know what to do" (Hampshire 2016). Some wanted to protect their privacy by setting boundaries to protect themselves from working outside of working hours (Valaitis 2005), and did not give their contact details to clients (Hampshire 2016; Schoen 2017): "I don't give my WhatsApp number to my community members because I need a barrier between personal and work life—it doesn't work out to mix them. I have to cut it off somewhere" (Schoen 2017).

142Finding 22: Health workers experienced the use of mobile technology to provide health care, as being met with both trust and skepticism from clients and the communities they served. They described how trust or skepticism in the device was translated into trust or skepticism of their service when using the device. Some found that using mobile devices raised their social status with clients, and even their families. Others were concerned that using expensive equipment would emphasise inequity between themselves and clients (high confidence in the evidence)

143Health workers reported that mHealth devices raised their social status (Coetzee 2017), increased their recognition from family and friends (Madon 2014; van der Wal 2016), and increased the trust and respect they received from clients (Ayiasi 2015; Coetzee 2017; Ilozumba 2018; Lodhia 2016; van der Wal 2016): "Now we are famous because of our work. People listen to what I tell them, they respect us because we have brought services nearer to the people" (Ayiasi 2015). This was in part due to the connotations, such as prestige, innovation, and trustworthiness, attached to the devices, (Ginsburg 2016), the applications on the devices (Ginsburg 2016), and the purpose for which the devices were used (Coetzee 2017; Ginsburg 2016). These connotations were sometimes transferred to the health workers, such as them being perceived as trustworthy as they were entrusted with expensive and modern devices (Coetzee 2017; Ilozumba 2018; Madon 2014; van der Wal 2016), or because these devices made them more thorough (Barnabee 2014; Ilozumba 2018; Madon 2014; Mitchell 2012; van der Wal 2016). Being seen as a trustworthy health worker made clients more receptive to workers' use of mobile devices (Ilozumba 2018): "I am not telling. The mobile is telling. They will hear its words. Whoever is nearby they also become silent and will here all things" (Ilozumba 2018). Some health workers believed that using mobile devices helped them build a relationship with clients, and that satisfied clients told each other about the quality of care they received through mobile devices (Jones 2012; Khan 2015). This encouraged more clients to come and see them (Khan 2015). The elevated status, trust and respect were also linked to their use of these devices to access higher‐level care (Ayiasi 2015; Khan 2015; Mwendwa 2016), perceived by some clients as health workers showing care (Ayiasi 2015; Cherrington 2015), being credible (Ayiasi 2015), and being immediate in their support (Cherrington 2015). Health workers said that some clients were attracted from outside their catchment areas because of the mobile devices they used. Some clients not having the illness condition for which the health workers used the devices for, would question why they were not treated with these mobile devices (Vedanthan 2015). In contrast, there were health workers who experienced clients and communities as less responsive if they were sceptical as to the devices and how it can be used (Lodhia 2016; Mwendwa 2016): "Some mothers still do not know Rapid SMS and do not understand how useful the system is so they are uncooperative and do not give us information" (Mwendwa 2016). Workers suggested that this skepticism be addressed through community education (Lodhia 2016). There were also concerns amongst workers that using expensive devices working in resource‐constrained communities would emphasise the social inequity between clients and health workers, and would impact negatively on connecting with underprivileged clients (Valaitis 2005).

144Finding 23: Health workers experienced clients as having an opinion not only about their use of mobile devices, but as having an opinion on the devices themselves, which influenced how they responded to care delivered with the support of these devices. Health workers ascribed clients' enthusiasm for mobile devices as due to these clients' perception of the devices as prestigious, offering trustworthy information, and providing confidentiality. They perceived clients as more receptive when these clients were familiar with the devices used. There were concerns that clients who felt that the use of these devices during care was too time consuming, and would respond negatively to its use (moderate confidence in the evidence)

145Health workers felt that some clients were enthusiastic about the use of mobile devices in support of their care, as they perceived it as prestigious and modern (Lodhia 2016; Mitchell 2012; Valaitis 2005; Vedanthan 2015), offering credible information (Ginsburg 2016; Ilozumba 2018; van der Wal 2016), and made health workers more thorough (Mitchell 2012). Some workers described how it attracted more clients (Jones 2012; Messinger 2017; Vedanthan 2015), and clients trusted the accuracy of mobile devices, with some perceiving the addition of these devices during care, to be better than standard care (Ilozumba 2018; Khan 2015; Mitchell 2012; Shao 2015). Discontinuation of the use of mobile devices caused problems with new and returning clients, who expected it to be used (Mitchell 2012). Some workers perceived clients to be more willing to report sensitive information through mobile devices, than doing so verbally or on paper (Bacchus 2016; Westergaard 2017): "So if they feel safe enough to do it on the tablet, feeling like it's a little anonymous, it starts to break down those walls and maybe next time they'll want to talk about it" (Bacchus 2016). Workers also reported clients' preference for mobile devices when these offer non‐invasive medical diagnosis, as compared to invasive diagnosis, such as drawing blood (Ginsburg 2016). They also perceived clients to be more receptive to mobile health, if they were already familiar with the type of device used by the health workers (Garg 2016; Lodhia 2016). In contrast to the positive reactions, health workers were concerned that clients who perceived using mobile devices during care, as too time consuming, would respond negatively to such devices (Ginsburg 2016; van der Wal 2016): "The few negative reactions from clients related to them being impatient with the lengthy consultations. [Health worker]: Before, we mainly asked about danger signs but now we have many questions… some patients were impatient to answer all the questions" (van der Wal 2016). Other health workers reported that clients, particularly the elderly, disliked the use of mobile devices during consultations (Schoen 2017).

146Finding 24: Some interventions required clients to have phones as well as health workers. Health workers described this as challenging for multiple reasons, including clients not having phones, changing their phone numbers regularly, not knowing how to use a phone, being a target for crime because of possession of the phone, and women being prohibited from accessing phones. Health workers suggested competitive pricing to increase clients' accesses to phones, and to issue clients with phones (moderate confidence in the evidence)

147Health workers identified that interventions that required communication between health workers and clients, might pose several challenges to the clients. These included clients who regularly changed their phone numbers without informing the health worker (Hirsch‐Moverman 2017): "For others you find that the patient has given you a certain number, in a blink of an eye he has changed it without telling you that he doesn't use that number anymore" (Hirsch‐Moverman 2017); clients who did not have phones (Chang 2011; Tewari 2017; Wolff‐Piggott 2018), or did not always have their phones with them (Tewari 2017); clients who did not have money to buy phone credit or access to electricity (Huq 2014; Wolff‐Piggott 2018); and clients who were afraid of being robbed of their phones: "some of them …. they say they can't have these telephones because they come very early." It was explained that women queuing in the dark might be targets for criminals (Wolff‐Piggott 2018). There were also clients who did not know how to use mobile phones (Tewari 2017; Wolff‐Piggott 2018). In one case, women's access to mobile phones was prohibited as a consequence of gender discrimination (Hirsch‐Moverman 2017). Competitive pricing of mobile phones and phone credit costs increased clients' accesses to phones, and thus eased health worker ‐ client communication (van der Wal 2016). In one study, health workers reported that providing clients with mobile phones, promoted these clients' general social connectedness (Murray 2015).

148Finding 25: Health workers were ambivalent about interventions that required clients to use the health workers' mobile devices during consultations. Their optimism was tempered by concern that there was a loss of meaningful engagement with clients (low confidence in the evidence)

149Health workers had mixed reactions to clients using their (health workers') mobile devices on their own during consultations. In one instance, workers expressed that giving clients the device to access surveys and health promotion material on their own during the consultation allowed clients to deal with sensitive topics in private (Bacchus 2016). Yet, these same health workers were concerned about the loss of meaningful interactions during this process, and raised concerns that they were unable to engage with the clients about the topic at hand, unless the client was willing to share the activity with them: "The challenge is how to keep it personal. If [women] answer positive on the tablet and then you just close the tablet and "oh thank you" and put it away then you've just told her, all I needed was for you to answer the questions. I'm not really here to help you. You have to say okay so this is how you answered and this is how you scored, let's talk more about that. The computer can't do that part, all it can do is take down the information and it's up to the nurse or home visitor to expand upon it and actually get her the assistance that she needs" (Bacchus 2016). In another instance, health workers felt that having material on the device which they could share with clients was useful on days when they (the health workers) were tired, and not up to interacting with the client (Coetzee 2017). However, it has to be noted that in this study, the authors pointed out that they did not agree with the health workers' interpretation that using the device to show health promotion material when they were tired, was a good thing.

150Finding 26: Health workers reported that their access to mobile devices was beneficial to clients and communities who were too poor to own mobile phones (very low confidence in the evidence)

151Health workers reported that their access to mobile devices benefited clients and communities who were too poor to own mobile phones, or to afford paying local merchants for the occasional use of a mobile phone (Chang 2011). Health workers' access to a mobile phone enabled them to access higher‐level care, on behalf of these clients: "I was saying that you can find a whole village without a phone. So, giving these PHWs phones, is helping a lot… By giving these PHWs phones I think it helped them a lot because they just go and contact their PHW, and the PHW calls us" (van der Wal 2016).

152Finding 27: Health workers felt that health promotion and educational messaging directed at clients using mobile health interventions, impacted positively on clients' health behaviours, but cautioned against repetitive showing of health promotion videos. In one instance, issuing clients with mobile phones led to increased use of healthcare services (moderate confidence in the evidence)

153Health workers perceived mobile health interventions as useful for health promotion and education (Coetzee 2017; Lodhia 2016), and that it impacted positively on clients' health behaviours (Bacchus 2016; Barnabee 2014; Chang 2011; Ginsburg 2016; Huq 2014; Ilozumba 2018; Jones 2012; Madon 2014; Praveen 2014): "…people have been motivated to take the drugs because they know that once you get to their home and you press the phone to send a message it means that you are reporting the patient that he has not taken the pills properly….Patients are motivated now to take their pills" (Chang 2011). In one study, health workers thought that clients were motivated to improve their adherence because they were aware that information about their adherence was being relayed to clinic staff by field staff in real time via the mobile devices (Chang 2011). Some workers reported that clients were more responsive to visual material than verbal messages (Coetzee 2017; Lodhia 2016). Some attributed this to a cultural preference for visual information over verbal information (Coetzee 2017). Other health workers cautioned that repetitive showing of health promotion videos would not improve uptake of the health messages (Bacchus 2016; Coetzee 2017): "My client looked at me one time, and she said "how many more times do we have to do this?" (Coetzee 2017). Workers reported that issuing phones to clients for health‐related usage led to increased use of healthcare services (Murray 2015). Health workers perceived that graphic displays on a device helped clients to better understand their condition: "It's wonderful. I got better results than I expected...If patients see the risk bar, they understand very well that they have a high risk of CVD...We gained knowledge from this percentage display too...This is 100% beneficial to the doctor" (Praveen 2014).

154Finding 28: Some health workers accepted bearing the costs of mHealth interventions themselves, but were dissatisfied when phone credit to use the phones was not delivered on time. Health workers felt that clients appreciated it when health workers called them, as it saved them costs (high confidence in the evidence)

155The cost implications for health workers, of mHealth was discussed across the studies, with differing opinions as to the appropriateness and affordability of bearing costs personally (Hampshire 2016; Messinger 2017; Watkins 2018; Wolff‐Piggott 2018). Bearing the costs personally was accepted by health workers either as part of their altruism (Hampshire 2016; Messinger 2017; Watkins 2018): "I know some nurses who will never use their own phones because they have no passion for the job. But, for some of us, it is the passion for the patients and the work that makes us continue" (Hampshire 2016); or from a sense that the investment would generate a greater demand for their services and thus better income (Khan 2015; Messinger 2017). There was less satisfaction with interventions that failed to deliver promised phone credit, on time (van der Wal 2016). Health workers also felt that clients appreciated it when the health workers saved them call costs by phoning them, rather than the other way round: "Even if any client calls me then I cut the line and call back from my phone. If I do so, client will say I am kind to them" (Messinger 2017).

156Finding 29: Health workers' digital literacy impacted on their experience and perceptions of the use of mobile devices in health service delivery: being digitally literate resulted in positive experiences and perceptions, whilst low digital literacy caused concerns about job security and embarrassment when making mistakes in front of clients. For some workers, prior exposure to mobile devices did not affect their perceptions and use of mobile health. Some turned their lack of digital literacy into building a relationship with clients by asking clients to show them how to use the devices. Not using the devices often enough, resulted in loss in digital literacy (moderate confidence in the evidence)

157The level of health workers' digital literacy, that is how comfortable they were in using mobile devices, shaped their perceptions and experiences of using mobile devices in their delivery of health care services (Cherrington 2015; Mitchell 2012; Murray 2011; Nguyen 2015; Praveen 2014; Quinn 2013; van der Wal 2016). Some health workers expressed initial hesitancy, but over time and with training, became more comfortable, to the point of expressing concern that they were becoming dependent on the intervention devices (Hirsch‐Moverman 2017; Mitchell 2012; Valaitis 2005). Health workers expected the training they received to alleviate their anxiety about using the technology (Coetzee 2017). Where health workers were able to use the application well, they regarded using it in service delivery (receiving results) as simple (Hao 2015). Health workers who had no or little digital literacy, initially did not know how to use the devices properly, and felt that they needed training to compensate for their lack of computer knowledge (Coetzee 2017; Hao 2015; Kolltveit 2017; Madon 2014; Murray 2011; Nguyen 2015; Praveen 2014; Quinn 2013; Shao 2015; Surka 2014; Valaitis 2005; van der Wal 2016; Watkins 2018). Such poor understanding of the technology could lead to dual systems, paper and electronic, and poor integration into normal routines (Murray 2011). Health workers, unfamiliar with the technology, were regarded by their trainers as needing emotional reassurance (Murray 2011). This emotional response was also seen in another study where health workers claimed to feel encouraged if sending a message worked, but feeling hopelessness if it did not (Mwendwa 2016). In one study, the authors expressed that those health workers that understood the mHealth application, expressed no concerns, but for other health workers who lacked this understanding, there was concern over errors in reading laboratory results (Hao 2015). A lack of ongoing training was said to lead to insecurity, and this insecurity in turn reduced enthusiasm for the intervention and hindered use of the device (Kolltveit 2017). Health workers across several studies suggested training as means to overcome poor digital literacy and unfamiliarity (Coetzee 2017; Kolltveit 2017; Murray 2011; Nguyen 2015; Praveen 2014). Accounts across several studies suggest unfamiliarity as the source of poor digital literacy, and that with familiarity came greater ease of use (Bacchus 2016; Coetzee 2017; Ilozumba 2018; Mitchell 2012; Quinn 2013; van der Wal 2016). In one study, where nurses already used the Internet or web applications on their phones for personal use, they were also more interested in gaining computer skills for their work, than those with no prior experience of using the Internet (Watkins 2018). It was not always clear where the difficulty stemmed from, faulty technology or unfamiliarity with the technology, but what was clear in one case, was that those who struggled, were the lowest educated (Ilozumba 2018). Poor digital literacy was not limited to health workers, but also to those with whom they had to engage, such as village leaders who expressed that they needed training because they were not able to understand reports and information given on the mobile phone (Madon 2014). One study suggested that irrespective of computer and smartphone literacy levels, that poor uptake was related to difficulty with typing, with health workers suggesting difficulty in use as a result (Shao 2015). Some health workers turned their digital illiteracy into an opportunity to build relationships with clients by asking clients to show them how to use the devices: "One of the older home visitors revealed that she used her own lack of experience with technology as a way of encouraging young women to open up to her with the computer tablet (it's like I'm saying you're really tech savvy with this and it's sort of like a prop you know. Like we're going to talk about this, but you get to use this tablet)" (Bacchus 2016). Others asked their children to assist them, which they viewed as appropriate given that there was no confidential information on the device (Coetzee 2017). It was reported that health workers with low client caseloads used the devices infrequently, and thus may forget how to use them (van der Wal 2016). In some instances, workers felt that poor competency with mobile devices threatened their job security (Madon 2014; Murray 2011): "At the same time, some VHWs were discriminated against for being unable to adapt to new ways of working as a result of the technology. For example, in one of the villages with low prior phone possession, some VHWs with visual impairments who had become used to entering health information onto paper registers were declared as being unfit to work because they could not easily operate the mobiles" (Madon 2014), and felt embarrassed when making mistakes whilst being with clients (Coetzee 2017). In one study while all doctors reported using the Internet at least once a week, the same was only true for a quarter of the nurses (Watkins 2018).

158Finding 30: Health workers expressed a need for training and familiarity with mobile devices to overcome their initial anxiety in using the devices. Peer training from technologically proficient colleagues was experienced as valuable. In several cases, health workers wanted refresher training and pointed to the importance of training replacement staff. Not having mentors who used mobile devices, impacted negatively on lower‐level workers' ability to learn how to use these devices (high confidence in the evidence)

159Some health workers experienced anxiety in understanding and using mobile devices, and felt that training and familiarity with these devices were needed to overcome this anxiety (Coetzee 2017; Ginsburg 2016; Hirsch‐Moverman 2017; Ilozumba 2018; Kolltveit 2017; Lodhia 2016; Mitchell 2012; Murray 2011; Mwendwa 2016; Nguyen 2015; Praveen 2014; Tewari 2017; Toda 2017; van der Wal 2016; Vedanthan 2015). In one study (Ilozumba 2018), younger health workers preferred group training and the older ones, individual training. In another study, the preference was for small group training (van der Wal 2016), and in yet another study, workers referred to the usefulness of peer training from colleagues who were proficient mobile health users: "... there are those who are sharp with the tablet they can show some of us things...So maybe during break you find someone they explain to you" (Vedanthan 2015). Health workers raised concerns about inadequate and too short training (Coetzee 2017; Mwendwa 2016; Toda 2017), which would lead to mistakes in using mobile devices (Coetzee 2017). In one instance, they reported receiving a detailed training manual in addition to in‐person training (Rothstein 2016). Some health workers recommended refresher training (Madon 2014; van der Wal 2016; Vedanthan 2015), and training replacement staff: "Ok if you look at a facility where we have only one nurse, and she was the one that was trained and she is being transferred, who else would... still be there to really continue?" (Toda 2017). Lacking refresher training made some health workers feel insecure and less inclined to use mobile devices (Kolltveit 2017). Some health workers felt hampered in learning to use mobile devices when it was not also used by their clinical mentors (Vedanthan 2015).

160Finding 31: All categories of health workers required technical support to solve user problems. At times, face‐to‐face support was provided, but technical support from proficient colleagues was useful too. Having technical problems solved through real‐time improvements worked well for some health workers, while others suggested a help function be added to the devices (high confidence in the evidence)

161Health workers, lay and professional, required technical support, face‐to‐face or telephonically when having difficulties in navigating mobile devices (Hao 2015; Ilozumba 2018; Kolltveit 2017; Lodhia 2016; Mwendwa 2016; Toda 2017; van der Wal 2016): "He [the supervisor] goes to their home, and asks them to sit on the chair and then he tells them to type to the people. He tells us [CHWs] to type like you do in front of me. Whatever you have a problem I am here for it. You do the typing" (Ilozumba 2018). The absence of technical support, created problems and frustration (Murray 2011; Rothstein 2016, van der Wal 2016). Health workers reported that they had technical problems solved through real‐time improvements and regular meetings (Cherrington 2015). In some instances, higher‐level staff provided technical support, and in other instances, peer‐to‐peer support (Garg 2016; Hao 2015; Madon 2014; van der Wal 2016), helped to solve technical problems. Workers felt that a help function on the devices would be useful to solve technical problems: [Health worker] "The error message is just a text. I can't call anyone for troubleshooting… it would be better to have a troubleshooting function in the application that suggests what to do when you receive an error message" (van der Wal 2016).

162Finding 32: Health workers highlighted that mobile technology applications should be user‐friendly, easy to learn, and improve the quality of their care. When the applications were not easy to use, health workers became frustrated and reluctant users of mobile devices (high confidence in the evidence)

163It was important to health workers that the software and applications on the mobile devices were easy to use (Ginsburg 2016; Khan 2015; Kolltveit 2017; Lodhia 2016; Praveen 2014; Rothstein 2016). In one study they appreciated multiple applications on their devices (Lodhia 2016). When the applications were easy to use, little training was required (Ginsburg 2016; Kolltveit 2017): "The simple design was "not difficult to use... [Y]ou just read and follow the instructions." Almost 80% of the HCPs said it took them a day or less to be familiar with the device and application when they used it for the first time during the training sessions" (Ginsburg 2016). Health workers also pointed to the importance of these devices improving the quality of their care (Ginsburg 2016; Khan 2015). Applications and software that were not user‐friendly, frustrated health workers and made some reluctant to use mobile devices (Kolltveit 2017; Ramirez 2017; Rothstein 2016; Schoen 2017). Some workers complained about slow, in one instance "painfully slow" (Ramirez 2017), and unreliable applications (Mwendwa 2016; Ramirez 2017).

164Finding 33: Health workers held mixed views on choosing between tablets and smartphones. Some felt that the type of content on the device was more important than the device itself. However, other health workers preferred tablets over smartphones, mainly because the bigger size of the screen was perceived as easier for client engagement (very low confidence in the evidence)

165In one study, half of the health workers were given smartphones and the other half were given tablets (Shao 2015). Their opinion about the two different tools were reported as being similar, showing more concern for the algorithm loaded on the devices, than the devices themselves (Shao 2015). In contrast, workers in another study who were issued with mobile phones, thought that tablets would have been better, because it would have made it easier to show clients what they were doing on the devices, and that it looked more professional (Schoen 2017).

166Finding 34: Some health workers felt that sustainable, at scale mHealth programmes required approval and stewardship from political leaders, such as ministries of health. Leadership interest in mHealth interventions was described as motivating to health workers. Health workers suggested that such leaders should be engaged early and continuously throughout the programme, and be provided with evidence of effectiveness, so as to secure their support. The lack of high‐level stewardship impacted negatively on the mHealth programme (low confidence in the evidence)

167Some health workers felt motivated when political interest was shown in mobile health programmes: "Even where healthcare professionals had to organize and manage the telemedicine use on their own, they could still feel some support from leaders when they experienced their attitudes toward telemedicine to be positive. [Health worker]: "There is an interest for this intervention by the leaders, but they are not so interested in knowing more about the intervention. My leader is pleased with the fact that I am handling it all" (Kolltveit 2017). They perceived that sustainable, at scale use of mobile devices required approval and stewardship from higher‐level leaders, including decision makers at national ministries of health (Ginsburg 2016; Kolltveit 2017; Lodhia 2016; van der Wal 2016). They felt that early and continued engagement with these leaders facilitated their support (Ginsburg 2016). Workers also reported that not including higher‐level professionals in programmes, affected intervention uptake (Kolltveit 2017). Receiving mobile devices and the means to use it, was perceived by some health workers as an acknowledgement of their work (van der Wal 2016). They also mentioned that evidence of effectiveness and cost‐effectiveness were important to ensure higher‐level support for mobile health programmes (Lodhia 2016). When there was lack of higher‐level stewardship, health workers felt it impacted negatively on the mobile health programme (Kolltveit 2017).

168Finding 35: Health worker accounts pointed to the strong influence of the health systems and social context in which the intervention was embedded. Contextual and systems issues, such as difference in language use between clients and health workers, gender discrimination, discomfort with professional hierarchies, poverty, resource constraints, staff attrition, and more, all of which were external to the technology and the physical device, influenced how health workers experienced mHealth and the use of mobile devices for service delivery, in their different contexts (moderate confidence in the evidence)

169Health workers' accounts showed that the systems in which mobile health programmes were implemented, and contextual issues external to the devices itself, shaped their experiences and perceptions regarding their use of the devices (Wolff‐Piggott 2018). These included language differences between workers and clients, which made communication difficult irrespective of using mobile devices or not (Khan 2015); cultural practices, such as gender discrimination against female use of mobile phones (Huq 2014); and educational and professional differences which caused strained relationships between lower‐ and higher‐level workers: "Two village doctors reported that the lack of comfort with the call centre doctors resulted in their reluctance to use the intervention. As one village doctor said, "When I talk with my patient I feel like we are brothers during our conversation and the patient feels comfortable (about sharing problems). But when I consult with call centre sometimes I didn't feel that warmth probably because they are from another place and we never met [sic]"" (Khan 2015. Furthermore, health workers felt that client poverty made the uptake of mobile health services challenging for clients if it required that they access a mobile device themselves, (Huq 2014; Tewari 2017). Staff attrition and shortages were often reported as main barriers to optimal implementation of the intervention, and uptake of the use of the devices as an additional activity, on top of existing staffing problems (Chang 2011; Kolltveit 2017; Lodhia 2016; Rothstein 2016; Shao 2015; Toda 2017): "In non‐urban settings, like this place, staff attrition is very high.... For the past about three years, we've been receiving an average of about 20 community health nurses every year.... These are new people. So they don't know anything about MOTECH (mHealth programme]. They have to be trained" (Rothstein 2016). The same applied to workers who reported that unsupportive higher‐level professionals made them less enthusiastic about mobile health (Praveen 2014; van der Wal 2016). Workers also found it challenging to learn how to use the new devices amidst the rush and unpredictable workday routines in primary healthcare facilities (Rothstein 2016). They also pointed out that other health systems' problems, such as when drugs were out of stock (Shao 2015), overstretched laboratory services (Shao 2015), and health workers not knowing pre‐mHealth reporting guidelines (Toda 2017), made it difficult to use mobile devices. In one disease surveillance programme which used mobile devices to record and report the surveillance data, staff felt that face‐to‐face supervision was an important enabler for high‐quality data, but resource constraints and poor road conditions made such supervision impossible, and that this lack of face‐to‐face supervision, not the technology, impacted negatively on the quality of the fieldwork staff (Toda 2017).

170Finding 36: It was important for health workers that mobile health interventions be integrated with other existing electronic health information systems. This interoperability made it more likely that mobile devices would be integrated into standard care practices, while the absence of integration frustrated health workers (moderate confidence in the evidence)

171Health workers felt it important to integrate mobile health interventions with other existing electronic health information systems (Garg 2016; Ginsburg 2016; Lodhia 2016). They appreciated it when integration worked well (Garg 2016; Rothstein 2016): "The cancellation that the patient is not [coming to their visit] – it's connected to the EHR or EPM [mHealth application] side of [this], so that automatically releases an appointment. Nobody has to do anything…which is gold. So that's automated" (Garg 2016), but struggled in the absence thereof (Lodhia 2016; Rothstein 2016). In one instance, where integration only happened at the district level, health workers suggested national government support for national level integration (Lodhia 2016). Health workers perceived having access to client information across different care points within the health system as a benefit of integration (Garg 2016; Rothstein 2016), and this made them enthusiastic to integrate their use of mobile devices into their routine workflow (Rothstein 2016).

172Finding 37: Health workers offered programmatic and implementation recommendations to improve mobile health interventions. The most cited of these was that the interventions be expanded to other settings and services, beyond what they were using it for as described in the studies. Other recommendations included raising community awareness about mHealth programmes, being involved in developing programmes, and appointing a 'mobile health champion'. Workers also suggested that those collecting surveillance data, must be informed of how the data are used (moderate confidence in the evidence)

173Health workers felt that mobile health interventions could be improved, and offered a range of recommendations related to programmatic and implementation issues. They suggested that mobile health programmes be implemented at scale, and expanded to other settings, services, and illnesses beyond what they were using it for, as described in the studies (Bacchus 2016; Medhanyie 2015; Mitchell 2012; Rothstein 2016; Toda 2017; van der Wal 2016). Across several studies, health workers suggested improving the social marketing of these programmes, including advertising and sharing information with users, clients, and communities, as this may enhance the acceptability and uptake of these programmes (Ginsburg 2016; Khan 2015; Lodhia 2016; Mwendwa 2016). They also recommended that those using the devices be consulted as end users during the planning and implementation of the intervention: "I [supervisor] think there needs to be more involvement of other people (users), to know, what do they think, how can this be done better, what are their inputs rather than pushing it down and ask them just to use the system" (Hao 2015); be included in decisions about system changes to be introduced because of the mobile health programme (Hao 2015); be given money for phone credit (Barnabee 2014); and that all paper‐based stationery, not just certain documents, made available on the mobile devices (Schoen 2017). Some recommended an automated response service for when health workers are in emergency situations (Mwendwa 2016). Workers advocated for the appointment of a 'champion' who could provide technical and intervention assistance (Kolltveit 2017; Murray 2015): "The importance of having a colleague who could champion this intervention was described as a prominent success condition: "Well, we have some among us who facilitate it all. They have encouraged us, and when some of us think this is too much work, they have been there with their enthusiasm. This enthusiasm has been valuable to us"" (Kolltveit 2017). There were recommendations that those collecting routine data using mobile devices, should be informed as to how the data are put to use, and that not getting this feedback made them feel like students who sat an exam without getting their results (Madon 2014).

174Finding 38: Health workers had several technical recommendations to improve mobile health devices, for instance solar panels to counter poor electricity access and using photos to track clients' recovery from illness. Other recommendations included using sturdier devices, bigger screens, and having common applications, such as work scheduling on the devices (moderate confidence in the evidence)

175Health workers suggested ways to improve the technical aspects of mobile health interventions. Password protection was proposed to keep client information confidential, installing tracking software to mitigate loss or theft of a device, and having theft alert protocols put in place (Coetzee 2017). Issuing solar panels (Henry 2016), backing‐up data (Ramirez 2017), and using battery‐powered systems (Lodhia 2016), were offered as possible solutions to the problem of access to electricity and power shortages. Some workers thought that taking photos of clients' improvement or participation in health promotion activities, could serve as encouragement to clients: "We [health workers] could also use it to motivate community members by showing them how they are improving. For example, if a community member [client] was taking good care of a wound but didn't think it was healing, we could show them a photo from last week and say "look here! It is better" (Schoen 2017). Other workers suggested sturdier devices: "… and it [mobile phone] [needs to be] a much tougher thing, it's going to be slapped around from my dressing table in the car, the houses, anywhere, the iPhones tends to be a lot more delicate …" (Quinn 2013), and features for easier user interface on the devices (Praveen 2014; Rothstein 2016). Applications commonly found in some studies, such as work scheduling, receiving reminders, and recording work done, were also recommended by those who did not have access to these features (Schoen 2017). There were also health workers who recommended using a stylus, i.e. a small pen‐shaped instrument, when typing on a tablet (Schoen 2017). In one instance workers thought that tablets, with their bigger screens, would make it easier for clients to see what the workers were doing on their devices, compared to small screen mobile phones (Schoen 2017).

176Finding 39: The main challenges health workers experienced in using mobile devices, were poor network connectivity, access to electricity, and the costs to recharge devices. Solutions offered, included using solar panels, using the powered‐up phone of a colleague, or reverting back to the paper‐based system. Sometimes poor connectivity resulted in client dissatisfaction because it created delays in receiving health care. Health workers' commitment to their clients motivated them to cope with these and other challenges (high confidence in the evidence)

177The most cited challenges for health workers using mobile devices were poor network connectivity (Hampshire 2016; Ilozumba 2018; Khan 2015; Lodhia 2016; Madon 2014; Nguyen 2015; Praveen 2014; Quinn 2013; Rothstein 2016; Schoen 2017; Toda 2017; van der Wal 2016), no easy access to electricity to charge their devices (Chang 2011; Ginsburg 2016; Hampshire 2016; Lodhia 2016; Madon 2014; Mwendwa 2016; Nguyen 2015; Schoen 2017; Toda 2017), and the costs to have their mobile devices charged (Hampshire 2016; Mwendwa 2016; Toda 2017). The challenge of charging was not only inconvenient, it could also impact on fully implementing the intervention. In one study it served as a deterrent to downloading the intervention software application (Ginsburg 2016). Poor connectivity impacted on clients, who became impatient in waiting to connect to a doctor or waiting to get a text message prescription (Khan 2015). To circumvent these challenges, workers used private solar panels (van der Wal 2016), though some complained that it took a long time to charge their devices (van der Wal 2016). In other instances, health workers solved poor connectivity by uploading their work at times when connectivity was good, but that may have been inconvenient to them (van der Wal 2016), or by looking for places with good connectivity: "We also have problems with network connectivity. For example, so the uploading may be a challenge. And sometimes they're typed, but it doesn't go through. Sometimes it doesn't go through at all. You have to go and climb a tree" (Rothstein 2016). Some had to walk long distances in search of good reception and/or electricity (Ginsburg 2016; Madon 2014; Nguyen 2015; Rothstein 2016). Some workers used their own resources to ensure connectivity (Quinn 2013; Watkins 2018), whilst others reverted back to the paper‐based systems when the mobile devices did not work (Nguyen 2015; Schoen 2017). Given problematic access to electricity, workers at times had to use the powered‐up phone of a colleague or otherwise put their SIM card into a colleague's powered‐up phone (Mwendwa 2016). In one case they used informal business networks to have their phones charged at a low cost (Chang 2011). Poor connectivity did not just challenge health workers, it challenged the success of the intervention being conducted as intended. It resulted in delays in providing health care (Quinn 2013), which upset some clients (Khan 2015), it caused complaints from workers that work was lost (Schoen 2017), devices 'froze' whilst they were working (Schoen 2017), and resulted in slow transmission and receiving of information (Schoen 2017; van der Wal 2016). However, health workers' commitment to their clients motivated them to cope with these and other challenges: " … [despite] feeling discouraged because of lack of Internet connectivity, they [health workers] kept stressing that the mHealth tools were in their community's best interest, good for their health system, and therefore felt a professional duty to accept and use the application. [Health worker]: "Our priority is mother and child care, so it is important that we fully succeed, isn't it?"" (van der Wal 2016).

178Finding 40: Health workers expressed dissatisfaction with mobile devices when technology changes were too rapid, showed a dislike for typing, and were concerned that mHealth impersonalised their interaction with clients. Since these dissatisfactions were only infrequently raised within the data set, it is unclear if these perceptions reflect wider experience (low confidence in the evidence)

179Some studies offered challenges raised by health workers that were not commonly discussed across the included studies. Health workers expressed dissatisfaction with aspects of mobile devices, which included when technology changes were too rapidly introduced (Valaitis 2005), or when their expectations of the devices were not met, for example when they anticipated it would make manual data capturing unnecessary, yet they still had to do it (Hao 2015), or not being able to record two visits simultaneously, particularly because it took too long to load the application for each new visit (Schoen 2017). There were also health workers who did not like typing on a mobile device (Schoen 2017). Other health workers felt that the devices at times impersonalised their interactions with clients when they required emotional support: [Home visitor]: "It's [tablet] cold…it’s just her interacting with a machine. So there's no sympathy, there's no condolences. There's no, I want to say loving interaction. No, um, it’s like no comfort, no support you know" (Schoen 2017). The infrequency with which these were raised, makes it unclear if these are widely‐held perceptions, reflecting a more general or broader experience.

180Finding 41: Health workers discussed challenges, beyond network and electricity issues, that sometimes were just an annoyance or a concern, but at other times also impeded their mHealth activities, and their ability to provide a service assisted by the use of mobile devices. These included damaged devices, loss and theft of devices, having to carry two devices, not being able to readily buy phone credit when needed, not being able to send long messages because of character limitations, and the limitations of the language capabilities of their devices (moderate confidence in the evidence)

181Apart from challenges with network connectivity and electricity issues, health workers listed a number of other challenges they faced when using mobile devices. These challenges included lost and damaged devices (Murray 2015; Schoen 2017), working in high‐crime areas and subsequently facing personal safety risks and risks of stolen devices (Chang 2011; Coetzee 2017; Schoen 2017; Toda 2017; Valaitis 2005). CHWs agreed that they were concerned about the safety and use of tablets in some of the communities and households they visited: "I was also afraid because of the places that I go to. The places that I go to criminals will be looking at me while they did not mind me before" (Coetzee 2017); workers' low proficiency with English and the unavailability of local language characters (Ilozumba 2018; Medhanyie 2015; Mwendwa 2016); character limitations for text messages (Hao 2015); small screens and keypads (Rothstein 2016; Schoen 2017; Valaitis 2005); old devices (Mwendwa 2016), problems with short‐life batteries (Praveen 2014; Quinn 2013; Schoen 2017; van der Wal 2016): "And the battery time was a bugbear with me. I hadn't got the time to be plugging it in everyday, I hadn't always got an office to plug it in" (Quinn 2013). It was also a concern when there were not enough devices (Lodhia 2016), and when they had to carry both a personal and work phone: "Many of the health workers complained about carrying two phones, the smartphone we gave them and their private phone. As a result, many of them stopped carrying their private phone and started using the smartphone as their primary phone" (Medhanyie 2015). Some workers complained when they lost work because it was not backed‐up (Cherrington 2015). In one instance, not having a vendor to purchase phone credit from, was perceived as a barrier to use mobile devices (Hampshire 2016), and in another, not having someone to repair their devices (Murray 2015; Schoen 2017). When health workers in one intervention uploaded photos, videos and music (sometimes private), it slowed the technology and disabled the mHealth application that they had to use (van der Wal 2016). These challenges were not simply an annoyance or irritation, they also impeded health workers in their service delivery. For example, one health worker explained that she/he could not call an ambulance for a violent psychiatric patient because she/he did not have credit on their phone and all shops were closed, so no credit could be purchased (Hampshire 2016). Character limitations in text messages prevented some health workers from sending required lab results (Hao 2015). Device shortage in combination with network and Internet coverage made for difficulties in sending images, client information, and for seeking advice (Lodhia 2016).

182Finding 42: Health workers complained when the tasks asked of them in mHealth interventions were felt to be beyond their clinical capacity, and when support from higher‐level workers was absent (very low confidence in the evidence)

183Health workers found it difficult to communicate or explain information to the client provided to them via their mobile devices when this information was beyond their clinical capacity. Examples of this included receptionists who had to screen clients for cardiovascular illnesses and relay the results to the clients: "Receptionists were unsure how to respond to patients' questions and generally felt this duty was not part of their role. They did not see the relevance of screening for stroke prevention. Patients would say, "Is this my heart rate?" and I would say, "I don't really know"" (Orchard 2014). Another example was lay health workers who preferred to refer screened clients to a doctor to receive their screening results (Praveen 2014). This difficulty was also expressed when there was an absence of higher‐level support in following up on the screening results (Praveen 2014).

184As described in the Methods section, our review was used alongside several Cochrane intervention Reviews on the effectiveness of mHealth interventions, commissioned by the WHO to inform their guidelines on digital interventions for health system strengthening (WHO 2019). These reviews assessed the effectiveness of diverse types of mobile devices used by health workers to improve their delivery of care. These included mobile devices for birth and death notification (Vasudevan 2018), stock notification (Agarwal 2018a), client to provider and provider to provider telemedicine (Gonçalves‐Bradley 2018a; Gonçalves‐Bradley 2018b, tracking of clients' health status (Agarwal 2018b), and decision support (Agarwal 2018c). The findings from our qualitative evidence synthesis were used as a source of information about intervention acceptability and feasibility, which the WHO's guideline panel used as a basis for their recommendations. The GRADE evidence‐to‐decision tables where this evidence is presented alongside evidence from the relevant Cochrane Reviews of effectiveness are available in the Guidelines appendices (www.who.int/reproductivehealth/publications/digital‐interventions‐health‐system‐strengthening).

185In addition, we also planned to develop hypotheses about why some mHealth technologies are more effective than others. After an assessment of the results from the reviews of effectiveness, as well as of our own review findings, we chose, however, not to develop specific hypotheses about individual mHealth technologies. One reason for this was that the preliminary findings of the reviews of effectiveness (apps.who.int/iris/bitstream/handle/10665/311980/WHO‐RHR‐19.10), showed large evidence gaps, while the evidence that was found was often of low or very low certainty. While there was some evidence of impacts on health outcomes, the available evidence suggested that these types of interventions may make little or no difference to the outcomes that were measured. While we do not currently have a clear picture of whether some mHealth technologies are more effective than others, the findings from our own review suggest that it is unrealistic to expect consistent positive outcomes in mobile health programmes. This includes outcomes tied to the health of clients, to service delivery and to the organisation of care. The qualitative evidence in our own review illustrates how these programmes are comprised of many interlinking and at times complex, components, for instance, health system arrangements whereby the programme is being implemented. Each of these components offer different issues that may contribute to positive outcomes, but also pose definite challenges that render mHealth susceptible to poor outcomes. These facilitators and barriers relate to, amongst others, (i) the devices and technology itself, for example, short‐life batteries and user‐friendly software (Findings 32, 41); (ii) health workers' aptitude for mobile devices and their digital literacy (Findings 15, 24, 29); and (iii) upstream issues such as health system arrangements and high‐level stewardship (Findings 34, 35). Therefore, we consider the following two key hypotheses based on our findings, may help explain and understand the effectiveness of mHealth programmes in delivering and supporting primary healthcare services.

186There are self‐evident benefits to mHealth, for example linking people living in rural areas to forms of health care that they would never otherwise have had access to (Findings 1, 2, 22), it eliminates inhibiting transport challenges (Finding 11), access to real‐time data (Findings 5, 10, 14), and its portability allows health workers to access healthcare information at the point‐of‐care (Findings 9, 12). However, seeking and delivering health care, regardless of the mode, remains a relational transaction between a client and a health worker. Our findings show that all of the usual relationship issues that make health worker‐client relationships successful or challenging, issues such as being trustworthy, being seen to offer high‐quality care, being seen to be knowledgeable about the condition being treated, remain. However, with the addition of mHealth, this becomes complicated when for example, a health worker's trustworthiness may be determined by the sense of trustworthiness of the information they relay from the mobile device, or a health worker's status is elevated because they are seen as being worthy enough of being entrusted by their employer with an expensive device. The existing relationship complexities, therefore interact with the new relationship complexities brought by mHealth. The use of a mobile device as part of the transactional relationship between health workers and clients is therefore, but one element of a complex relationship, and understanding the eventual health outcome achieved, requires acknowledgement of this complexity, and acceptance that there may be factors which cannot be controlled for or easily explained. However, a health worker who is already regarded as trustworthy, and is able to show extra competence through use of a mobile device, is likely to be even more highly regarded, and the message they deliver through the use of the device may be regarded as more credible. The opposite however, is equally likely, a previously well‐regarded health worker, using a faulty device, may lose face with clients, and so may begin to lose credibility, and struggle to convince clients to adopt the behaviour that they are suggesting. The outcome therefore is sensitive to the interaction between these multilayered, complex factors and interactions.

187No matter the sophistication of the mobile health technology or programme, if it is implemented in a health system that has challenges, then the mHealth programme is also likely to be challenged. In those instances, it is the dysfunctional system, rather than the device, technology, or programme, which becomes the barrier to positive effectiveness outcomes. Mobile health programmes are embedded in larger systems, and therefore impacted by contextual issues external to the technology itself, and how health workers use mobile devices. This was evident in health workers' reporting how health system and contextual factors impacted on their optimal use of the devices (Findings 3, 16, 34, 35, 39). Examples of these factors include network connectivity, access to electricity, staff shortages, unresponsive emergency services, inadequate supervision, and strained relationships between lower‐ and higher‐level health workers. It is our contention that the reverse also holds true: mHealth can have a positive effect when used within an already functional system and can sometimes close a specific gap in a system that is not fully functional.

188In summary, the complexities of healthcare delivery and of human interactions defy drafting simplistic hypotheses that can predict the effectiveness of mHealth programmes to provide and support primary healthcare services. The effectiveness of these programmes results from the interplay between technology, context, and the human attributes of clients and health workers. Detailed programmatic and process description information and realist evaluations (Pawson 2001), alongside effectiveness studies will be a starting point to unravel this interplay and formulate hypotheses regarding the effectiveness of mobile health. Below, we offer a few recommendations regarding implementation practices that may improve the likelihood of positive outcomes when using mobile devices to provide and support the delivery of primary healthcare services.

189We described our initial positioning earlier (see 'Review author reflexivity' in the 'Methods' section above). Our views remained the same during the review, though our continued team discussions led to more nuanced definitions regarding mobile health, whether it is used to deliver or support healthcare services, and primary health care. Whilst writing the 'Discussion' and the 'Conclusions', we were particularly aware of the risk of overlooking data that refuted our own experiences that mobile health intervention outcomes are usually a mix of having positive and no effects.

190For a summary of the main findings, please see the 'Plain Language Summary'.

191The majority of studies (74%) on which our findings are based, were conducted in low‐ or middle‐income countries. Most of these were from Africa (71%) while only one study was from Latin America (Brazil). Of the 12 studies conducted in high‐income countries, five were conducted in the USA, two in Canada, and one each in Australia, Ireland, Norway and Scotland. We downgraded our confidence in several of our findings because of the limited range of settings that the data were from.

192Regarding the health issues addressed in the mobile health programmes, our data are a fair representation of the main services offered in primary health care in the study countries. Seventeen studies focused on maternal, neonatal and child health, four addressed HIV/AIDS, and two, malaria and tropical diseases. Other health issues included, amongst others, cardiovascular diseases, intimate partner violence, eye care, hypertension, wound care, and mental health. Similarly, there was an even spread of studies reporting on lay and professional health workers' experiences and perceptions: 10 studies reported on lay health workers, 12 on health professionals, and in 17, both categories were reported on. The health worker category was unclear in the remaining four studies.

193We would also argue that the included studies mirror by and large, current device and application use, with tablets and personal digital assistants (PDAs) less used compared to mobile phones, most likely due to costs and convenience. Tablets and PDAs were used in only 11 studies (26%), and mobile phones, identified as iPhones or smartphones, in 31 studies (72%). In one study, both tablets and mobile phones were used. Device applications ranged from text messaging, screening and diagnostic algorithms, and pre‐loaded health promotion materials, to the recording of surveillance data and work scheduling.

194Studies exploring healthcare workers’ use of digital health strategies are now published increasingly often. We have listed 85 studies under “Studies awaiting classification” that appear relevant for our review and that have been published since we finalised our analysis. In our next update of this review, we will sample from these studies, focusing in particular on those findings that we have downgraded using the GRADE‐CERQual approach.

195A mixed‐methods review by Konttila and colleagues (Konttila 2019), on key competencies required from healthcare professionals to use digital technology, and the organisational factors that shape their use of it, resonates with several of our findings. They reported that using digital devices impacted both positively and negatively on the health worker‐client relationship (Findings 18, 22); led to concerns amongst health workers that they might lose their clinical competencies (Finding 13); were influenced by the users' level of digital literacy (Finding 29); and required refresher training and technical support (Findings 30, 31). Importantly, these authors echoed our views that digital technology is embedded within larger systems that can either advance or undermine the effectiveness of mobile health programmes (Konttila 2019).

196We concur with Dunn and colleagues, that primary research is needed to understand the pathways of how mobile health effects change (Dunn 2018). It is clear from our findings that the effectiveness of mobile devices cannot be separated from those who use it to deliver health care. Mobile health interventions are a combination of device and health worker, each with unique attributes, but also jointly, that impact clients' health behaviours, and how primary health care is supported and delivered. Though our review reported on health workers perceptions of how clients perceived them and the devices they use, asking clients themselves may further our understanding of the pathways of effect. We are in agreement with Lee et al (Lee 2016), that clearer descriptions of the mobile health interventions may help to better understand its impact. Contrary to Marcolino's review of effectiveness reviews (Marcolino 2018), where it was found that the majority of studies were conducted in high‐income countries, 74% of the studies in our review were conducted in low‐ or middle‐income countries. No obvious reason for the difference was found, but it does suggest a need for a more even spread of studies conducted in low‐, middle‐ and high‐income countries.

197It appears from this review and others, that mobile health programmes are increasingly maturing from their 'pilotitis' status (Huang 2017), in so far as pilot studies that usually have small client participant numbers. Across the included studies, four had programmes serving between 101 to 1000 clients, 22 were implemented across multiple facilities, communities, and subdistricts, and seven of these were implemented in 20 or more sites. One study reported on a national mobile health programme. However, programmes remain relatively small in terms of health worker participants: only three studies had more than 100 worker participants, and 26 studies had between 11 and 100 workers. Mobile health programmes suffer from 'pilotitis' with respect to sustaining it over time. Only two have been running for six years or longer, three between two and three years, and two studies were implemented between one and two years. The implementation period of the remaining studies was either unclear or shorter than one year. From our full‐text screening, it is also clear that the number of qualitative studies reporting on mobile health programmes is increasing.

198Our findings resonate with results from primary studies that did not meet our inclusion criteria, but which reported benefits and challenges similar to what we found, when using mobile devices to deliver or support primary healthcare services. This includes benefits, such as reducing travel time, and its related expenses, for health workers and clients alike (Hurt 2016), real‐time communication (Wang 2018), and sourcing information from the Internet (Anderson‐Lewis 2018). Challenges listed in these primary studies include similar issues found in our review, for instance poor network connectivity, costs, and protecting client information (Barron 2018; Boonstra 2018; McCloud 2016; Siedner 2012).

199Our synthesis identified issues that have been less reported in peer‐reviewed literature. These include: health workers advocating for political stewardship from ministries of health; workers reporting that it is more likely to turn intervention care into routine care when the mobile platform is integrated with other routine electronic systems; an acknowledgement that there is still a need for face‐to‐face interaction with colleagues and clients, and workers who often use their personal devices without remuneration for its associated costs. Though not surprising, there were health workers who perceived some clients to be more responsive and taking ownership for their health, because their consultations included the use of mobile devices. The reasons offered for this ranged from clients being concerned about the immediacy with which workers could report when they defaulted on treatment, to clients perceiving the devices as more trustworthy than standard care.

200As we no longer had access to the Global Health Ovid database for the 2018 search, this could be considered a limitation of the review. The 2015 search yielded only 870 records from that database, and it could be expected to have yielded less in 2018, given the shorter time period. However, it remains that some eligible studies may have been missed.

201Through our sampling, we excluded 10 studies because of serious methodological limitations, and it is possible that some of these may have contained data that could have added nuances to our findings and/or resulted in new findings. However, this concern is to an extent balanced out by the fact these additional studies would likely not have increased the confidence in existing findings because of their methodological limitations.

202Below are a set of questions that are drawn from the high‐ and moderate‐confidence findings in this review, and that may help implementation agencies, ministries of health, programme managers, and other stakeholders to plan, implement, or manage mobile health programmes.

203Will health workers be part of the planning, implementation, and evaluation processes of mobile health programmes? Will their views be sought, and their perspectives taken at each stage of the programme?To what extent is political buy‐in from health ministries required, and achieved, for the successful implementation of the mobile health programme?Has a proper assessment been made on whether health workers' use of mobile devices is adding to or alleviating their workload? How will the extra workload that may occur, be accommodated for?If your intervention is intended to improve efficiency and co‐ordination, is the health system in which it is set prepared for the extra demands that this may imply? For example, if a health worker calls for an ambulance or for professional backup, will such support be available; if an mHealth screening intervention results in increased clients at facilities, will the existing capacity of facilities be able to handle the increased workload? If no preparation is in place for extra demands on the health system, have you engaged with those who may be required to provide additional services, so they can make preparations?Do higher‐level health workers have the time and means to respond when lower‐level workers send them requests via mobile devices, and have lower‐level workers' use of mobile devices been properly explained to all higher‐level workers with whom they interact in delivering health care?Does your intervention require health workers at the same or different levels of hierarchy, to interact with each other? If so, are these health workers prepared for, and willing for the changes that may arise as a result of this interaction, such as new forms of supervision and accountability, immediacy of contact, and telephonic request for advice? What needs to be done to better prepare these relationships for the anticipated changes in expectations on all parties as a result of mHealth?

204Will health workers be part of the planning, implementation, and evaluation processes of mobile health programmes? Will their views be sought, and their perspectives taken at each stage of the programme?

205To what extent is political buy‐in from health ministries required, and achieved, for the successful implementation of the mobile health programme?

206Has a proper assessment been made on whether health workers' use of mobile devices is adding to or alleviating their workload? How will the extra workload that may occur, be accommodated for?

207If your intervention is intended to improve efficiency and co‐ordination, is the health system in which it is set prepared for the extra demands that this may imply? For example, if a health worker calls for an ambulance or for professional backup, will such support be available; if an mHealth screening intervention results in increased clients at facilities, will the existing capacity of facilities be able to handle the increased workload? If no preparation is in place for extra demands on the health system, have you engaged with those who may be required to provide additional services, so they can make preparations?

208Do higher‐level health workers have the time and means to respond when lower‐level workers send them requests via mobile devices, and have lower‐level workers' use of mobile devices been properly explained to all higher‐level workers with whom they interact in delivering health care?

209Does your intervention require health workers at the same or different levels of hierarchy, to interact with each other? If so, are these health workers prepared for, and willing for the changes that may arise as a result of this interaction, such as new forms of supervision and accountability, immediacy of contact, and telephonic request for advice? What needs to be done to better prepare these relationships for the anticipated changes in expectations on all parties as a result of mHealth?

210Does your setting have the necessary infrastructural and technological capacity to support the level of sophistication intended by the intervention, for example: is there sufficient electricity supply and electricity coverage, network capacity, technical support, and vendors to purchase phone credit or data for the level of intervention that you intend to implement? Have you considered how these might vary by region?Are the devices being used in the intervention sufficiently sophisticated for the level of intervention being planned, and are these devices replaceable or repairable within your setting? Have you considered who will repair them, and who will cover the costs?When planning mHealth programmes, has the number of staff and clients who have access to mobile devices been taken into account? Are there strategies in place when clients change their mobile phone numbers?Has adequate provision been made for health workers to have sufficient phone credit and data, without having to use their own resources?Is there a strategy to integrate the mobile health platform within existing electronic health information systems? Have you considered the requirements to ensure interoperability?

211Does your setting have the necessary infrastructural and technological capacity to support the level of sophistication intended by the intervention, for example: is there sufficient electricity supply and electricity coverage, network capacity, technical support, and vendors to purchase phone credit or data for the level of intervention that you intend to implement? Have you considered how these might vary by region?

212Are the devices being used in the intervention sufficiently sophisticated for the level of intervention being planned, and are these devices replaceable or repairable within your setting? Have you considered who will repair them, and who will cover the costs?

213When planning mHealth programmes, has the number of staff and clients who have access to mobile devices been taken into account? Are there strategies in place when clients change their mobile phone numbers?

214Has adequate provision been made for health workers to have sufficient phone credit and data, without having to use their own resources?

215Is there a strategy to integrate the mobile health platform within existing electronic health information systems? Have you considered the requirements to ensure interoperability?

216Has the programme management budgeted for adequate training of initial staff, refresher training and in‐service training for new staff members?What is the level of digital literacy amongst those health workers who will implement the intervention, as well as managers and supervisors who will support them? What further interventions are needed to ensure adequate skill levels are present at the beginning of the intervention and maintained over the course of the intervention?Has the programme management identified 'champions' amongst the workers whom they can call upon to assist those struggling with the devices?When the device allows the health worker to screen and diagnose clients, are they clinically equipped to respond appropriately to the results of the screening and diagnosing? Are they able to explain the results to the patient?Is there a system in place to allow staff who dislike, or who are not sufficiently digitally literate to use mobile devices, to continue with standard practice, such as a paper‐based system for recording work?

217Has the programme management budgeted for adequate training of initial staff, refresher training and in‐service training for new staff members?

218What is the level of digital literacy amongst those health workers who will implement the intervention, as well as managers and supervisors who will support them? What further interventions are needed to ensure adequate skill levels are present at the beginning of the intervention and maintained over the course of the intervention?

219Has the programme management identified 'champions' amongst the workers whom they can call upon to assist those struggling with the devices?

220When the device allows the health worker to screen and diagnose clients, are they clinically equipped to respond appropriately to the results of the screening and diagnosing? Are they able to explain the results to the patient?

221Is there a system in place to allow staff who dislike, or who are not sufficiently digitally literate to use mobile devices, to continue with standard practice, such as a paper‐based system for recording work?

222Has enough been done to raise community‐ and client‐level awareness of the mobile health programme, and its implications for the services delivered by it?What is the level of cultural acceptability of mHealth in the proposed setting? What is the existing level of trust between healthcare workers and the community? Have you considered that low levels of trust may be exacerbated by mHealth, for example fears about personal data?What other interventions are needed to increase trust, enhance acceptability of mHealth, and reduce skepticism amongst recipient communities?Are there specific social or geographical barriers which may interact with the intervention, such as women not being allowed access to phones? How might these be addressed in advance?Have you considered how barriers to mHealth use may further increase inequity, and what other interventions are required to reduce these inequities?

223Has enough been done to raise community‐ and client‐level awareness of the mobile health programme, and its implications for the services delivered by it?

224What is the level of cultural acceptability of mHealth in the proposed setting? What is the existing level of trust between healthcare workers and the community? Have you considered that low levels of trust may be exacerbated by mHealth, for example fears about personal data?

225What other interventions are needed to increase trust, enhance acceptability of mHealth, and reduce skepticism amongst recipient communities?

226Are there specific social or geographical barriers which may interact with the intervention, such as women not being allowed access to phones? How might these be addressed in advance?

227Have you considered how barriers to mHealth use may further increase inequity, and what other interventions are required to reduce these inequities?

228More studies are needed from high‐income countries and low‐ and middle‐income countries outside of Africa.In general, researchers should aim for better reporting of their studies. This includes providing detailed information on:the contexts in which the mobile devices are used, as this is likely to shape the acceptability, feasibility, and effectiveness of using mobile devices;their methods of sampling, data collection and analysis; andreflections on how the researchers' views and positions may have influenced the results.Suggestions regarding how to report mHealth interventions can be found in the 'mHealth evidence reporting and assessment (mERA) checklist' (Agarwal 2016).Researchers should give prominence to participants' voices in their studies, and present rich data, where important for a proper understanding of the phenomenon.More qualitative research should be conducted alongside effectiveness studies to explore the results of effectiveness studies. We suggest that detailed programmatic and realist evaluations (Pawson 2001), become part of effectiveness studies.Though individual mobile health programmes may be implemented at scale regarding client participants, and to a lesser extent, health worker participants, more longitudinal research of these programmes are needed to assess the sustainable integration of mHealth into standard care.

229More studies are needed from high‐income countries and low‐ and middle‐income countries outside of Africa.

230In general, researchers should aim for better reporting of their studies. This includes providing detailed information on:

231the contexts in which the mobile devices are used, as this is likely to shape the acceptability, feasibility, and effectiveness of using mobile devices;

232their methods of sampling, data collection and analysis; and

233reflections on how the researchers' views and positions may have influenced the results.

234Suggestions regarding how to report mHealth interventions can be found in the 'mHealth evidence reporting and assessment (mERA) checklist' (Agarwal 2016).

235Researchers should give prominence to participants' voices in their studies, and present rich data, where important for a proper understanding of the phenomenon.

236More qualitative research should be conducted alongside effectiveness studies to explore the results of effectiveness studies. We suggest that detailed programmatic and realist evaluations (Pawson 2001), become part of effectiveness studies.

237Though individual mobile health programmes may be implemented at scale regarding client participants, and to a lesser extent, health worker participants, more longitudinal research of these programmes are needed to assess the sustainable integration of mHealth into standard care.