A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare

1Medicine has embraced technology over the last five decades and many of the most exciting developments are a consequence of this willingness to incorporate new and novel advances into clinical practice. Of the present areas of development and research, artificial intelligence (AI) has garnered the most attention, captured people’s imaginations, and is unique in that it is the only technology that has the potential to revolutionise every area and speciality.

2The Dartmouth workshop on “Summer Research Project of Artificial Intelligence” in 1956 is widely considered to be the founding event in the field of AI where this terminology was started. The term AI now refers to computer systems and technologies that behave in ways that resemble processes associated with human intelligence. AI in its myriad guises is already embedded in the financial sector, commerce, and science and is increasingly being employed and investigated in the healthcare sector.

3Although AI encompasses a number of technologies. In this literature review, AI mainly refers to machine learning. Machine learning encompasses the use of data and algorithms to mimic the human decision-making process, with the overall aim of improving its accuracy and is one of the most common forms of AI being used in different areas of healthcare. This is being used to predict the success rates of treatment protocols in specific patient cohorts, to predict the risk of disease acquisition using neural network learning, and to predict outcomes by deep learning where machine learning involves multiple variables and features.

4Unfortunately, new technologies take time to be embedded in healthcare and despite all the theoretical potential of AI, the uptake in healthcare has been uneven and slower than anticipated and there remain a number of barriers, both overt and covert, which have limited its incorporation. The aim of this review is to identify and examine these barriers to help produce a framework for health systems to implement AI.

5Materials and methods

6A review of reports published by healthcare organisations, arm’s length health bodies, and think tanks in the UK was undertaken. This helped to inform the search criteria and ensured that the work produced by the East and West Midlands Clinical Senates would add value, avoid duplication of previous reports, and aid the development of the data extraction tool. Organisations included the UK Government and Parliament, NHS England, NHSX, National Institute for Clinical Excellence (NICE), Health Education England, Care Quality Commission (CQC), Academy of Medical Royal Colleges, Royal College of Physicians, Royal College of General Practitioners, National Institute for Health Research (NIHR), Academic Health Science Networks - England, Reform, the Public Service Think Tank, and The Health Foundation. Although this list of healthcare organisations is not exhaustive, it has assisted in providing a greater understanding of the emerging AI healthcare landscape.

7Searches

8Searches of the following databases were performed on 22/12/2021: AMED, BNI, CINAH, EMBASE, EMCARE, HMIC, MEDLINE, PsycINFO, and PUBMED. The search had no limitations for year of publication, type of study, or language.

9The following terms were used:

10“Artificial intelligence” OR “deep learning” OR “machine learning”

11AND

12“Barriers” OR “obstacles” OR “difficulties”

13AND

14“Implementation” OR “application” OR “utilisation”

15Articles Screened by Title and Abstract

16Titles and abstracts were reviewed for potential inclusion in the full-text review according to the following criteria: articles highlighting the possible barriers to the implementation of AI in the healthcare setting and all research methodologies were accepted.

17Full-Text Article Assessment

18The selected full texts were included for further analysis and data extraction was done according to the following criteria: articles that have investigated barriers to the implementation of AI in healthcare, where a full-text version has not been published, an abstract only may only be included if there is a significant discussion about barriers to implementation, and an English language version of the study is available.

19This process was done independently by two screeners. Each screener was blinded to the other screener’s decisions and when both screeners had compiled a list of included articles these were compared. Any disagreements between the screening reviewers were resolved by discussion and if necessary, a third reviewer was involved as an impartial mediator.

20Secondary Searches

21All the references from the articles in the primary search were reviewed.

22Data Extraction Tool

23After review of the healthcare organisation reports, each reviewer independently compiled a list of the barriers to the implementation of AI. These were collated and refined into key broad themes of barriers and further divided into sub-themes to produce the data extraction tool that allowed qualitative assessment of the review articles. This framework was used to analyse the articles found during the literature searches.

24Data Synthesis Strategy

25Data are presented in key themes. We performed a narrative synthesis of the evidence for each of the key barrier themes.

26Results

27Search Results

28A total of 43 articles were included for data extraction following a review of the records identified through database searching. A further 16 articles were included following secondary searches; therefore, there were a total of 59 articles included for data extraction. Figure 1 shows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.

29Details of the articles and the barriers noted in these articles are highlighted in Table 1.

30Themes

31Six key themes were identified from the review of background literature that formed a framework which was used to analyse the articles. There were several key themes (and a number of articles discussing the theme) as follows: ethical barriers (39 articles), technological barriers (55 articles), liability and regulatory barriers (37 articles), workforce barriers (35 articles), patient safety barriers (24 articles), and social barriers (18 articles). These themes are explored in the discussion.

32Discussion

33The key themes identified from the literature are discussed in the order of prevalence within the literature. There was overlap between the themes and the key aspects are explored further.

34Ethical Barriers

35There are significant ethical concerns that act as barriers to the implementation of AI in healthcare and 39 studies reviewed referred to these ethical barriers. Within the theme, the most prominent were privacy, trust, consent, and conflicts of interest.

36Twenty of the studies in this review have highlighted that privacy may be a significant barrier when implementing AI in healthcare. The creation of accurate AI algorithms requires that developers have access to large datasets for training and there is an understandable apprehension that the use of this data may sometimes be at odds with the patient's right to confidentiality. In the European Union, the General Data Protection Regulation (GDPR) is clear and states that “patients own and control their own data and must give explicit consent for its use or when it is shared”.

37It is essential that patients are fully informed about the fate of their data, and it must be mandatory that they consent for its use when it is to be shared with AI developers. They should understand who can access their data, how it will be stored or used, and how their privacy will be protected.

38Data ownership and consent to store data was highlighted as a concern in 10 studies. He et al. suggested that with wider sharing of data, there is a need to redefine the concepts of patient confidentiality and data privacy. Discussions concerning the development of cybersecurity measures are inevitably going to become increasingly prominent to address emerging issues of improper data sharing.

39There is a need for clear governance of AI implementation in healthcare that clarifies the role of all stakeholders in data ownership and use. Marcu et al. highlighted the importance of having data privacy policies for secure data storage, usage, and sharing to avoid breaches of sensitive confidential data.

40Another barrier noted in 25 of the studies was concerns about whether AI technology could be “trusted”. Chen et al. noted that doctors struggled to trust AI for several reasons. Firstly, health professionals only have limited or no training in evaluating AI software. Most of the evidence relating to the use of AI in healthcare is retrospective and there is a paucity of randomised controlled trials on AI for clinicians to appraise. If these technologies do not have a rigorous evaluation prior to their introduction and use in the real world, this could lead to significant patient harm, irreversible loss of confidence for the medical profession, and inaccurate conclusions being made at the population level. There are also concerns that as these technologies are often being developed commercially with the main driver being profit for the parent company, this may affect the level of scrutiny that they undergo prior to their release into the clinical environment.

41Secondly, the overwhelming majority of algorithms have a lack of interpretability or transparency, and developers cannot (or will not) explain the basis of their decision making, the so-called black box phenomenon. Sakamoto et al. suggested that this could lead to issues with both validation and clinical oversight. It may also affect the doctor-patient relationship as clinicians may be unable to explain decisions made by AI algorithms to their patients.

42Another potential problem is the utility of AI in complex multisystem diseases. Health professionals may be reluctant to adopt AI into practice if they feel that the system is not sufficiently flexible or sophisticated to cope with complex conditions especially if the decisions cannot easily be reviewed and under these circumstances, trust becomes a significant barrier.

43Conflicts of interest have been highlighted as a barrier to AI implementation in five studies. Brady et al. highlighted this as a current issue for radiologists who may have commercial conflicts of interests by implementing AI; they may be involved in developing or potentially may even receive royalties from. They suggest that this issue should be addressed in an analogous fashion to issues of conflicts of interest while developing medications or medical devices.

44There is no overall consensus on how to deal with these complex ethical dilemmas, particularly in the absence of a strong regulatory framework and the lack of agreement among experts. It is however increasingly being recognised by AI developers that ethical analysis is difficult to incorporate into AI development. A new concept of algorithmic fairness is being incorporated where the AI technologies are focussed on identifying any latent biases and demonstrating an absence of between-group bias. However, this approach is not infallible and ethical dilemmas are inevitably going to emerge while implementing AI technologies in real-world healthcare settings in different patient populations. These ethical dilemmas particularly surrounding privacy, trust, transparency, and accountability are major barriers to the implementation of AI in healthcare settings. In the future, they will have to be addressed at the stage of AI development and incorporated into the training of healthcare professionals.

45Technological Barriers

46Technology is a central and prominent theme of barriers to AI. Forty-two articles discussed data barriers including data quality, data access, and data-set size. Twenty-three articles examined how AI’s application can be a barrier with issues relating to interoperability, usability, and integration into current workflows. Validation of data was raised by 18 articles and 15 of those discussed hardware and infrastructure. There were other barrier themes that emerged and are discussed in this section which are also explored elsewhere including explainability and transparency.

47Data quality, accuracy, and dataset size: Implementing large data sets into AI represents a considerable challenge. It is difficult to acquire and amalgamate large, high-quality data sets that are complete and diverse. AI platforms are limited by the concept of “what goes in is what comes out” implying that the algorithm is only as good as the data source “teaching” it.

48Apart from data quality, data quantity is relevant for the training of machine learning algorithms. When small data sets are used to validate AI there are inevitable problems. AI may not be able to differentiate between variations of normal and abnormal, it may not be able to allow for the effect of known confounders such as gender, age and ethnicity or unknown confounders which will reduce generalisability. It may also be blind to biases from using retrospective data, underrepresented patients, or medication selection.

49Many conditions or diagnoses cannot be clearly defined, leading to imprecise phenotyping of diseases and their treatments. There is also substantial variability between experts who evaluate data, and this variability may result in biased labels. This cannot be completely avoided but can be mitigated by having multiple experts evaluating the same case. Additional biases occur including over-fitting which results from the application of AI trained in one population to a different population without retraining on the new dataset. A balance needs to be found between widening access to data while ensuring confidentiality and respect for privacy. Recurring concerns are also raised regarding the utilisation of non-standardised reporting standards (or those that are not pre-defined) within datasets and with different sample characteristics.

50These arguments expose the brittleness of AI, where the tool which has been developed only works on the training data and is easily misled. Even with large datasets, AI is not infallible as the high-profile failure of GoogleFlu demonstrated.

51Although a huge number of guidelines do exist, often in narrative form, there remains a paucity of coherent consensus on the most effective of these guidelines and applications. Willemink et al. suggest that nearly all limitations can be attributed to one substantial problem, the lack of available data for the training and testing of AI algorithms. There is a clear lack of variety in the meta data and quality of information and in addition there is continued use of outdated machines, low-quality data, and the use of numerous open-source data sets restricted to research only use.

52Future aims should include the creation of data systems that will channel prospective data into AI algorithms that will prospectively adapt and change. To aid future comparisons, AI systems should be tested on analogous datasets so that performance between systems can be assessed and compared. Ultimately, there is a need for high-quality data to be standardised.

53The data AI is trained on is important. The quality and quantity of data on which the algorithms are trained are two important determinants of the performance and generalisability of the algorithm. Future AI datasets will need to include aggregates of multiple data sets that ensures algorithms match the depth and breadth of a doctor’s knowledge.

54Interoperability and usability of data and its integration into the workflow: A clear example of an interoperability barrier is seen where AI is applied to medical imaging. A major limitation of current commercial imaging systems is that they acquire annotations and storing them in a format which does not permit reuse in AI development. The lack of data interoperability between systems and the lack of standardisation in entered data ultimately affects the quality of predictive AI models. These models are yet to be integrated into clinical workflows in a fashion that improves clinical care or outcomes and there remains a divide between those using and those implementing the AI models providing a significant challenge for those responsible for the model when it is used clinically.

55Data validation: Validation of AI efficacy following its development is also a barrier and there is a lack of high-quality evidence and data to support AI systems use, with limited randomised controlled trials and a lack of prospective data with virtually all data to date having been acquired retrospectively. Ultimately, well designed, multi-site, multi-centre (ideally heterogenous population) validation studies will be necessary to understand the real-world impacts of AI in healthcare and explore robustness, interpretability, and trust of data.

56Hardware and infrastructure: There is currently no clear commercial or institutional leader in the field of healthcare AI. This lack of established leaders of AI in individual countries and national healthcare systems results in the use of different products to address single areas of need, making the identification, selection, and utilisation of AI extremely challenging.

57Presently, no AI programme continually receives feedback about its diagnostic performance, a task that is essential to learning from experience. In addition, a major technical challenge is the development of “a reasonable” detection rate of abnormalities without an excessive rate of false-positive findings compared with a comparable clinician-based system.

58The widespread implementation of AI will require substantial reconfiguration of IT systems which will have resource implications and currently it is difficult if not impossible to allocate sufficient investment where there are competing demands for healthcare equipment. Moving forward there are vital prerequisites including support systems, strategic plans, and budget planning and the infrastructure for data sharing and secure transfer of data is costly and presently lacking. There is also the risk of data breaches which reduces the number of organisations who will be prepared or able to adopt even the most promising initiatives.

59Three papers discussed security being an issue relating to AI implementation. As AI becomes increasingly widespread, algorithms may be the target of cyber-attacks and there is concern that AI models could subsequently cause deliberate harm or be used for fraudulent purposes. A particular challenge that emerged from the review of the literature is the issue organisations face when trying to assess myriad vendors marketing AI platforms in the absence of any established databases or source of information and feedback relating to AI vendors and companies.

60Explainability, transparency, and the black box theory: Another barrier to AI implementation was explainability, transparency, and the complex issue of the black box theory. The “black box” phenomenon refers to when an AI algorithm reaches a conclusion without users being able to understand the mechanism or basis or ‘see inside’ the system. This concern in explaining AI decision making exposes the technical vulnerabilities with a lack of human-interpretable explanation for model predictions and is strong evidence that significant issues remain before AI is in apposition to replace clinicians. There also remain concerns about the discordance between scientific advances and the on-going announcements made in the media about advances in AI and there a persistent tendency to focus on the innovative aspect of the tool and not its utility.

61Liability and Regulatory Barriers

62The theme of regulation was prominent with 37 papers discussing this issue and describing it as significant. Within the theme, the most important sub-theme (30 papers) was the question of who or what was responsible for the outcomes and decisions provided by an AI. The other prominent sub-theme was regulation and legal aspects of AI, which includes data governance (23 papers).

63Liability: Addressing liability relating to the use of AI is a particular challenge and clinical staff are used to taking responsibility for their decision making. The issue becomes more complex when decision-making is guided by, or even entirely devolved to AI and questions relating to who or what is liable for a bad outcome where AI have been utilised are ubiquitous. Candidates include the clinicians who used the AI tool, the software developer who designed and developed the system, the vendor who retails the product, the healthcare service that purchased it or the regulator who approved it. This issue remains unanswered due to the combination of the legal and ethical questions raised and presently healthcare professionals remain liable for the decisions they make, even if based on an AI algorithm for which they may have little or no insight.

64This lack of attributable accountability is likely to prevent those medical professionals who are currently responsible for clinical decisions from embracing the technology. Contrary to this issue however, if a professional fails to use or abide by the advice of an AI tool and there is a poor outcome, it is conceivable that this may be considered clinical negligence. Consequently, the serious complications that may occur as a result of these decisions will deter physicians who will inevitably not be prepared to be responsible for outcomes from the use of new and emerging technologies which function and are tested in ways they do not fully understand and are not privy to.

65Regulation and legal barriers: For users and developers, both regulation and a lack of regulation can act as barriers to the adoption of AI in healthcare. Systems and professional regulators need to properly understand and trust the operation of the AIs that will be employed in their speciality to be able to regulate effectively. Unfortunately, transparency is lacking or often completely absent in most of the presently available technologies. This is another consequence of the “black box” problem where the inner workings of a system are not understood and therefore not trusted. This is a prominent barrier and has a profound impact influencing whether society and regulatory bodies are prepared or able to trust an AI in routine clinical practice.

66The absence of a specific approval system for AI is largely based on this lack of trust, but even where they exist the delay of approval can hinder progress often for considerable periods. There are additional challenges for regulators due to the lack of an agreed framework for the evaluation of the performance of these algorithms with no “gold standard” or accepted baseline for comparison. Further questions remain regarding a method to continuously regulate AI algorithms during cycles of iterative modifications and updates.

67Regulation aimed at protecting commissioners and patients results in barriers for developers and it is accepted that rigorous top-down regulation could stifle the AI innovation. In 2021, the UK Government published guidance outlining their expectations for AI development in the NHS, covering the need for the transparency of algorithms and accountability. The onus is on developers to meet these regulatory requirements, although the guidance acknowledges that it can become complex when a product performs actions that fall under the aegis of multiple regulatory bodies.

68Data and information governance is also perceived as a significant barrier to the implementation of AI systems. The GDPR guidance outlines a comprehensive set of regulations for the governance of personal information which became effective in May 2018 and influences AI implementation in several different ways. Specific arrangements may therefore need to be developed to provide necessary protection for patients, organisations, and clinicians using data for AI systems. The most appropriate regulatory approach for AI remains to be clarified but will need to be addressed to remove this hurdle and encourage developers.

69Workforce Barriers

70The theme of workforce is considered an important barrier to the adoption of AI with 35 studies citing it as a major concern. Within the theme, the education and training of the workforce was the most prominent aspect referred to in 27 studies. Other barriers identified related to workforce and there was considerable overlap; willingness/engagement of the workforce (13 studies), fears of job displacement (9 studies), finances (7 studies), and time (6 studies).

71Education and training: The education and training of the healthcare workforce is a significant barrier to the adoption of AI and was highlighted in 27 studies. Detailed knowledge regarding the potential and workings of AI in the medical community remains rudimentary and considerable AI education and training will be needed. The varying levels of technological literacy, satisfaction with and understanding of the existing technology will need to be overcome before the workforce that will be required to use the technology are comfortable or competent in clinical practice. Unfortunately, frustrations may potentially become more acute as physicians learn how to incorporate AI platforms while still struggling with existing healthcare technologies. Healthcare providers will need to develop training programmes specifically targeted at clinicians required to use AI systems and designed to ameliorate the multiple concerns resulting from unfamiliar technology.

72Clinical staff will also need to be educated to enable them to critically analyse the opportunities, pitfalls, and challenges associated with AI. Unfortunately, today’s medical education system is lacking in AI training representing a significant barrier in both the medium and long-term. Future medical undergraduate and postgraduate curricula should be updated to include a basic understanding of AI methodology and limitations including advanced statistical and computational skills. Some medical schools have already begun to include AI education courses in their curricula, although educators do not fully understand how AI will affect clinicians’ roles. In addition, there are limited individuals in medical faculties who are AI competent and capable of teaching the relevance and importance of AI in the healthcare setting. The barrier resulting from a lack of education also extends beyond clinical staff, as specific technical expertise and mathematical knowledge is required to develop and utilise AI tools and proficiency is still rare within healthcare settings.

73Willingness/engagement: Physician ‘buy-in’ and engagement remains a necessary and uniquely challenging barrier for AI and was discussed in 13 papers. Some physicians do not engage fully with a new AI process and this lack of engagement may produce clinical problems and ignorance of or reluctance to understand AI could lead to poor patient outcomes. Without ensuring adequate engagement of clinical staff and healthcare leaders even the best AI tools are unlikely to be accepted and integrated and will be unable to improve clinical outcomes.

74When a system is perceived by clinicians as threatening and interfering with autonomy and critical thinking, uptake will inevitably be compromised and consistent and determined effort will be essential to nurture clinicians’ engagement with the AI tool. An additional but separate issue for a number of clinical staff is scepticism regarding the current capabilities for AI in particularly complex patients or with conditions where management is known to be difficult requiring input from multiple specialities to achieve the best outcome.

75Engagement of clinicians can also be affected by the required algorithm-human interaction with the AI technology. Information overload or alert fatigue are barriers to using AI tools. Further work is required to understand this algorithm-human interaction to ensure maximum benefit is derived from the technology and system development should consider the need to produce systems which are user-centred.

76Job displacement: The fear of job displacement is a consistent theme and clinicians are not immune to the fear of unemployment or radically changed job plans arising from the emergence of increasingly complex and competent AI. This concern could theoretically lead to behaviours and practices in the future designed to ensure the continuing relevance and roles of human practitioners in healthcare.

77In clinical specialities where investigations and results can be readily digitised and interpreted by autonomous AI there are already concerns and ophthalmologists, cardiologists, pathologists and radiologists fear deskilling and the change in their role in clinical pathways. As technologies improve this could lead to over-reliance on non-clinical systems and AI needs to be used appropriately and sensitively, allowing the role of healthcare providers to evolve with their introduction with the new technology augmenting rather than replacing clinical staff. Whether AI systems will eventually replace radiologists is the wrong question, the more apt question to be asked is “will radiologists who use AI replace radiologists who don’t”.

78Finances: Workforce funding limitations are a clear barrier even when projected efficiencies from the introduction of AI systems are considered. Expert input for training data for AI algorithms is very expensive to obtain (58 studies) and AI maintenance activity will require significant ongoing funding. Health system leaders must prepare for significant investment in personnel and technology to test, validate, implement, and improve AI tools. Calculating the impact of the changes and allocating healthcare budgets appropriately will be confounded by the likelihood that the benefits and costs of using AI may be unequally divided between departments and providers.

79Time: The time required for the training of AI systems and certainly in the early stages following implementation needed to utilise the systems is likely to remain a barrier for the foreseeable future. Initially AI will require data, some of which will have to be collected by the healthcare workforce. In one study of the implementation of a decision support AI tool, the largest resource barrier was found to be personnel time and tasks such as labelling of raw data are tedious and time-consuming and will not encourage staff to embrace the new technology. The education and training of the healthcare workforce has been discussed in the context of the expertise not being presently available, and the topic not included in curricula, but it will also be important for healthcare leaders and clinicians to engage with developers and providers to ensure that adequate funding and time is allocated.

80Patient Safety Barriers

81It is believed that AI will have a positive impact on improving patient safety with several studies demonstrating that its utilisation will reduce mortality and improve diagnostic speed and accuracy. AI however is not a panacea and 24 studies raised concerns about the impact of AI on patient safety and these safety issues will need to be carefully considered during introduction into clinical practice. Failure to mitigate these safety issues could dramatically affect the implementation of the technology in healthcare with patients and staff rapidly losing confidence.

82Distributional shift: The first patient safety issue to highlight is the concept of “distributional shift” which may be familiar to clinicians who find that their previous clinical experience may be inadequate when encountering new situations. Clinicians are used to working in these difficult environments with limited or poor data and are good at weighing up the cost and consequences of their decisions as well as the impact of false positive or negative results on their patients’ condition. This can often lead to a cautious approach, especially when there is uncertainty in the diagnosis or the best management whereas contrasted with Al algorithms which are best utilised on controlled, curated datasets similar to the data with which they were trained. Real-world data is also likely to be less tightly grouped and will display regional and population variations and be influenced by the equipment being utilised and differing levels of curation. AI systems do not reliably recognise important context or a pertinent change in data that should alter its predictions. Secondly, most tools perform a region of interest analysis to investigate a single disorder of interest and are not created to investigating multiple disorders or complex multisystem complaints synchronously. This can result in a system that fails to identify secondary pathologies or that will confidently make incorrect predictions without considering individual context, misleading clinicians and acting as a barrier with clinicians reluctant to consider its use in the future.

83Automation bias: As AI becomes increasingly accepted and incorporated into healthcare management, there is a risk that clinicians will become complacent and accept that guidance from an automated systems is infallible and consequently fail to search for confirmatory evidence, a practice known as automation bias. This situation may deteriorate over time as algorithms age and may cease to recognise changes in diseases, clinical practice and medication and may reinforce outdated or inappropriate practice. For example, when a usually reliable AI system encounters unrecognised data, it may not ‘fail safely’, but make erroneous predictions which may not be questioned by the busy clinician who would previously have considered alternative diagnoses or treatments. Another concern is that established AI algorithms may be poorly receptive to new medical innovations as there would be no prior data to retrain the system inhibiting medical innovation and preventing it being safely introduced into daily practice.

84Improvements and evolution of AI algorithms: AI algorithms will use whatever signals are available to achieve the best possible performance, which is why they have been so well utilised playing chess, poker, and alpha go. They can dynamically learn new rules and explore new techniques to improve their performance, the key to mastering games, but this may be a significant risk in healthcare. AI in healthcare may attempt to ‘game the system’ and learn to deliver results that appear successful in the short-term fulfilling their initial remit but running contrary to longer term health goals or stopping the system becoming generalisable. For instance, in one classic example, an AI model did not learn the intrinsic difference between dogs and wolves, but instead learned that wolves are usually pictured standing on snow, while dogs usually appear on grass. Further concerns have also been raised on the ability of autonomous AI systems to safely push the boundaries of treatments to discover new strategies to improve their current outcomes. Although this may lead to eventual improvements in care there is also a risk it will lead to multiple failures and there must be strict guidelines detailing which new strategies are safe to explore.

85It is beyond doubt that AI technologies have the potential to irreversibly change medicine and bring about improvements in patient management and diagnosis. It is nevertheless important that AI systems are designed to constantly assess their own confidence in the predictions they make and to include failsafe behaviours in which they would refuse to make a diagnosis if there was poor data or low confidence in the end result. This section also indicates the need for truly “explainable AI”, rather than the opaque black box algorithms that do not allow for interrogation of the decisions that have been made. Understanding the algorithms would allow humans to assess the likelihood of a level of confidence for each diagnosis in real-time rather than when a mistake has been made. It is crucial that patient safety remains at the heart of AI both for the management of individual patients and because any safety issues, particularly any which are high profile or enter the public domain would have a dramatic and immediate effect on confidence in the technology and could act a barrier to the technology's eventual success. This is analogous to a self-driving car which even if orders of magnitude safer than a human driver would instantly become difficult to implement if it resulted in the death of a pedestrian.

86Social Barriers

87The conditions to which we are born, live, and work effect all aspects of our mental and physical wellbeing and our ability to access equitable healthcare. Health inequalities are unjust and avoidable differences in health across the population and between different groups within our society and are unacceptable.

88This gap in health can be further exacerbated by healthcare bias, where patients receive different standards of care because of their race, age, gender, or other characteristics. It is therefore of paramount importance that any new technology does not contribute to the inequalities in society and actually aims to reduce the inherent bias in a healthcare system. Health inequalities and healthcare bias was discussed in 19 articles in the context of acting as a barrier to the successful implementation of AI in healthcare.

89Selection bias: There is huge potential that the under-representation of different cohorts in the initial formation of an AI model will lead to reduced accuracy and efficiency in these groups, a form of selection bias. For example, a machine learning algorithm was found to be less accurate at the detection of melanoma in darker skinned individuals, as it had mainly been trained on fair skinned patients. It is also mandatory that vulnerable and minority groups are engaged by developers of machine learning and artificial intelligence models to ensure the data fully capture the diverse healthcare landscape and avoids perpetuating healthcare bias. As a minimum full disclosure of the demographics of patients that have been used to train and validate these models would go some way to ensuring that they match the target population and reduce inherent selection bias which would render the technology unfit for purpose.

90McCradden et al. raised some interesting points in this respect and acknowledges that in many cases it is difficult to distinguish between acknowledging difference and propagating discrimination. The difficulty arises when trying to ascertain if these differences may have a causative and tangible effect on the outcome. For example, acknowledging that gender may impact the effect of pharmacological compounds is not unjust and would ensure the safest algorithm was created. However, including non-causative factors in algorithm design would lead to no tangible benefit and could lead to further health-related discrimination. One possible way to help mitigate some of these issues is to ensure AI products use fairness solutions to help create bias-neutral models. These solutions can omit variables such as gender or race from the model, derive outcomes that are independent of these factors after controlling for their estimated risk, or mathematically balance benefit and harm to all groups.

91Digital health inequality: Individuals may have different access to AI driven healthcare, due to cultural, geographic, economic, or a number of other factors. As Ahmad et al. observed there is a risk that a two-tier health system could emerge where more AI-enabled healthcare organisations provide superior patient care and better outcomes. The NHS should strive to “level up” infrastructure equally across the country to ensure that there is not a postcode lottery in respect of access to these new technologies. There also needs to be a concerted effort to reduce widening digital exclusion that would limit the uptake of these technologies especially in vulnerable groups.

92Social acceptance: Ten studies indicated that social acceptance of artificial intelligence models remains a barrier to implementation. Singh et al. states that the aim of AI implementation is to create value for the organisation making healthcare more affordable and efficient by identifying key performance indices and tracking return on investment. However, the ‘black box’ nature of AI acts a significant barrier to its social acceptance, and it is not clear whether patients will be willing to accept a diagnosis from a computer instead of one from a clinician especially if it is perceived that it is designed to save time and money without clear evidence that quality has not been compromised. In an autonomous diagnostic setting, will patients depend on non-expert device operators for comfort and clarification. Loncaric et al. have predicted that there will be a loss of the holistic medical approach as healthcare shifts towards data analytics and away from patient interactions. However, the study notes that presently patient preference is dramatically weighted towards human interaction rather than automation, even if it means lower performance, as above all else as humans unlike presently available machines are able to provide gentleness and compassion.

93The significant barriers to implementing AI in healthcare have been highlighted and discussed in this review. Defining and understanding the barriers preventing the acceptance and implementation of AI in the setting of healthcare is a crucial first step enabling clinical staff and healthcare managers to overcome the identified hurdles. The development of strategies and systems to overcome these barriers will be the real challenge. Gaining a complete understanding of the barriers to using AI in healthcare is an important consideration for those designing and delivering healthcare within rapidly changing healthcare systems. It is especially important to understand which of the barriers are the most relevant in real-world health settings so strategies can be developed to overcome them.

94AI faces significant barriers that need to be overcome before its widespread, safe, and successful use is achieved in healthcare. Its expansion seems inevitable, but only by addressing these barriers thoughtfully can the potential of AI be used for the good of the health of the population and the healthcare system.