Introduction

Digital transformation in the healthcare sector has entered a more complex phase with the presence of Artificial Intelligence. Intelligence (AI) as an integral part of clinical decision-making. AI is no longer just an administrative tool, but has evolved into a Clinical Decision A Diagnostic and Statistical Support System (CDSS) capable of analyzing large-scale medical data and providing rapid and precise diagnostic and therapeutic recommendations. In practice, this technology is used in various fields, such as radiology, digital pathology, chronic disease prediction, and chatbot- based triage systems .

Despite the promise of increased accuracy and efficiency, the use of AI in healthcare raises complex legal challenges . One crucial issue concerns the legal liability of healthcare workers in the event of medical errors involving AI systems. This is further complicated by the fact that AI often operates as a black box. box system , where the algorithmic decision-making process cannot be fully understood even by the medical personnel who use it [1].

In the context of positive law in Indonesia, the responsibilities of health workers are generally regulated in Law Number 17 of 2023 concerning Health and the Law Number 29 of 2004 concerning Medical Practice. Both regulations emphasize the principle of professionalism. standard and duty of care as the basis for accountability. However, these regulations do not explicitly address the use of AI as a new variable in medical practice. As a result, a legal vacuum has emerged regarding the boundaries of responsibility between healthcare workers as users, healthcare facilities as providers, and technology developers as creators of AI systems.

This issue is not hypothetical, but has emerged in global practice. One widely discussed case is the use of patient risk prediction algorithms in the United States, developed to assist in the allocation of healthcare. Research has shown that these algorithms have significant racial bias, resulting in Black patients with serious health conditions being overlooked as high priorities [2]. In this context, a fundamental question arises: can healthcare professionals be held accountable for decisions influenced by inherently biased systems?

Another case occurred in the implementation of AI in breast cancer detection in Europe, where the AI system failed to identify a number of cancer cases at an early stage ( false negative ). Even though doctors use the system as an aid, legal responsibility remains directed at medical personnel because they are considered to have the final authority in clinical decision-making [3]. This shows a tendency for the legal system to continue to place humans at the center of responsibility, even though these decisions are influenced by complex technology.

From a legal theory perspective, this condition creates tension between the fault-based approach liability that focuses on individual errors, with the need to accommodate risk-based Liability in the context of advanced technologies such as AI. If the responsibility is placed entirely on healthcare workers, this has the potential to give rise to defensiveness. medicine , where medical personnel are reluctant to utilize technology due to high legal risks. Conversely, if the responsibility is shifted entirely to technology developers, it can neglect the role of health professionals in the decision-making process [4].

Furthermore, this ambiguity impacts legal certainty and legal protection for healthcare workers and patients. Without a clear regulatory framework, the use of AI can actually increase the risk of medical disputes and injustice in law enforcement. Therefore, an in-depth normative analysis is needed to examine how the law should regulate the relationship between humans and technology in healthcare.

Based on this background, this research focuses on analyzing the use of AI in medical diagnosis and services and its legal implications for the responsibilities of healthcare professionals. Specifically, this research aims to identify potential malpractice risks arising from the use of AI and evaluate the extent to which the existing legal framework provides legal certainty in addressing these technological developments.

Method

This research uses a normative juridical method that focuses on the analysis of positive legal norms and legal doctrines that have developed in response to the use of Artificial Intelligence. Intelligence (AI) in healthcare. This approach was chosen because the issues studied are not solely related to medical practice, but also concern the gap in norms and the need for new legal constructions that can keep pace with the rapid development of technology. In this context, law is no longer understood as a static system, but rather as an instrument that must adapt to digital disruption, particularly in the healthcare sector, which poses a high risk to human safety [5].

The research approach used includes several complementary methods. First, the statute approach . The approach was conducted by examining various relevant regulations, such as Law Number 17 of 2023 concerning Health and Law Number 29 of 2004 concerning Medical Practice. The analysis focused on how these norms regulate the responsibilities of healthcare workers and the extent to which existing provisions are able to accommodate the use of AI in medical practice. In this case, it was found that existing regulations are still oriented towards conventional relationships between medical personnel and patients, so they do not explicitly anticipate the involvement of algorithm-based technology in clinical decision-making [6].

Next, the conceptual approach ( conceptual approach ) is used to examine relevant legal concepts, such as legal responsibility ( liability ), malpractice , standards of care and duty of care . This approach also involves analysis of technological concepts such as black AI and algorithmic bias, which are important factors in determining the level of error and accountability in the use of AI. In modern legal literature, the idea of shared liability or shared responsibility between health workers, health service facilities, and technology developers in response to the complexity of AI-based systems [7].

In addition, this research also uses a case study approach . approach ) by examining a number of actual cases related to the use of AI in healthcare services in various countries. This approach aims to identify patterns of legal issues that arise in practice, particularly related to misdiagnosis, algorithmic bias, and AI system failures. For example, the case of the use of predictive algorithms in the United States, which was proven to have racial bias in determining patient priorities, is one concrete example of how technology can have significant legal impacts [8]. Analysis of such cases is important to provide empirical insights that strengthen normative arguments.

To enrich the analysis, this study also adopts a comparative approach . approach ) by examining the legal framework in several jurisdictions, such as the European Union through Artificial Intelligence Act and ethical guidelines of the World Health Organization (WHO) [9]. This approach aims to find a more progressive regulatory model and can be used as a reference in building a national legal system that is responsive to technology. The European Union, for example, has classified AI in the health sector as high-risk. a system that requires strict supervision, thus providing a clearer direction in terms of legal accountability.

The legal materials used in this study consist of primary, secondary, and tertiary legal materials. Primary legal materials include laws and regulations and official documents that have binding force. Secondary legal materials include academic literature such as scientific journals, textbooks, and research reports relevant to the topic of AI and health law. Meanwhile, tertiary legal materials are used as supporting materials to clarify the terminology and concepts used in this study.

The collection of legal materials is carried out through library studies . research ) by exploring various literature sources, both conventionally and digitally. This search was conducted systematically to ensure the materials used were highly relevant and credible. Furthermore, a qualitative analysis of the legal materials was conducted using legal interpretation techniques, including grammatical, systematic, and teleological interpretation . This approach allows researchers to not only understand the textual content of norms but also interpret their purpose and relevance in the context of AI technology developments.

Finally, this study employs a prescriptive approach, providing legal arguments and normative recommendations regarding the ideal accountability model for the use of AI in the healthcare sector. Through this approach, the research goes beyond identifying problems and also seeks to offer solutions that can strengthen legal certainty and provide balanced protection for healthcare workers and patients in the digital age.

Results and Discussion

A. Use of AI in Medical Diagnosis

Utilization of Artificial Intelligence (AI) in medical diagnosis has revolutionized the way healthcare professionals identify and predict diseases. AI, particularly through machine learning approaches, learning and deep learning capable of analyzing large amounts of medical data with a level of complexity that is difficult for humans to achieve. In clinical practice, this technology is widely used in the fields of radiology, pathology, and cardiology, where AI is able to detect anomalies based on medical images such as CT scans , MRIs, and mammograms with a high level of accuracy [10].

However, the effectiveness of AI in medical diagnosis cannot be separated from the epistemological issues related to how a medical decision is made. In Black's theory, In AI, the system produces output without providing an explanation that can be transparently verified by the user. This raises serious issues in the medical context, because a diagnosis is not just an end result, but also a rational process that must be scientifically and legally accountable [11]. When healthcare professionals do not understand the logic behind AI recommendations, there is a shift from informed decision -making towards assisted opacity , which is a condition where decisions are made based on a system that is not fully understood.

A recent case illustrating this problem occurred in the implementation of AI in breast cancer screening in the UK through the National Health Service (NHS). In 2024, a trial of an AI system for reading mammograms showed that while AI reduced the workload of radiologists by almost 30%, there were still a number of cases where the AI failed to identify early-stage cancer lesions ( false negative ) [12]. In such situations, radiologists who rely too much on AI results tend not to perform in-depth follow-up examinations. This phenomenon is known in the literature as automation bias, which is the human tendency to trust automated systems more than their own judgment, even though the system is not always accurate [13].

standard theory of care , the use of AI raises fundamental questions about the professional standards that healthcare workers must meet. Traditionally, standards of Care is measured based on the actions taken by competent medical personnel in similar situations. However, with the advent of AI, these standards are transforming. Healthcare workers are not only required to make independent diagnoses but also to be able to evaluate and critique the output produced by AI systems. Therefore, professional competence is no longer limited to clinical aspects but also encompasses technological literacy [14].

At the same time, the use of AI in diagnosis is also closely related to risk theory. society as proposed by Ulrich Beck. In modern society, risks no longer stem solely from natural factors, but also from human-created technology. AI in this context is manufactured risk , namely the risk that arises as a consequence of technological innovation itself [15]. In medical diagnosis, this risk manifests itself in the form of algorithmic errors, data bias, and systemic dependence on technology. This shows that the use of AI not only creates solutions, but also generates new types of risks that require different management approaches.

Another relevant case is the use of large-scale AI-based models. language models in digital medical triage services in 2023–2025. Several studies have shown that these systems are capable of providing fairly accurate initial recommendations, but in certain cases provide erroneous or unsafe advice due to limited understanding of the patient's clinical context [16]. This suggests that AI has limitations in understanding complex clinical nuances, such as patient history or comorbid conditions , which are often key factors in medical diagnosis.

From this analysis, it can be concluded that the use of AI in medical diagnosis should be positioned as an assistive tool . tool ) and not as a substitute for clinical judgment. The human-in- the - loop theory is relevant in this context, where humans remain the primary decision-makers, with AI supporting them. This approach is crucial for maintaining a balance between the use of technology and the professional responsibilities of healthcare workers.

Critically, the primary challenge lies not in the technology's capabilities, but rather in how it is integrated within existing medical practice and legal frameworks. Without clear standards for the use of AI, there is a risk that this technology will actually degrade the quality of clinical decision-making due to over-reliance. Therefore, strengthening regulations and ethical guidelines that emphasize transparency, accountability, and the competence of healthcare professionals in using AI is necessary.

B. Malpractice Risk Analysis in the Use of AI

Use of Artificial Intelligence (AI) in medical services has direct consequences for the expansion of the meaning and legal construction of malpractice . In the classical concept, malpractice is understood as a form of negligence by health workers that does not meet professional standards ( standards). of care ) and cause harm to patients. However, with the presence of AI as part of the clinical decision-making process, the structure of responsibility becomes more complex because it involves interactions between humans and technological systems.

One of the main risks in the use of AI is the emergence of automation bias, namely the tendency of healthcare professionals to uncritically accept system recommendations. In clinical practice, this condition can reduce the independence of professional judgment and potentially lead to misdiagnosis. A recent study showed that in the use of AI systems for the detection of cardiovascular conditions, healthcare professionals tended to follow AI results despite contradictory clinical indications [17]. In a legal context, this situation can be classified as negligence if healthcare professionals fail to adequately verify the system's output .

A similar case occurred in the implementation of an AI sepsis prediction system in the United States. Research showed that the algorithm used by a large hospital failed to identify a significant proportion of critically ill patients, despite the system's widespread clinical use. This failure was not only due to technical flaws, but also because healthcare professionals relied too heavily on the system without conducting further evaluations [18]. From a legal perspective, this raises questions about whether the error was a form of individual negligence or a systemic failure involving the technology developers.

Furthermore, the risk of malpractice also arises from algorithmic bias, a condition in which an AI system produces unfair or discriminatory output due to unrepresentative training data. The case of a healthcare management algorithm in the United States demonstrated that the system systematically disadvantaged patients from certain racial groups in determining treatment priorities [19]. In this situation, healthcare professionals using the system may be indirectly involved in discriminatory practices, even without intent. This broadens the dimension of malpractice from mere negligence to a potential violation of the principle of justice in healthcare.

Doctrinally , this condition challenges the application of the fault-based concept . Liability has long been the basis for accountability in malpractice cases . In AI-based systems, errors cannot always be traced to individual actions, but are often the result of complex interactions between various system components. Therefore, a risk-based approach is emerging. liability , which focuses on risk distribution rather than proving individual fault alone. This approach is more relevant in a technological context because it recognizes that risk is an inherent consequence of using complex systems.

malpractice elements in the context of AI has also shifted. The duty element of Care remains inherent in healthcare workers, but its scope has become broader, including the obligation to understand the limitations of the technology used. Elements of breach of duty ) is no longer only measured by direct medical actions, but also by the failure to criticize or supervise the use of AI. Meanwhile, the element of causality ( causation ) becomes more difficult to prove because it involves technological factors that are not completely transparent [20].

In practice, there is a tendency for healthcare professionals to remain the most vulnerable parties to be held accountable, even when errors originate from AI systems. This is due to the principle that doctors are the ultimate decision-makers . decision-maker ). However, this approach becomes problematic when AI has a significant influence on those decisions. If responsibility remains solely with healthcare workers, this has the potential to create inequities and hinder the adoption of technology in healthcare.

Critically, the risk of malpractice in the use of AI cannot be addressed simply by expanding the responsibilities of healthcare professionals. This requires a more comprehensive approach through the concept of shared healthcare . liability , where responsibility is shared between healthcare workers, healthcare facilities, and technology developers. Furthermore, clear operational standards regarding the use of AI need to be developed, including mandatory system validation, training of healthcare workers, and algorithm audit mechanisms. Based on this, this analysis shows that the use of AI will not only transform medical practice but also require a reconstruction of the concept of malpractice in healthcare law. Without these adjustments, the legal system will struggle to provide justice and certainty in handling technology-based medical disputes.

C. Legal Certainty for Health Workers

Legal certainty for health workers in the use of Artificial Intelligence (AI) in medical services is a central issue, highlighting the gap between technological development and regulatory readiness. In the Indonesian context, despite the enactment of Law No. 17 of 2023 concerning Health and Law No. 29 of 2004 concerning Medical Practice, both legal instruments are still based on the conventional medical relationship paradigm, which places healthcare professionals as the primary actors in clinical decision-making. This paradigm has not fully accommodated the new reality in which medical decisions are the result of collaboration between healthcare professionals and algorithmic systems that operate automatically and adaptively.

Legal uncertainty arises from the lack of explicit regulations regarding the legal status of AI in medical practice. AI is not clearly classified as a medical device, a decision support system, or an entity with its own liability implications. As a result, when a misdiagnosis or medical procedure involving AI occurs, the determination of the legal entity responsible is inconsistent. In practice, healthcare workers remain the parties most often held accountable because they are considered to have a duty of care. of care and ultimate control over clinical decisions [21].

However, this approach is beginning to be questioned in modern legal doctrine. In technology-mediated theory In decision -making, decisions made in AI-based systems can no longer be viewed as purely individual decisions, but rather as the result of a socio-technical process that simultaneously involves humans, data, and algorithms. Based on this, placing all responsibility on healthcare workers without considering the contribution of technological systems can create normative injustice . imbalance ) [22].

The implementation of AI in various global healthcare systems reinforces this problem. For example, the use of AI in patient triage systems in several European hospitals shows that treatment prioritization decisions are often influenced by opaque algorithms. When prioritization errors occur, resulting in delays in patient care, healthcare workers remain the first to be held accountable, even though the decision was largely determined by the system [23]. This demonstrates that the law still operates under the assumption of full human control, when in practice, that control is distributed.

Viewed from the perspective of legal certainty theory , this situation gives rise to three main problems. First, the vagueness of norms. of norms ) related to the status and function of AI in medical services. Second, inconsistencies in determining the subject of legal responsibility. Third, the legal system's inability to anticipate the risks of dynamic and adaptive technology. These three issues contribute to the increased risk of litigation against healthcare workers and the potential emergence of defensive practices. medicine [24].

When compared with the development of international law, there is a more progressive approach through the European Union Artificial Intelligence Act that classifies AI in the healthcare sector as high-risk system . This regulation emphasizes the principles of human oversight , algorithm transparency, and the division of responsibility between developers, service providers, and end users. This approach demonstrates that legal certainty is not only achieved through enforcing individual responsibility, but through regulatory design that governs the technology ecosystem as a whole [25].

Based on these conditions, Indonesia needs to construct ideal norms to regulate the use of AI in the healthcare sector. First , normative recognition is needed that AI in medical services constitutes medical care. decision support system , not an independent decision-making entity. Thus, primary responsibility remains within the collaborative framework between humans and technology. Second , a shared management scheme needs to be established. Structured liability , which divides responsibility proportionally between three main actors: healthcare workers as end users, healthcare facilities as system providers, and technology developers as algorithm creators. This division should be based on each party's level of control and contribution to the occurrence of medical errors.

Third , regulations are needed regarding minimum standards for the use of AI in medical practice, including mandatory clinical verification of AI output , regular audits of algorithmic systems, and specific training for healthcare workers on digital medical literacy . Without these standards, the use of AI has the potential to create uncritical dependency and increase the risk of misdiagnosis. Fourth , the law needs to adopt a risk-based approach. Regulation , that is, regulation that focuses not only on errors that have already occurred but also on risk prevention from the system design stage. This approach aligns with the predictive and adaptive characteristics of AI, so legal regulation must be proactive, not reactive.

Ultimately, legal certainty for healthcare workers cannot be achieved simply by expanding individual responsibilities, but must be achieved through systemic reforms that encompass technological regulation, institutions, and professional standards simultaneously. Without such reforms, the imbalance between technological and legal developments will continue to increase the risk of injustice in healthcare practices.

Conclusion

Use of Artificial Intelligence (AI) in healthcare has brought about a significant transformation in the process of diagnosis and medical decision-making, particularly through the use of clinical decision support A system capable of improving the efficiency and accuracy of clinical analysis. However, studies show that this technological advancement has not been adequately accompanied by the development of a legal framework governing the responsibilities of healthcare workers, creating a normative gap that impacts legal uncertainty. In practice, AI cannot be positioned as a neutral entity that functions solely as a technical aid, as this system has black box characteristics. Box , data reliance, and the potential for algorithmic bias that can influence diagnostic results. This situation has led to a shift in the structure of medical decision-making from an individual model to a complex socio-technical model. However, positive law in Indonesia still places healthcare workers as the primary party responsible based on the principle of duty of care. of care and final decision authority, without explicitly regulating the contribution of AI systems in the process.

This study found that a legal approach that is entirely based on individual fault ( fault-based) liability ) is no longer adequate to address the complexity of AI-based medical risks. This is evident in the increasing potential for automation bias, reliance on algorithmic systems , and the difficulty in determining causality when misdiagnoses occur. In this context, healthcare professionals are in a vulnerable position because they remain primarily responsible even though medical decisions are the result of interactions between humans and technology. Based on a comparison with international regulatory developments, particularly the European Union Artificial Intelligence ( EUAI), Intelligence Act , it is seen that the risk - based approach The approach with human oversight and a clear division of responsibilities between developers, service providers, and users is a more adaptive model. This model provides a more definite direction in building accountability without stifling technological innovation.

Therefore, this study concludes that it is necessary to reformulate health law in Indonesia which explicitly recognizes AI as decision-support. systems in medical services, as well as developing shared schemes Proportional liability between healthcare workers, healthcare facilities, and technology developers is crucial. Furthermore, strengthening professional standards, including medical digital literacy , mandatory clinical verification of AI output , and algorithm audit mechanisms, are essential for healthcare safety standards. Therefore, legal certainty regarding the use of AI in the healthcare sector can only be achieved through a systemic , adaptive, and risk-based regulatory approach. This reform is crucial not only to protect healthcare workers from disproportionate liability but also to ensure that AI utilization remains within the bounds of patient safety, professional ethics, and the principles of legal justice.

Acknowledgement

The author expresses his sincerity appreciation to all parties who have contributed to the research process and writing of this article . I express my deepest gratitude to my supervisors for their continuous guidance , motivation , and assistance from the beginning to the end of this work . Without their support , this research would not have been completed successfully . I also express my gratitude to my beloved family and friends who have provided constant encouragement , attention , and understanding throughout this process .