Agentic AI is changing healthcare, but its adoption is moving faster than the industry can handle safely. Moving too quickly or too slowly carries risks beyond financial losses, including delayed care, privacy violations, and a loss of trust between clinicians and the systems designed to support them.
The Five Layers of Agentic AI
Agentic AI does not function as a single system. It operates through five interconnected layers, each presenting unique risks:
- Power requirement: Cloud computing depends on a stable energy grid to keep AI agents running.
- Infrastructure/hardware: Data centers store the information AI agents need to function.
- Network layer: Digital protections must prevent private data from being exposed.
- Data layer: Rules control how data moves within the system.
- Application layer: The user interface where clinicians and patients interact with AI.
Weaknesses in any layer can affect the entire system. A secure network cannot fix flawed data, and a well-designed application cannot compensate for an unreliable power supply. Recognizing these dependencies is essential for safe deployment.
Where Responsible Use Breaks Down
Most conversations about AI safety focus on the network, data, and application layers, where users engage most directly. Yet even carefully designed systems can fail. A large language model might misread a patient’s pain description, for instance, recording “severe pain” in a clinical summary when the physician never used that phrase. Such mistakes distort medical records, raise legal risks, and reduce trust in the technology.
Safe AI in healthcare demands more than secure infrastructure. Three key principles must guide its use:
Related: Ari Robicsek named NCQA’s first chief medical officer
- Bias mitigation: AI trained on historical medical data can reinforce existing biases. Audits must consider age, race, gender, and other factors—not as a formality, but as a clinical necessity.
- Observability: Clinicians and administrators need clear insight into how AI agents operate and perform. Without this, there is no way to confirm the system stays within its intended limits.
- Explainability: AI decisions must be understandable. Third-party audits help, but organizations also need the ability to clarify a system’s purpose to stakeholders outside the core team.
These principles have real-world impact. A failure in any of them can lead to misdiagnoses or damage patient trust. The challenge is not just deploying AI but ensuring it works within the realities of clinical practice.
Many healthcare organizations feel pressured to adopt AI quickly. Those doing it well are not the fastest—they are the ones building systems clinicians can rely on. They integrate industry standards like NIST’s Risk Management Framework and OWASP’s security guidelines into their governance, rather than treating compliance as a formality. HIPAA remains essential, but responsible AI requires more than legal adherence.
Slowing Down to Move Forward
Health systems that rush AI deployment risk more than inefficiency. They risk losing the trust of clinicians who depend on these tools. The most effective adopters are not those with the most advanced models, but those who prioritize trust in both the technology and the people using it.
The healthcare data exchange setting is evolving, and groups like the Coalition for Health AI are developing best practices for agentic AI. However, frameworks alone are not enough. Organizations must adapt these standards to their workflows, patient populations, and risk profiles. The goal is not just to deploy AI but to make it a reliable tool that improves care without sacrificing safety.
Achieving this takes time. It requires audits, transparency, and a willingness to pause when needed. In an industry where speed often seems like the only priority, that may be the most difficult lesson to accept.
