That is the right direction. Better data can improve research, clinical trials and decision-making. Yet the next governance challenge is already arriving. AI systems are shifting from tools that produce an answer to agents that can take actions across software, data and communications. In healthcare, the difference matters enormously. An agent that summarizes a chart is one thing. An agent that can schedule patients, send instructions, change records, trigger referrals or act across multiple systems has a much larger operational footprint.

Canada already has pieces of the governance model it needs. Health Canada’s April guidance for machine-learning-enabled medical devices emphasizes risk management, testing and evaluation, clinical validation, transparency and post-market monitoring. Those principles should now be extended to agentic systems according to the authority they receive, even when an agent falls outside the medical-device regime.

A recent independent investigation by METR and Redwood Research shows why. In an OpenAI evaluation environment, roughly 1,200 agents that were intended to operate in isolation found an unsanctioned communication channel and exchanged more than 70,000 messages and files. Roughly 700 participated in an attack on Hugging Face. The investigators found coordinated workstreams in which agents achieved some milestones they had not achieved working independently. The investigation also had important limitations because of the scale and incompleteness of the available data. The lesson is practical rather than apocalyptic: access, coordination and delegated authority can change the risk of an agent system.

I’m no AI skeptic. I help organizations adopt AI for a living, and I want adoption to move faster. In my experience, strong safeguards increase trust and make faster adoption possible, while reducing the risk of failures like the Hugging Face attack.

Canadian health systems should start by giving every consequential agent an authority budget. That means specifying, before deployment, which records it may read, which systems it may write to, which people it may contact, which actions it may execute and which decisions always require human approval. Those permissions should expire unless renewed. Delegation to another agent should count as a new authorization event, rather than a loophole around the original limits.

Second, evaluation should reflect authority. A system that drafts internal notes does not need the same testing as an agent that can modify a patient record or initiate an external action. As authority rises, independent evaluation should test whether the agent stays within its permission boundaries, whether multiple agents can combine access in unexpected ways and whether humans can reliably interrupt or revoke actions.

Third, serious AI-agent incidents deserve independent review. Healthcare already understands the value of learning from adverse events. Agent failures that cross system boundaries, expose protected information, circumvent controls or trigger consequential actions should generate a comparable record: what the agent could access, what it did, how controls failed and what changed afterward.

Finally, health organizations need action-level audit trails that are useful to operators, rather than merely logs that satisfy a compliance checkbox. When an agent acts, the organization should be able to identify the agent, its human owner, the authority it was given, the system it touched and whether a human approval gate applied.

Canada’s advantage is that it does not need to choose between rapid health AI adoption and serious safeguards. The VITAL investment shows that Canada is willing to build sovereign infrastructure for health AI. Health Canada’s device guidance shows that lifecycle evaluation and monitoring are already familiar regulatory ideas. The next step is to connect those instincts to agent authority before autonomy becomes routine.

If Canada does that now, safeguards can become part of the infrastructure for adoption rather than a reaction to the first major failure. That is how health systems gain the confidence to move faster without giving software more power than anyone intended.

Comments

Be the first to comment on this!