Most engineering teams at companies that build medical devices learned threat modeling from the Playbook for Threat Modeling Medical Devices, published in 2021 by MITRE and the Medical Device Innovation Consortium with FDA funding. It is a good document and it is still the right place to start. It covers data flow diagrams, trust boundaries, STRIDE, attack trees, and how to connect the output to your quality system. None of that needs replacing.

What it does not cover is a model. Its main worked example is a home stroke monitor whose readings travel to a cloud service and are, as the Playbook puts it, "analyzed by an automated algorithm". The algorithm sits inside a larger box on the diagram. The method then applies STRIDE to four kinds of element: external entities, processes, data stores, and data flows. A trained model becomes a process. A training set becomes a data store. Both are technically correct, and both close down the analysis before it reaches anything useful.

That is not a fault in the Playbook. AI-enabled devices already existed in 2021, but the threats against them, the regulatory attention on them, and the techniques for attacking a model directly have all grown sharply since, driven by the wider AI boom. The Playbook was written before that shift and says openly that it is not prescriptive. The gap belongs to those of us still using it unchanged. If your device now contains a model, the element list has to grow before the method can find the threats that matter.

What the regulators actually say

Two documents drive this and they carry very different weight. Keeping that straight matters more here than in most topics.

The EU AI Act is binding law. Medical devices reach the high risk category through Article 6(1), which covers AI systems used as a safety component of a product, or which are a product themselves, where the product falls under the legislation listed in Annex I and has to undergo third party conformity assessment. The MDR and IVDR are in Annex I. Annex III covers a different set of use cases and is not the route medical devices take.

Article 15(5) is the text that does the work. High risk AI systems must be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities. The technical solutions must include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks on the training data set, attacks on pre-trained components used in training, inputs crafted to make the model get things wrong, confidentiality attacks, and model flaws. The Act names these as data poisoning, model poisoning, adversarial examples or model evasion.

Article 15(5) never actually mentioned threat modeling but sets a property the product has to have. The threat model is how you demonstrate you went looking for those vulnerabilities, and it is the obvious place to record it, but the Act leaves the method to you.

FDA sits in a different position. The draft guidance on AI-enabled device software functions from January 2025 is still a draft, and the FDA guidance database still labels it not for implementation with non-binding recommendations. Its cybersecurity content sits in Section XII, starting on page 34, and asks sponsors to submit any additional threat modeling elements where there are unique AI considerations.

The wording is narrower than it first appears. FDA does not tell you to do AI threat modeling. It assumes you already produce a threat model, and asks that your submission include additional elements in the cybersecurity risk management report, threat modeling and other deliverables wherever there are unique AI considerations. The recommendation is about what goes in the submission, and it is conditional on those unique considerations existing.

In practice that condition is easy to meet, because the same section lists seven AI risks that cyber threats can drive such as data poisoning, model inversion and stealing, model evasion, data leakage, overfitting, model bias, and performance drift. It then asks for malformed input testing and penetration testing at a minimum, and for a Security Use Case View covering the AI parts of the device.

Put the two together and the shape of the problem is clear. The binding text names the attacks but not the artifact. The draft guidance names the artifact but does not define its contents. Manufacturers are expected to fill the space in between.

Three kinds of device, three different threat sets

Article 15(5) applies to every high risk AI system and the FDA draft applies to every AI-enabled device software function. The attacks do not spread evenly across them. Before you pick a framework, decide which of these you have actually built, because using the wrong one fills your threat model with attacks that cannot happen and hides the ones that can.

1. A model that is trained once and shipped

Most AI/ML-based devices cleared by FDA to date use a locked algorithm, one that gives the same result each time for the same input. The model is fixed when the device ships and does not change until the next release.

The important thing about this group is that the worst attack happens before the product exists. Poisoning is a development time attack. Someone with access to the training data or the labelling process alters a small portion of it so the finished model behaves normally almost all of the time and fails in one specific way. OWASP's Top 10 for LLM Applications 2026 records a healthcare version of this, where minimal poisoning of medical training data significantly alters model outputs while passing standard evaluations. If your threat model only covers the shipped device, this attack is invisible to it, because by the time the device exists the attack has already succeeded.

An attacker who controls the input, whether by posing as the scanner or crafting the data itself, can make the model wrong without ever taking it offline, so every uptime check stays green while the diagnosis is silently incorrect.

For this group, add the parts your diagram is missing, the model, its training data, and the inference boundary, and name the attacks against them using MITRE ATLAS, the Adversarial Threat Landscape for Artificial-Intelligence Systems. ATLAS does for AI attacks what MITRE ATT&CK, Adversarial Tactics, Techniques, and Common Knowledge, does for conventional IT attacks. Once you have identified a threat, ATLAS gives it a standard name so a reviewer recognises it instead of interpreting your own description of it. ATLAS itself separates AI systems that classify or score data from AI systems that generate content such as text or images, and a locked imaging classifier belongs entirely to the classifying group.

2. A model that keeps changing after release

If you have a predetermined change control plan, the model is authorised to change in the field and the plan describes those changes in advance. Everything above still applies, and three more things become live attack surface: 1) the data that feeds retraining, 2) the trigger that decides when retraining happens, and 3) the channel that pushes new weights to deployed devices.

This is where the wording of Article 15(5) bites hardest, because it protects use, outputs and performance. Performance is allowed to change here by design, which gives an attacker somewhere to hide. A slow, deliberate shift inside the tolerance your plan permits looks like ordinary model improvement.

Article 15(4) is worth reading alongside it. Systems that keep learning after being placed on the market must be built to reduce the risk of biased outputs feeding back into future operations, and any such feedback loops must be addressed with mitigation measures. That is a design obligation, and it maps directly onto a threat for example, an attacker who can influence what the device sees in the field can influence what it learns from. Your threat model has to describe the versions the plan allows, not just the version you submitted.

3. A device feature built on a language model

Clinical documentation, summarisation, report drafting, patient-facing text. This is the group where the OWASP Top 10 for LLM Applications 2026 genuinely applies and where a lot of the current AI security material was written to fit. Prompt injection, unsafe handling of model output, and sensitive information disclosure all land here. Its poisoning entry, LLM05, is broader than the classic training-data case and covers pre-training, fine-tuning, embedding creation, retrieval augmentation and model distribution.

If the feature retrieves from a record system or a document store, that store is part of your attack surface. Poisoned content in the documents it retrieves from can surface as authoritative output without touching the model at all.

Check your threat model against what attackers are actually doing

A threat model is only as good as its last update. AI attack techniques move faster than most device teams' review cycles, so treat this as something you check periodically rather than something you finish once.

Start with the OWASP AI Exchange. It catalogues AI threats and their controls organised by what gets attacked, and its structure lines up with the three paths described earlier in this essay. Its development-time threats section covers the build path, poisoning during training, compromised labelling, supply chain risk in pretrained components. Its runtime threats section covers the use path, attacks against a model that's already deployed and answering. Read the section that matches the path you're missing on your diagram, not the whole document at once.

Once you've found a threat, name it with MITRE ATLAS. ATLAS gives each attack technique a standard identifier, the same way ATT&CK does for conventional IT attacks, and it separates techniques for systems that classify or score data from techniques for systems that generate content, matching the distinction this essay draws between a locked classifier and a language model feature. Naming a threat this way means a reviewer recognises it immediately instead of having to interpret your own description of it.

Then turn what you found into a requirement using the OWASP AI Security Verification Standard. Its training data requirements are specific enough to test against directly: an inventory of every data source with its origin, licence and collection method, controls on who can create or approve labels, poisoning detection built into the training pipeline, and dataset lineage recorded across every transformation. A threat model that names data poisoning as a risk and a verification requirement that asks whether your pipeline can detect it are two different documents, and a reviewer will ask for the second one. This is also where Article 15(5) stops being abstract. The Act says prevent and detect data poisoning. AISVS is what a control satisfying that actually looks like.

Then test it. FDA's draft guidance asks for malformed input testing and penetration testing at a minimum, aimed specifically at the AI risks it lists. For a locked classifier, that means feeding it corrupted, out of distribution, and deliberately adversarial images and checking what it does with them, not just checking that it runs. For a language model feature, it means testing with inputs designed to look like instructions rather than data, since that's what a prompt injection attempt actually looks like. A threat model that was never tested against real inputs is a document, not evidence.

Start with what you already have

If your last signed-off threat model covered a device with a model in it, pull it out and look for four things: the training data, the model weights file, the retraining trigger, and the update channel. If none of them is on the diagram, you have not found a new problem. You have found the reason your existing process never reported one.

Then decide which of the three kinds of device you have, before anyone picks a framework. That single decision will save you more review time than any tool.

References

• Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 6(1), 15(4), 15(5), Recital 76. http://data.europa.eu/eli/reg/2024/1689/oj

• FDA, Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, draft guidance, January 2025, docket FDA-2024-D-4488. Section XII Cybersecurity, pages 34 to 36. Seven AI risks, page 35. Submission content including threat modeling, page 35. Draft, non-binding.

• MITRE and MDIC, Playbook for Threat Modeling Medical Devices, November 2021. STRIDE element types and the worked example, chapter 2.

• IMDRF/AIML WG/N88 FINAL:2025, Good Machine Learning Practice for Medical Device Development: Guiding Principles, Principle 2.

• OWASP, Artificial Intelligence Security Verification Standard (AISVS) 1.0, June 2026, chapter C1 Training Data Integrity and Traceability, and chapter C12.

• OWASP GenAI Security Project, Top 10 for LLM Applications 2026, LLM05 Data and Model Poisoning.

• MITRE ATLAS, adversarial technique knowledge base for AI systems. https://atlas.mitre.org/