Generative AI in Medical Devices: 7 Questions the FDA Is Asking in 2026

Generative AI in Medical Devices: Explore 7 key questions shaping FDA regulatory expectations in 2026, from AI model validation and patient safety to human oversight and continuous monitoring.

Key takeaways

• In August 2026, the FDA published a discussion paper on regulating generative AI medical devices and asked the industry for feedback.

• The paper is not guidance. It does not set new requirements, but it shows how the FDA is thinking.

• Seven themes run through it: risk, users, premarket proof, clinical evidence, conversations, postmarket monitoring and change control.

• Teams that start answering these questions during design will be better prepared than teams that wait for final rules.

Generative AI medical devices are moving into healthcare faster than most regulatory frameworks can follow. Tools that draft clinical notes, answer patient questions or help clinicians interpret data are already being built into software and devices.

In August 2026, the FDA’s Digital Health Center of Excellence published a discussion paper titled Considerations for the Regulation of Generative AI-Enabled Medical Devices and invited public feedback through October 19, 2026. This matters for digital health founders, product and regulatory teams and healthcare leaders who are building or buying these products.

One caution first. The FDA states that the paper is for discussion only and does not represent draft or final guidance. It does not communicate proposed or final regulatory expectations. Nothing below is a requirement. These are the questions the FDA is asking, and they are worth answering early.

Why generative AI medical devices need a different approach

Traditional AI devices usually produce a fixed type of output, such as a score or a flag on an image. The team can define the possible outputs, test them and set acceptance criteria.

Generative AI works differently. It can produce open-ended text, images or conversations, and the range of possible inputs is very large. Three problems follow from this:

• Outputs can look credible and still be wrong. A fluent answer is not the same as a correct answer, and users may trust it too much.

• Testing cannot cover everything. You cannot list every question a user might ask, so a normal test plan with fixed cases is not enough on its own.

• The intended use is harder to hold. A user can steer a generative system outside the task it was built for.

On top of this, many products are built on a third-party foundation model, which can change without the device maker deciding it. The FDA paper is an early attempt to deal with these problems. It is also separate from the FDA’s earlier work on AI-enabled devices, including its final guidance on predetermined change control plans (December 2024) and its January 2025 draft guidance on lifecycle management and marketing submissions for AI-enabled device software functions. For a wider view of AI in devices, see 1,600+ AI-Enabled Medical Devices: 7 Questions to Ask Before Adding AI to Your Product.

What the FDA paper covers

The paper is organised around four areas. Each one contains questions for the industry:

1. Risk assessment: how to decide how much scrutiny a generative AI function needs.

2. Premarket evaluation: how to show the device works before it reaches patients.

3. Postmarket monitoring: how to keep it safe and effective after launch.

4. Other topics: including foundation models and agentic AI, which are systems that plan and act across several steps.

The seven questions below follow these areas.

Question 1: What does the function do, and what happens if the output is wrong?

What the FDA is exploring: A two-axis risk framework. One axis is activity, meaning how independently the function directs or takes action, from sharing information to acting on its own. The other axis is consequences, meaning how severe the harm could be if someone relied on an incorrect output.

Why it matters: A tool that drafts administrative notes carries a very different risk from a tool that supports a diagnosis or treatment choice. The paper also separates functions that direct an action from those that take the action themselves. Your place on these two axes is likely to shape how much evidence you need.

Illustrative risk assessment showing how the potential harm of generative AI in medical devices increases with greater autonomy and the consequences of incorrect outputs, based on the FDA’s 2026 discussion paper.

A simple example: Think of two products built on the same language model. One summarises appointment notes for hospital staff. The other tells patients whether to seek urgent care. The model may be the same, but the risk of a wrong output is completely different.

What to prepare:

• Write a clear description of what the function does and how much autonomy it has.

• List the worst realistic outcome of a wrong output.

• Do this before design decisions are locked, because it affects scope, interface and testing.

Common mistake: Describing the product by its technology (“an AI assistant”) instead of by its function and its possible harm.

Question 2: Who is using it, a patient, a generalist clinician or a specialist?

What the FDA is exploring:Whether risk changes with the user. The paper discusses patient-facing versus clinician-facing functions, and generalist versus specialist clinicians.

Why it matters: A specialist can spot a wrong answer in their own field. A patient or a generalist may not. The same output can be safe for one user and risky for another.

What to prepare:

• Define your intended user precisely, including their training and setting.

• Test with that user, not only with your engineers.

• Check whether the wording, warnings and level of detail suit that user.

This links directly to usability, which is covered in Lessons in Human-Centered Design: Why the Best Healthcare Products Feel Invisible.

Common mistake: Building for clinicians and then allowing patients to use the same feature without any change.

Question 3: How will you prove performance when outputs are open-ended?

What the FDA is exploring: A “competency-based” approach. It combines non-clinical benchmarking of the final user-facing device with clinical confirmation. The benchmarking groups in the paper are:

• Safety: safety-critical escalation, staying within scope and communicating uncertainty

• Clinical proficiency: clinical knowledge, information gathering, quantitative analysis and communication

• Generalizability: robustness, reproducibility and performance across subgroups

• Agentic capabilities: only for systems that plan and act across steps

Why it matters: You cannot test every possible question. The FDA is asking how a team can still show, with a defined method, that the device is competent at its intended job. The paper also notes that the evaluation is of the final user-facing device, not only the underlying model. It also raises the risk that publicly available benchmarks may be affected by data contamination, meaning the model may have already seen the test material.

What to prepare:

• Decide your test method, scoring criteria and acceptance thresholds in advance.

• Build test sets that reflect your real users, settings and patient groups.

• Test the whole product, including the interface, safeguards and workflow, not only the model.

  
  

Common mistake: Reporting a strong score on a general public benchmark and treating it as proof for your specific device.

Question 4: What real-world evidence shows it works in use?

What the FDA is exploring: Clinical confirmation options, listed in rising order of rigor and patient exposure:

Option	What it means
Retrospective evaluation	Testing on real past inputs
Shadow deployment	Running the device alongside real care without affecting decisions
Standardized patient interactions	Scripted interactions to test behaviour
Clinician adjudication	Clinicians reviewing real cases and outputs
Prospective clinical study	A planned study, possibly randomised

The paper does not say every device needs a prospective study. It asks how a sponsor should choose and justify the right approach for a given device.

Why it matters: The right level of evidence should match the risk of the function. A low-risk administrative tool and a treatment-guiding tool should not face the same plan.

What to prepare:

• Match your evidence plan to your risk level, and keep the reasoning documented.

• Consider how you will measure performance with statistical meaning when outputs are open-ended.

• Think about who could review outputs independently, such as clinicians outside your team.

Investors also look for this kind of readiness, as explained in What Investors Look For Beyond the Prototype.

Common mistake: Leaving the evidence plan until after the product is built, when the data you need can no longer be collected.

Question 5: What happens in a long conversation, and does the device escalate care correctly?

What the FDA is exploring: Conversational devices can drift from giving information to giving directions across several turns. The paper also highlights two failures: under-escalation (not recommending needed care) and over-escalation (sending people to emergency care without need).

Why it matters: A single test question does not show how a device behaves over a full conversation. A product that starts by sharing general information may slowly move towards advice that acts like a recommendation.

What to prepare:

• Test realistic multi-turn scenarios, not only single questions.

• Define when the device must stop, hand over to a clinician or advise urgent care.

• Test both missed escalation and unnecessary escalation, since both cause harm.

Common mistake: Testing only the first answer and assuming the rest of the conversation will stay within scope.

Question 6: How will you monitor performance after launch?

What the FDA is exploring: Risk-based postmarket options:

• Periodic re-benchmarking of the device

• Periodic sample-based review by clinicians

• Monitoring for performance degradation

The paper also asks whether premarket benchmarking can serve as a baseline for later re-evaluation. It raises the idea that stronger postmarket monitoring might justify accepting more uncertainty before launch. That is only a question at this stage and not an established policy.

Why it matters: Generative systems and their environments change. A device that performs well at launch can behave differently later, because the model, the data or the way users interact with it has changed.

What to prepare:

• Build monitoring into the product from the start, with data capture, clinician review and clear triggers for action.

• Decide who owns the monitoring, including after the team changes.

• Keep your launch benchmark so you can compare future performance against it.

Common mistake: Treating monitoring as a later phase, which usually means the product lacks the data it needs.

Question 7: How will you control changes, especially to a third-party foundation model?

What the FDA is exploring: How change-control ideas such as a Predetermined Change Control Plan (PCCP) might be adapted for generative AI, especially when a third-party model provider changes the model underneath your product. The paper also floats voluntary Foundation Model Device Master Files, which model developers could use to share confidential information with the FDA. These would not authorise the model for any use.

Why it matters: If your product depends on someone else’s model, an update outside your control can change your device’s behaviour. You are still responsible for the final product.

What to prepare:

• Put update notifications and re-testing triggers into your supplier agreements.

• Define which changes need re-benchmarking and which do not.

• For AI-enabled devices in general, the FDA finalised its PCCP guidance in December 2024, which is a useful starting point for your own change plan.

Common mistake: Assuming that a supplier’s model update is a routine upgrade, without checking its effect on your device.

Seven essential cybersecurity decisions to consider during the medical device design stage to strengthen patient safety, protect sensitive data, manage software vulnerabilities and support long-term regulatory compliance.

The seven questions at a glance

Question	FDA topic	What to prepare
What does it do, and what if it is wrong?	Risk: activity and consequences	Clear function and harm analysis
Who is the user?	Risk: patient, generalist, specialist	Defined users and user testing
How is performance proven?	Premarket benchmarking	Predefined tests and thresholds
What real-world evidence exists?	Clinical confirmation	Evidence plan matched to risk
How does it behave in conversation?	Multi-turn and escalation	Scenario tests and handover rules
How is it monitored after launch?	Postmarket monitoring	Monitoring and review process
How are changes controlled?	Change control, foundation models	Supplier terms, PCCP-style plan

What is still undecided

It helps to be clear about what the paper does not do:

• It does not create new requirements or change current FDA policy.

• It does not say whether new legal authority is needed.

• It asks questions and invites feedback, so the final approach may look different.

For now, teams still follow the FDA’s existing device requirements and its current AI-related guidance. The discussion paper is best used as a planning aid that shows where regulators may look closely.

What product teams should do now

1. Write your intended use in plain words. Say what the device does, who uses it and what it must not do.

   
   

2. Map your risk. Place the function on activity and consequence, and record the reasoning.

   
   

3. Design the safeguards into the product. Scope limits, uncertainty messages and escalation rules belong in the design, not in a manual.

   
   

4. Plan your evidence early. Decide how you will test and what real-world confirmation you will gather.

   
   

5. Plan monitoring and change control. Include supplier changes.

   
   

6. Do not forget security and approvals in other markets. Connected AI products also face cyber risk, which we cover in Medical Device Cybersecurity. If you plan to sell in India as well, read CDSCO Approval for Medical Devices in India.

   
   

Planning a Medical Device With AI Inside?

Building a generative AI medical device means making decisions about risk, testing, safeguards and change control long before the first prototype is ready. Getting these right at the design stage saves time, cost and rework later.

At iNSPIRE Design, we help MedTech teams take a product from idea to manufacturing-ready design, with usability, engineering and regulatory thinking built in from the start. Whether you are shaping an early concept or preparing for your next development stage, our team can help you plan the path.

Reach out to us – Inspire Design for any assistance

Final Thoughts: Treat Generative AI Medical Devices as a Design Problem First

The FDA’s discussion paper does not give final answers, and the rules may change after feedback. But the direction is clear. The agency wants to know what your device does, who it serves, how you prove it works, how you watch it after launch and how you control change.

Teams that can answer these seven questions early are in a stronger position, with regulators, investors and hospitals. Many of the answers come from design choices such as scope, interface, safeguards and testing, so they are far easier to settle at the start. If you are planning a product like this, see Why Great Healthcare Ideas Never Become Great Products, and talk to our team about NPD services at iNSPIRE Design.

FAQs

1. What is the FDA discussion paper on generative AI medical devices? 

It is a paper published by the FDA’s Digital Health Center of Excellence in August 2026, titled “Considerations for the Regulation of Generative AI-Enabled Medical Devices.” It asks for public feedback on risk assessment, premarket evaluation and postmarket monitoring.

2. Is the discussion paper a new FDA rule?

No. The FDA states that it is for discussion only and does not represent draft or final guidance or proposed regulatory expectations.

3. What is a Predetermined Change Control Plan (PCCP)?

A PCCP describes in advance the changes a manufacturer plans to make to an AI-enabled device and how they will be tested. The FDA finalised guidance on PCCPs for AI-enabled device software in December 2024.

4. Why is testing generative AI harder than testing traditional AI?

Generative AI can produce open-ended outputs for a very wide range of inputs, so it is not practical to test every case. The FDA paper asks how to show competent performance in that situation.

5. What is agentic AI in healthcare devices? 

The paper describes agentic AI as systems that plan and act across multiple steps. Some of these systems may fall within device oversight and may need extra considerations.

6. Does this apply to Indian companies?

Any company that wants to sell in the US must meet FDA expectations, wherever it is based. Indian companies should also check the requirements of CDSCO for the Indian market.

About the Author

Kunal Bijlani is the Co-founder of Inspire Design, a product design and engineering company working with clinicians, startups and healthcare companies on medical device design and development.

Through Inspire Design's work across 60+ healthcare product development projects, Kunal has worked with teams moving from early clinical ideas through product design, engineering, prototyping and Design for Manufacturing