Meet-Up #25: How to Clinically Validate LLMs

Overview
Large Language Models (LLMs) are rapidly entering healthcare products, clinical workflows, and Software as a Medical Device (SaMD) solutions. While their potential is significant, demonstrating clinical performance, safety, and regulatory compliance remains a major challenge for manufacturers and regulators alike.
Questions continue to emerge around clinical validation methodologies, evidence generation, intended use, performance monitoring, transparency, and the application of existing regulatory frameworks to LLM-based medical devices.
This European SaMD Meetup session will focus on how organisations approach the clinical validation of LLMs, from regulatory strategy and market access considerations through to notified body expectations and practical implementation experiences.
The discussion is intended for professionals working across regulatory affairs, quality, software development, clinical affairs, AI governance, and digital health.

Agenda
Introduction (5–10 minutes) – Host (Ivo Flipse)

• Welcome and overview of LLM adoption within regulated healthcare.
• Key challenges around clinical validation and regulatory compliance.

Session 1 (10–15 minutes + Q&A) – TBC

• Regulatory frameworks applicable to LLM-based medical devices.
• Navigating evolving requirements across MDR, IVDR, AI Act, and related guidance.

Session 2 (10–15 minutes + Q&A) – TBC

• Bringing an LLM-based product to market as a medical device.
• Practical lessons learned from development, validation, and regulatory submission.

Session 3 (10–15 minutes + Q&A) – TBC

• Clinical validation from a notified body perspective.
• Common challenges, expectations, and evidence considerations during conformity assessment.

Panel Discussion (20–30 minutes)

• Validation approaches for LLM-based healthcare applications.
• Regulatory expectations, future developments, and open industry challenges.

Conclusion (5 minutes)
Wrap-up, discussion, and key takeaways.

Event Date & Time
July 21st – 7:30 PM – 9:30 PM CETleon