[throwawaaarrgh]

I've worked on medical devices. As a techie, what surprised me more is not the lack of trials for new devices. It's that for software (and other kinds of components), it's often acceptable to have black box testing to certify a component as functional.

This means basically saying "I pressed a button and the thing did X". That's the test. No need to understand how that works, or provide any more technical documentation or specs, or record system state or variables, or do anything else. Press button, it does X. That's good enough to be certified.

The reason for this is simple enough: sometimes the thing you're using is proprietary and its manufacturer/vendor simply won't give you anything else to certify it. But this black box testing is even used for open source software and components. It's like a short cut you can use to cut through a lot of the testing you could have done. I imagine this will remain standard practice as the "AI" companies push their hallucinating dreck and unexplainable magic into the medical device space.

[neandrake]

I work with software medical devices. Another aspect here aside from 510k is that it is required, or effectively required, to comply with something like IEC-62304 (developing software as a medical device), and ISO-13485 (quality management systems for medical devices).

For the scenario you describe the piece that’s missing is risk analysis, a requirement. In preparation to release to market they must evaluate the probability and severity of the button not doing X or doing X incorrectly, and develop mitigations if the risk is unacceptable. What you ask - documentation and specs - exist at some level, but the manufacturer has to define what level is necessary for them. I could see an argument against the manufacturer deciding this for themselves, though it’s likely impractical to do so.

For software medical devices that have hundreds of transitive dependencies it’s not feasible to go at the level you’re describing. Some management of dependencies is necessary but treating as a black box - with quality/test management and risk analysis of the black box - is what the current system defines as a reasonable trade-off. Again I could see arguments for changing this, though for many manufacturers the EU has instituted stricter regulatory in the past ~5 years which has been a bit painful but overall probably a good thing.

Today one of the aspects of medical device development which is under tighter scrutiny is cybersecurity. It’s pretty painful right now. Previously there was not much related to cybersecurity required - obviously not ideal - but the pendulum has swung to the other end of the spectrum making it a significant burden. We’ll see, most of it is adopting new processes which is always painful and slows down progress at first. After the initial hump it should be eased into, and ultimately better for patient care and medical institutions in the long run.

[iancmceachern]

Agree with you here.

Wanted to echo " It's (cybersecurity) pretty painful right now".

The FDA just implemented new requirements. Basically they require penetration testing on all new medical devices. The issue is they don't have the expertise in house to know the technical details, and they haven't defined the tests, etc. Additionally they're isn't yet an ecosystem of partners and service providers yet to provide and compete in providing those penetrstion testing services.

Pragmatically what it means for folks trying to get a device cleared at the current moment:

You need to send your device to the one and only penetration testing house that does this for the FDA now and let them try to physically hack into your medical device. You have to make it impossible and evident if someone tampers with it in any way. This is in addition to all the software security stuff we need.

Imagine of you were making computer monitors. One day you are suddenly required to make it so a technical expert cannot open the monitor up using specialized tools.

[jtwaleson]

It depends on the risk class according to IEC 62304. When you're developing Class C software you'll need acceptance criteria that goes into how you initialize variables etc.