Ensuring Quality in Medical Devices: The Role of Software QA in the Healthcare Industry
Scroll ↓
In some of my previous blog posts, I have mentioned that my background lies in the medical device industry. My educational background is in biomedical engineering, but currently, I am employed at a company that focuses on implementing IT solutions in various sectors like finance, education, and automobiles. However, the company has also ventured into the digital health domain, which continues to be the most captivating field for me. Not only is it complex and intriguing, but it is also crucial for the future. The healthcare sector has numerous applications and areas where it can provide innovative digital solutions that are future-oriented. In this blog post, I aim to share my passion for this industry by discussing my experiences as a Software QA Engineer for medical devices and highlighting the essential terms to be familiar with. It’s important to note that my insights are based on my personal knowledge and experiences, and different companies may have their own unique solutions. However, this blog post will present a comprehensive summary.
Different Classes of Medical Devices
Currently, there exists a vast range of over 500,000 distinct medical devices. These devices encompass a spectrum, ranging from basic tools like glasses to intricate ones such as pacemakers. To categorize and organize them, medical devices are classified into various risk classes. It is essential to note that the higher the risk class, the greater the potential risk for patients. It is crucial to remember that when working with these devices, you are not only interacting with users but also with patients, and high-risk situations could potentially result in fatal outcomes. The classification system comprises four classes:
Class I (Low Risk): These devices are considered to have a low risk to the patient and/or user. Examples might include stethoscopes, bandages, or manual surgical instruments. These devices are subject to the least regulatory control.
Class IIa (Low-Medium Risk): These devices are considered to have a low-to-medium risk. Examples could include hearing aids, dental fillings, or ultrasound devices. They require more stringent regulatory control than Class I devices.
Class IIb (Medium-High Risk): These are medium-to-high risk devices. Examples could include surgical lasers, defibrillators, or infusion pumps. Class IIb devices are subject to even more stringent regulatory controls.
Class III (High Risk): These are the highest risk devices and are subject to the most stringent regulatory control. Examples include heart valves, implantable defibrillators, or devices that are totally or partially introduced into the human body and expected to remain there for at least 30 days.
Verification vs. Validation
In the medical device industry, you will frequently encounter the terms “verification” and “validation,” which are often used interchangeably, although they have different meanings. These two processes are closely connected and integral to the development and production of medical devices. Both verification and validation play essential roles in quality management systems, ensuring the safety, effectiveness, and adherence to the intended use of a device. Furthermore, Software QA engineers are often called verification and validation engineers in this industry.
Here’s a breakdown of the two:
Verification: This process is about making sure that the device has been designed correctly. It involves checking that the product meets specified requirements at various stages of the product development. In other words, verification answers the question: “Are we building the product right?” It is about ensuring that each step in the design and development process is being done correctly and that the output of each step meets the requirements set for that step. Verification activities may include inspections, reviews, walk-throughs, and other types of analyses.
Validation: This process is about making sure that the correct device has been designed — that is, the device will meet the needs of the user and the intended use of the device in the real world. Validation answers the question: “Are we building the right product?” It involves activities to ensure that the device as a whole (as opposed to individual components or stages of design) meets the user’s needs and intended uses. Validation activities may include clinical evaluations, usability testing, and performance testing under simulated or actual use conditions.
The primary distinction lies in the fact that verification focuses on confirming that the design outputs align with the design inputs, whereas validation centers around ensuring that the end product fulfills user requirements and intended purposes. Both processes are crucial in guaranteeing the safety, effectiveness, and compliance of a medical device with regulatory standards. For instance, in the case of a defibrillator, validation would ensure that it possesses a voltage level sufficient to restore a normal heart rhythm by delivering an electric shock. Additionally, verification entails verifying product specifications, such as the device’s appearance and any additional functionalities it may have.
The V-Modell
At my previous company, we utilized a framework known as the V-Model. The V-Model serves as a visual depiction of the software development and testing lifecycle. Its name is derived from its flowchart-like structure, resembling the letter “V.” The left side of the V represents the breakdown of requirements, while the right side represents the integration of components and their subsequent testing.
The steps on the left side of the V represent the specification of requirements and the design process:
User Requirements: This is where the needs of the user are identified and documented. It’s important to understand what the user needs from the system.
System Requirements: This is where the overall system requirements are established that will meet the user’s needs.
High-Level Design: This is where the overall architecture of the system is designed. This stage involves determining the main components of the system and how they interact with each other.
Detailed Design: This stage involves the detailed design of each component of the system.
On the right side of the V, the testing takes place:
Unit Testing: This is where each component is built and tested individually to make sure it works as expected.
Integration Testing: This is where the individual components are brought together and tested as a whole to make sure they work together correctly.
System Testing: This is where the entire system is tested to ensure it meets the specified system requirements.
Acceptance Testing: This is where the system is tested to ensure it meets the needs of the user. This could involve beta testing with a small group of users or other forms of user acceptance testing.
The ‘V’ shape shows the relationship between each stage of the development lifecycle and its associated phase of testing. For example, the Detailed Design phase is directly linked to the Unit Testing phase, and so on.
In the context of medical device software, the V-Model is particularly useful because it aligns well with regulatory requirements. Regulatory bodies like the Food and Drug Administration (FDA) and the European Medicines Agency (EMA) require comprehensive documentation demonstrating that medical device software has been thoroughly specified, designed, implemented, and tested. The structured, systematic, and rigorous approach of the V-Model can help achieve this and ensure the safety and effectiveness of the software.
Traceability and Test Coverage
Traceability and test coverage are fundamental aspects of quality assurance (QA) in the development of medical devices. Traceability refers to the ability to track a requirement through its life cycle, from its origin, through its development and specification, to its subsequent deployment and use, and through periods of ongoing refinement and iteration. This is often accomplished through the use of traceability matrices, which map relationships between requirements, design elements, and test cases, thereby ensuring that every requirement is adequately tested. On the other hand, test coverage measures the extent to which the software has been tested. It provides what proportion of the codebase or functionality has been checked under test conditions. In the context of medical devices, where safety and reliability are paramount, high test coverage is essential to ensure that all parts of the device work correctly under all possible conditions. Together, traceability and test coverage help ensure that all requirements have been implemented correctly, all parts of the device have been thoroughly tested, and any changes or issues can be effectively managed and documented, thereby contributing to the safety and effectiveness of the final product.
Bug Reporting
In my previous company, we had a strict policy of reporting every single bug we encountered, regardless of whether it was during user story verification, test plan execution, or exploratory testing. The aim was to have a comprehensive record of all known errors within the system, enabling us to identify areas of improvement. In dedicated meetings, we would assess the risk associated with each bug and determine if and when it should be addressed. This approach provided a comprehensive overview of the product’s areas that required attention, highlighting where the majority of bugs were concentrated. Moreover, we maintained a bug report or known bug list that could be shared with stakeholders to keep them informed about the current project status. Regulatory authorities closely monitored these bug reports and the manufacturer’s responses to ensure ongoing patient safety and compliance with regulatory requirements.
Risk Assessment
Risk assessment plays a vital role in the field of medical devices as it enables manufacturers to identify, assess, and address potential hazards associated with device usage. As mentioned earlier, each software bug underwent thorough discussion and classification based on its risk level during dedicated meetings. In these meetings, the severity and likelihood of each identified bug were determined. This involved evaluating the probability of bug occurrence and its potential impact on users or patients. By considering both these aspects, a risk profile could be established for each potential hazard, aiding in the prioritization of risk mitigation strategies. For instance, risks characterized by both high likelihood and high severity would typically receive immediate attention and priority for resolution (emergency or high). Adopting this risk-based approach ensures the utmost safety and effectiveness of medical devices, while also adhering to regulatory requirements in various jurisdictions.
Documentation
In this industry, documentation plays a significant role, and if you intend to work in this field, you should be prepared for extensive documentation. Documentation serves multiple purposes, including serving as proof of regulatory compliance, maintaining a traceable record of activities, and ensuring transparency and consistency throughout the development process. While there are numerous types of documents involved, I won’t provide an exhaustive list. However, to give you an example, a Software QA professional may be expected to provide a Design Verification and Validation Plan for each software release with all the necessary points:
Design Verification and Validation Plan
Objective
Scope
Regulatory Requirements
Verification Plan
Validation Plan
Traceability Matrix
Test Coverage
Test Reports
Risk Management
Resources and Responsibilities
Schedule and Milestones
Change Control
Review and Approval
Basically, you gotta write down everything you do. It’s like being in a never-ending paperwork party where you’re the guest of honor. This meticulous documentation is absolutely crucial. It’s your golden ticket to show those regulatory bodies that you’re playing by the rules and helps navigate through any sticky situations that might come your way. So, grab your pen and get ready to document like a champ!
Personal Experience
You might be thinking, “Wow, working in such a regulated environment must be dreadful. One mistake and someone’s life could be on the line.” Well, let me share some insights from my experience working in this industry.
First and foremost, if you’re working in a reputable company, you can expect a well-defined QA process that will be scrutinized by an auditor at some stage. In order to manufacture and sell medical devices, your company must have an established quality system in place. This entails correctly defining numerous processes, which will undergo external evaluations. As a result, there’s a significant emphasis on static testing, such as reviews, within these industries. This means you’re never left alone with your decisions; someone else has reviewed and possibly approved them. While the documentation aspect may be a tad frustrating for some individuals, it proves to be highly beneficial, especially when starting. You have access to all the information you need, allowing you to track your past actions and even identify any mistakes you may have made along the way.
Knowing that I had to work with precision and could be as meticulous as I desired always served as an extra source of motivation for me. Sometimes, as a QA, you may feel the urge to delve deeper into a particular topic, but due to time constraints or someone dismissing its importance, you may have to hold back. However, in the highly regulated medical device industry, I never encountered such limitations. People genuinely strive to ensure everything functions correctly, and if you express your desire to test something with greater precision to the stakeholders, they usually understand and support you. Moreover, I was incredibly lucky to be surrounded by amazing coworkers and a supportive team leader who not only imparted valuable knowledge about the processes but was also consistently there to address any queries I had.
Further Information
If you are interested in further information about this topic here are some books I would recommend:
Medical Device Software: Verification, Validation, and Compliance” by David A Vogel
“Introduction to Medical Software: Foundations for Digital Health, Devices, and Diagnostics” by Xenophon Papademetris