Enterprise Validation Management

In 2015, the U.S. FDA issues new guidance for Mobile Medical Applications. Given the widespread adoption and use of mobile technologies, new guidance was necessary to understand how the FDA looks at these devices and enforces them. While the guidance was prospective in looking at technologies considered to be medical devices, it also pointed out those which were NOT considered to be medical devices including educational tools, training mobile apps, or apps that are intended to provide electronic copies of medical text books and the like. I was happy to learn that my Apple Watch® was in the category of “Mobile apps that provide patients with simple tools to organize and track their health information” and that the FDA intends to exercise enforcement discretion over these devices. (US FDA Mobile Medical Applications – Guidance for Industry and Food and Drug Administration Staff, Page 16)

What drew my attention to mobile apps in a validation context were recent changes in major software vendor technologies. I attended a Microsoft Summit meeting in 2016 and I was facinated with some of the changes that Microsoft has added to their Dynamics 365 platform. Large software vendors such as Microsoft and Oracle have released mobile technologies integrated directly within their platforms. Take a look at the Azure platform in the figure below.

You will notice that Microsoft included a “Microsoft AppSource” store and Azure Internet of Things (IoT) to the solution. Mr. Nadella, CEO of Microsoft, shared his ambitious vision for the Microsoft platform which is to “reinvent productivity and business processes”. He believes that businesses of all sizes will not only use these technologies but will become digital entities able to engage their own customers, empower their own employees, optimize their own operations and transform their own products. As a validation practitioner, I strongly agree with this approach. Software SHOULD empower businesses to deliver value to their clients. What has been so elusive for many software companies is the interoperability of software applications to work together. He spoke of “silos” of information and unlocking data stored within these applications to drive new insights and change the way we work. This was my takeaway from Nadella’s talk. However, it leads us back to MOBILITY.

“There’s an App For That…”

One of the key things to note about today’s enterprise applications is that they can be integrated with Apps. Sometimes Apps are integrated into desktop applications and sometimes they have a mobile companion as well. The Apps in the AppSource store are developed by various partners – not necessarily Microsoft. These Apps can run seamlessly on tablets or other mobile devices. The questions from a validation perspective regarding mobile applications.

How do you conduct a supplier audit for multiple App vendors?
How do you control changes to Apps integrated in your applications?
How do you maintain the validated state with these Apps?

The answer to these questions are sometimes challenging. Supplier audit principles still ensure when you are discussing mobile applications. You still must be careful to select your suppliers carefully. However, it is impractical to audit every supplier in the AppSource store. In the case of Microsoft, these Apps become a part of the overall system thus a part of the Dynamics 365 application itself. Therefore, I would rely on rigorous testing to overcome the deficit in evaluating each and every App provider which may not be possible.

Regarding changes to your Apps, I employ the concept of Continuous Testing. This is a test strategy for cloud based service offerings where by you define your testing intervals at either quarterly or semi-annual for these types of technologies and you conduct frequent testing to help maintain the validated state.

Generally, there are two types of mobile testing: (1) Hardware Testing and (2) Mobile Application Testing. Whether you are using native, mobile web apps or hybrid apps, your strategy should take into account the risk associated with these applications and their impact on critical quality attributes.

VALIDATION AND VERIFICATION OF MOBILE APPLICATIONS

So how does one go about testing (validating) mobile applications in a regulated systems environment. The IQ/OQ/PQ testing paradigm still applies.

INSTALLATION QUALIFICATION (IQ) OF MOBILE DEVICES

The Installation Qualification (IQ) of mobile applications consists of:

Installation tests– Testing of mobile applications by installing /uninstalling it onto devices.
Services testing– Testing the services of the application online and offline.

The Operational Qualification (OQ) of mobile applications consists of:

Usability testing– To ensure that mobile applications are easy to use and deliver a satisfactory user experience to the customers
Compatibility testing– Testing of the application in different mobiles devices, browsers, screen sizes and operating system versions according in accordance with written user requirement specifications.
Interface testing– Testing of menu options, buttons, bookmarks, history, settings, and navigation flow of the application.
Low-level resource testing: Testing of memory usage, auto-deletion of temporary files, local database growing issues.
Mobile Operations Testing – Testing of backups and recovery plan if a battery goes down, or data loss while upgrading the application from a store.
Mobile Security Testing– Confirmation tests of mobile applications to confirm the protection of system data.

The Performance Qualification (PQ) of mobile applications consists of:

Performance testing– Testing the performance of the application by changing the connection from 2G, 3G to WIFI, sharing the documents, battery consumption, and other related tests.

It is important to understand that validating mobile applications is an essential part of validation and should be conducted based on risk and impact to regulatory requirements and GMP. You must always have a process around the acquisition, installation and support of mobile apps. Be aware of the changes that major software players are making and the impact on your systems environments. It is important to understand the impact of these changes on your validated systems environment. With the validation of mobile applications, what is old is new again. WE ARE BACK TO THE FUTURE WITH MOBILE APPLICATION VALIDATION. The principles of validation still endure in this new environment. You must keep up with the latest trends and respond accordingly. Watch this space for more information!

As a long-time validation engineer, I often ponder questions such as “what does it mean to achieve software quality and is it sustainable over time?” I ask myself these questions because in today’s systems environments, there are many factors that can impact software quality assurance.

Cyber threats are the elephant in the room. Most validation projects include IQ/OQ/PQ and UAT testing but do not address cyber threats at all. Can you really ensure that your validated environments are safe and secure without considering cybersecurity as part of your overall validation strategy? The International Software Testing Qualifications Board (ISTQB) defines software quality as “…The totality of functionality and features of a software product that bear on its ability to satisfy stated or implied needs…” Another definition is “…the degree of conformance to explicit or implicit requirements and expectations…” Finally, IEEE calls software quality “…The degree to which a system, component, or process meets specified requirements, customer, user needs or expectations…” As shown by the definitions above, software quality is somewhat subjective.

Data integrity is also a critical concern for validated systems. It is also a key imperative for software quality. Data integrity is a hot topic lately and generally refers to the accuracy and consistency of information stored in corporate databases, data warehouses or other such constructs. Data integrity ensures that information is accurate and reliable and in today’s environments, legally defensible. The accuracy and trustworthiness of data within your systems MUST NOT be in question.

Why is data integrity so important? Because companies make decisions routinely bases on information housed within corporate databases.

The lack of data integrity over the lifecycle of a system could cause adulterated product to get to the market, incorrect shipping of controlled materials/substances, and a wide variety of issues affecting the quality, safety and efficacy of a company’s products. Data integrity is not the purview of technology alone. To manage data integrity in the broadest sense requires people, processes and technology.

The ALCOA principle as highlighted in the figure below requires that data be attributable to the individual responsible for recording the data/activity. The “L” in ALCOA means that information must be clear and legible after it is recorded and permanent. The “C” in ALCOA means that the data must be recorded at the time it was generated. The “O” means data must be preserved in a unaltered state. The final “A” in ALCOA means that data must be accurate and reflect the action or observation made. Modifications must be explained if they are not self-explanatory.

No matter what the definition, software quality is all about providing assurance that a system is suitable for its intended use in some way. We confirm this through testing. However, it should be noted that testing alone cannot in and of itself ensure software quality. Testing merely provides a level of assurance or confidence in a software application under specific controlled conditions.

You cannot discuss software quality without a discussion on data integrity. To derive the true meaning of software quality it is important to consider the following key activities:

Establish SOPs That Provide Governance For Software Quality Assurance and Data Integrity
Document Everything (if its not documented, it didn’t happen)
Establish a Rigorous Software Change Management Process
Attain Level 5 Validation Processes Through Automation
Enforce Standards For Testing and Documentation
Identify Track and Manage Software Quality Metrics and KPIs
Conduct Positive and Negative Software Testing

The first step on your way to software quality and data integrity is to establish and follow procedures that provide governance over the process. You must have procedures that cover everything from validation to data integrity, automation, and everything in between. Secondly, you must document everything you do to ensure software quality and integrity. Third, you must establish a rigorous software change management process that helps track and manage all changes made to a cloud-based or on-premise system and who made the changes and why.

Forth, you must drive your organization to Level 5 validation processes. This is derived from the validation capability maturity model as illustrated in the figure below.

Level 5 validation means your processes are automated and optimized in a way to ensure quality and compliance. Fifth, you must enforce all standards for testing and documentation. This will also require Level 5 automation to achieve your objectives. Sixth, you must identify and track software quality metrics. You cannot achieve what you don’t measure. Peter Drucker often said “… you can’t manage what you can’t measure…” He also said “… what gets measured gets improved…” You must identify and track metrics to ensure you stay on track.

And finally, in all of your validation testing, conduct positive and negative testing against applications. The FDA states in the General Principles of Software Validation; Final Guidance For Industry and FDA Staff issued on Jan 11, 2002, that “… A good test case has a high probability of exposing an error; A successful test is one that finds an error…” This may be somewhat counter-intuitive but I am often stunned at how many validation test scripts are written so that they PASS rather than written to discover an error. A good software test will reveal errors if written correctly. When I interrogate applications, I often am looking to reveal problems that may arise during production.

It has been often said that software quality is no accident. It is the deliberate result of intelligent planning, hard work and rigorous execution.

Software quality is NOT error or bug-free software. It is about software that is of high quality and sufficiently meets the demands and expectations of the end user community. AUTOMATION IS KEY. Automated testing helps easily replicate tests, increases test coverage, reduces errors, improves consistency, and delivers automated traceability enabling more software defects to be discovered and addressed.

The issues surrounding software quality and data integrity are increasing across the globe. Your organization must be ready to deal with the challenges presented by these issues. WILL YOUR ORGANIZATION BE READY ?- Think about it.

Mobility and Validation: Back to the Future

The True Meaning of Software Quality