In Case You Missed It: CSV Validation Master Class

In case you missed it, in December 11- 13 KenX conducted the Computer System Validation and Data Integrity Congress.  Recently, the FDA has issued warning letters regarding non-compliance of validated computer systems.  Findings have included issues such as inadequate risk analysis, non-independent audit trails (audit trails that could be manipulated or turned on/off), failure to established SOPs that provide governance for the validation process and other such issues.  The event featured a guest speaker from the FDA who highlighted challenges associated with data integrity and the approach taken by the Agency.

Recently, I have been speaking to the regulatory community about the many challenges we face in validating today’s computer systems.  Cybersecurity, mobility, cloud applications at the enterprise level, the Internet of Things (IoT) and many other changes affecting the installation, qualification, deployment and function of computer systems have compelled me to rethink strategies around computer systems validation.  As a 30-year practitioner in the field, I have developed key best practices to support cloud validation, address cybersecurity and the other challenges associated with today’s technology.

In case you missed it, I conducted one of the first Computer Systems Validation Master Classes.  The Master Class presented a broad range of topics related to the Independent Validation and Verification of today’s technologies.  We addressed topics such as:

  • Lean Validation Principles, Best Practices and Lessons Learned
  • Computer Systems Validation Automation and Best Practicess
  • Cybersecurity & Computer Systems Validation: What You Should Know
  • Cybersecurity Qualification: The Next Frontier in Validation Testing
  • Cloud Validation Best Practices
  • Continuous Testing In The Cloud
  • Leveraging Service Organization Control (SOC) Reports For Supplier Audits
  • … and much more

The 90-minute session was a lively discussion of many topics for validation contemporaries that will help them master validation of the latest technologies and ensure sustained quality and compliance.

Our Master Class format encouraged knowledge-exchange, where each topic was not only debated from the practitioners’ perspective, but participants delivered insights from their experiences presenting the latest best practices, regulatory guidance and practical CSV scenarios resulting in a comprehensive discussion of each topic as well as practical tips, tools and techniques to ensure software quality and a more relevant validation process which takes into account today’s technologies and their profound impact on the validation process writ large.

For participation in the Validation Master Class workshop, I offered participants a copy of my lean validation process templates, a cybersecurity qualification (CyQ) template, a cloud validation SOP, cybersecurity validation SOP, a system risk assessment template and sample SOC 1/SOC2/SOC3 data center reports for cloud providers.  (if you would like to obtain a copy of these materials please contact me using the contact form provided)

In case you missed it, I can report that the event was a huge success as measured by the feedback from the session and the response of all participants.  Check out our events and join us at one of our weekly webinars or industry events!

Understanding Lean Validation: A Practical Approach

Lean manufacturing has been with us since 1988.   Lean principles were derived from the Japanese manufacturing industry.  The “Lean” process was originally created and adopted by Toyota designed to eliminate waste and inefficiency in its manufacturing operations.  Lean processes led to the Toyota Production System (TPS) which is arguably one of the greatest manufacturing success stories of all time.  The focus of lean was the elimination of waste and inefficiencies throughout the manufacturing process.

To identify and eliminate waste from the production process, Toyota believed it was important to understand exactly the nature of waste and where it existed. While Toyota’s products significantly differed between factories, the typical wastes found in manufacturing environments were similar in nature. For each waste, Toyota developed an effective strategy to reduce or eliminate its effects, thereby improving overall performance and quality.  The process became so successful that it has been embraced in manufacturing sectors around the world. Today’s manufacturers have embraced the concepts and philosophies of lean.  Being lean is considered critical competitive advantage and strategic imperative.  It has made Toyota an automotive success story.

I have been a validation practitioner for over 30 years.  In the process of validating large engineering systems and life sciences quality and compliance management technologies, I have experienced first hand the waste involved in the validation processes.  From manually cutting and pasting screenshots into test cases to manually tracing requirements to test scripts through manual document route/review processes and rewriting scripts for regression testing, waste abounds in the validation process.  As I pondered my work over the past three decades, I began to think of a better way to validate computer systems.

In my initial research, I started to think of all of the wastes throughout validation processes and categorized them.

validation waste

Wastes in the validation process include planning process wastes, testing wastes, documentation wastes, defect wastes, wasteful requirement processes, quality and incident management wastes and wastes associated with the re-validation and re-testing of software applications.  As I began to ponder solutions to eliminate these wastes throughout the validation process, I embarked on the development of a Lean Validation strategy.  We endeavor to save our clients both time and money throughout the validation process.  OnShore Technology Group is the only company whose practice focuses on the the delivery of Lean Validation services.  Our products and services are uniquely designed to power lean validation processes.

Lean Validation

Using lean validation principles can result in the elimination of wasteful manual validation processes and significant improvements in validation efficiency, document cycle times, increased testing productivity and greater ability to identify and correct software defects leading to enhanced software quality, lower costs and improved regulatory compliance.

OnShore Technology Group has pioneered the principles and best practices of “Lean Validation” – the process of eliminating waste and inefficiencies while driving greater software quality throughout the validation process.  In addition to the delivery of expert lean validation services, OnShore’s flagship software application is known as ValidationMaster™ – the FIRST Enterprise Validation Management and Quality system designed to automate lean validation processes.  ValidationMaster™ delivers a single source of truth for any type of validation including software, equipment, process, cold chain, facility, and other types of validation projects.

ValidationMaster™ is also the first validation management system accessible via any Window, Apple or Android mobile device.  The system includes key features such as a validation dashboard, fully automated test script development and execution, automatic requirements traceability, custom report development and generation and a full range of quality management capabilities (training, audit management, change management, controlled document management, ISO, validation KPI’s, CAPA, nonconformance management and much more.

In 2017, OnShore Technology Group was recognized as the “Best IV&V Automation Solutions Provider” and the “Software Validation Testing Experts of the Year”.  In 2016, OnShore made the coveted annual list of CIOReview Magazine as one of the 20 Most Promising Pharma & Life Sciences Tech Solution Providers”.

Contact us today to learn more and see a live demonstration of ValidationMaster™.

What Is The ROI For Automated Validation Testing?

Independent Verification and Validation (IV&V) is a regulatory imperative for software deployments within the life sciences industry.  The FDA provides guidance on what is expected during the validation process and how to conduct due diligence for the validation process.

IEEE 1012 is the industry standard for IV&V.  This industry standard provides the background, rationale, process, list of deliverables and standards for the conduct of validation exercises.  A common question always comes up during my discussions with clients and prospects and that is the question of return on investment of the validation process.  As life sciences firms implement validation processes for their enterprise cloud and on-premise systems, increasingly many companies are turning to outsourcing of the IV&V process to save both time and money.

Validation is traditionally thought of as a regulatory imperative and is generally accounted for in capitalized expenses or direct expenses to the business.  However, as outsourcing becomes more prevalent and companies consider validation as more of a legal best practice, life sciences executives are rethinking IV&V strategies and seeking a cost-effective approach to ensuring compliance and software quality.

It is understood that releasing enterprise software with defects is risky business and costly.  For software applications driving highly regulated processes deficient software can have a direct impact on product quality or other critical quality attributes.  IV&V is designed to improve the quality and reliability of software systems and detect bugs/defects prior to production use.

IV&V activities should be conducted INDEPENDENTLY of the development team.  Many organizations often blur the lines between the two teams but they are distinct.  The figure below shows a typical development team and the correlation between IV&V activities.  The IV&V team receives information from the development team

ieee 12207 sw process with ivv1

The typical V-model for independent validation and verification is highlighted in the figure below.  Lean validation methods and agile processes are designed to streamline efforts, eliminate waste from the validation and improve efficiencies thereby improving ROI.

WHAT IS RETURN ON INVESTMENT (ROI) & WHY SHOULD YOU CARE?

 

ROI is a performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments. To calculate ROI, the benefit (return) of an investment is divided by the cost of the investment; the result is expressed as a percentage or a ratio.

A return on investment formula can be expressed as:

ROI = (Gain from Investment – Cost of Investment)/Cost of Investment

For most life sciences companies, validation is considered a regulatory imperative where the return on investment is not necessarily factored into the decision to validate.  However, validation can be thought of as a legal best practice which can offer return on investment and key benefits.  The key questions to consider are:

  1. What is the cost of the IV&V investment?
  2. What are the tangible and non-tangible benefits gained by life sciences companies?

IV&V along with the cost of quality associated with such projects can typically cost 5 to 10 percent of the total cost of an IT project, depending on the complexity of the project and the specific scope of QA/IV&V activities.

A defect that costs $1 to fix in the requirements or design phase costs $100 to fix  after the software goes into production (live) use.

Several studies indicate that the ROI on IV&V and software quality assurance investments can be 2 to 10 times the investment in QA/IV&V activities depending on how its measured.  There are several studies on the market that address the return on investment for validation as highlighted below.

ROI analysis is an important tool to understand the cost-effectiveness of your validation efforts.  This is why you should care.  Automation of the validation process can play an important role in maximizing your return on investment.  Join us for one of our weekly webinars and let us show you how.

WHAT SHOULD YOU PAY FOR IV&V?

IV&V delivers tremendous value for life sciences companies in early detection and elimination of software defects.  It is reasonable to expect IV&V cost to at least equal internal defect discovery costs or from 5- 10% of a typical project.  This depends on several factors including how customized the system is.  Bespoke development projects are must costlier than COTS systems that require little or no customization.

The simple cost Break Even Point is reached when:

  • IV&V cost = IV&V Savings
  • (1)(0.3)($x/SM)(Zsize)= (100)( Y)($x/SM)
  • (0.3) Zsize = 100 Y
  • (0.3)Zsize/100 = Y *SM – size metric could be expressed as Lines of Code, Function Points, …

One way to achieve greater ROI is through automation.  Automated validation testing can accelerate validation and improve ROI over time.  Automated testing systems like ValidationMaster™ allow you to create a reusable test script library to facilitate regression testing and improve initial testing.

CLOSING THOUGHTS

It should be understood that there is tremendous value inherent in the validation process.  There are several final points for your consideration:

  • No commercial or bespoke software is defect free
  • Early detection is designed to save both time and money
  • Reducing costs by minimizing validation may actually result in INCREASED COST during the operational phase of the system.
  • Validation testing should be automated to achieve maximum ROI