A LIMS Audit Framework: What To Audit & How To Prepare

Laboratory information management systems (LIMS), as well as providing workstream value within the laboratory, are inevitably the subject of regulatory focus and audit. This is due to the criticality of the data and the pivotal role LIMS play in the batch release process. This article considers what to audit and how to audit a LIMS, presenting a framework that will be useful to those planning to undertake a LIMS audit and for laboratory managers who need to prepare for such an audit.

An audit of a LIMS is an examination of the procedures used in a computer system to evaluate their effectiveness and correctness and to recommend improvements. The scope of an audit will vary, and audits of LIMS are often wider for new systems than with established systems that have been subject to multiple audits. In addition, since LIMS will generate data used for batch release, issues relating to the data integrity of batch records will be arguably of greatest concern. Areas to focus on include:¹

• How the LIMS fits with the quality management system
• Whether the LIMS has been subject to quality risk management
• If a supplier/system vendor audit has been conducted
• The design specification for the LIMS
• The user requirement specification for the LIMS
• The validation procedure for implementing the LIMS, including the approach taken, the validation master plan, and the verification method. The validation plan describes all activities, such as review of the user requirement specification, review of the development plan (design), test strategy, verification of the data migration (if applicable), review of the validation documents, and the acceptance testing of the whole system.
• The change control for the LIMS. All new systems should go through change control, and change control should be used appropriately for existing systems. In the event of changes in the computerized system, including version updates, these should be done first in a test environment, after which the validation status needs to be reestablished. If a revalidation is needed, it should be conducted not just for validation of the individual change, but also to determine the extent and impact of that change on the entire computerized system.
• Configuration management of the LIMS
• Training to use the LIMS

With system validation, the purpose of validation is to confirm that the LIMS specifications conform to the users’ needs and intended uses by examination and provision of objective evidence and that the particular requirements can be consistently fulfilled.² However, the extent of validation will depend on the complexity and intended use of the computerized system being validated. The validation effort can be scaled and adapted to the type of system, justified by documented risk assessment.

LIMS Suppliers

Prior to purchasing a LIMS, the supplier should be assessed and audited. For the assessment, policies and procedures for the specification, purchase, development, and implementation of computerized systems should ideally be in place. For the audit, a formal extensive review of the history of the supplier and the software package should be undertaken to gain an additional degree of assurance of the reliability of the software. Several international standards can assist with this process. ISO 9001 provides a quality system model for quality assurance in relation to design, development, production, installation, and servicing, where these key principles can be readily applied to computerized systems. In addition, ISO/IEC/IEEE 12207:2017 provides guidance on acceptable practices for information technology – software life cycle processes and ISO 9004, ISO 10005, and ISO 10007 provide guidance on quality management and system elements, including quality plans and configuration management.

Once installed, users should keep detailed records of the performance of the LIMS and address any errors that arise, as well as drawing together any common themes in an annual review. Areas to review for such an overview are the logbook and audit trail.

Documents

The organization should have procedures relating to data integrity. These will vary from firm to firm, but may include:

• System maintenance SOP – to ensure that appropriate maintenance is carried out in a controlled way.
• Physical security SOP – for control of secure access (intrusion).
• Logical security SOP – for user and password policy.
• Incident and problem management SOP – to describe how to manage and communicate possible problems.
• System change control SOP – covering areas like how the change may affect the process.
• Disaster recovery SOP – to ensure data protection and recovery of processes.
• Backup and restoration SOP – for the control of regular data backup.

Electronic Records

A LIMS is designed to produce electronic records. Electronic record means any combination of text, graphics, data, audio, pictorial, or other information representation in digital form that is created, modified, maintained, archived, retrieved, or distributed by a computer system. In terms of audit checks, electronic records must contain data and metadata and must be available in a readable format. Furthermore, electronic records must be ready to retrieve during the entire period of retention.

Passwords

Since the LIMS generates digitally signed records that must be controlled according to identification rules, the auditor should assess whether the organization:

• maintains unique passwords and ID codes (usernames)
• ensures periodic password checks/changes are in place
• configures ID privileges individually or on a group basis
• assigns operators, supervisors, and administrators different levels of accessibility
• ensures operators do not have database management rights.

Audit Trails

Arguably the biggest focus with a LIMS audit is the system’s audit trail. The audit trail is data in the form of a logical path linking a sequence of events that is used to trace the transactions that have affected the contents of a record. The computerized system should keep a record of any critical actions that occur, such as, for example, who has accessed it and when, any deletion or change of data, etc. Users must not be allowed to amend or switch off the audit trails or alternative means of providing traceability of user actions.

The auditor should check that audit trails are reviewed by supervisors when results are checked. The review of the audit trail should not only be for the test or activity that has been presented but also for any recent activities that may not have been reported, such as a laboratory technician who has run duplicate tests. The auditor should also assess that the audit trail has been time-stamped. It must record the date and time of operator entries and actions that create, modify, or delete an electronic record.

Data Entry

For some systems, and for many GxP systems, there should be procedures in place for critical data entry. This may require a second check, as determined by risk assessment (such as with the entry of manufacturing formula or the transfer of laboratory data and results from paper records). Where a secondary check is made, this will be by someone with a different login name and identification at a given time and date. Should changes to data entry be required, such corrections must be captured in the audit trail.

With data entry, the system must have defined time zone(s) and date standard referencing with relative transaction linking (it is important to note that complex systems may span several time zones).

Electronic Signatures

An electronic (or digital) signature is based upon cryptographic methods of originator authentication, computed using a set of rules and a set of parameters such that the identity of the signer and the integrity of the data can be verified. To be compliant, signed electronic records need to contain:

• The printed name of the signer,
• The date and time when the signature was executed, and
• The action taken (such as review, approval, responsibility, or authorship).

Training

All personnel administering the systems must have appropriate security clearances, training, and skills, together with the necessary knowledge to operate the systems and understand their purpose. Training records should exist for all users of a LIMS, with the records traceable to specific procedures. In relation to training, there should be procedures in place that ensure that entities authorized to use electronic signatures are aware of their responsibilities for actions initiated under their electronic signatures.

Administration

The LIMS should have an appointed administrator. Only the responsible person (this could be designated IT personnel and certainly someone independent of the operation) should have administrative rights to implement any LIMS updates or change critical system settings (such as audit trails or alterations to time/date settings). Such an appointed person should be in place to manage permissions for other users. Other routine tasks, such as for analysis, should be based on user accounts and passwords that do not have administrative rights.

The granting and control of administrative rights should be documented and only be granted to personnel with system maintenance roles that are fully independent of the personnel responsible for the day-to-day use of the system (such as inputting manufacturing data, running laboratory analyses, and so on).

System Security

The security of the system and security of the data need to be assessed and the procedures and records pertaining to these aspects should be based on company-wide policies. Examples of areas to assess include:³

• Access rights for all operators are clearly defined and controlled, including physical and logical access.
• Different levels of access are assigned to users, including separate management and administrator accounts.
• Each individual using the LIMS should have an individual password, traceable to the individual and known only to the individual and an independent administrator.
• A system is in place for removing users who no longer work for the company.
• Procedures are in place to ensure that identification codes and password issuance are periodically checked, recalled, or revised.
• Loss management procedures exist to electronically invalidate lost, stolen, or potentially compromised passwords.
• The system should be capable of enforcing regular changes of passwords.
• Procedures identify prohibited passwords.
• An audit log of breaches of password security should be kept and measures should be in place to address breaches of password security.
• The system should enforce revoking of access after a specified number of unsuccessful logon attempts.
• Measures are needed to ensure the validated recovery of original information and data following backup, media transfer, transcription, archiving, or system failure.
• Attempted breaches of security safeguards should be recorded and investigated.

System Backup

There should be a documented and validated backup procedure including storage facilities and media. With this, all GxP related data, including audit trails, should be backed up. The process should assure data integrity. The frequency of backup is dependent on the computer system functions and the risk assessment of a loss of data. Performance of backups should be visible via a review of the audit trail, and a record of rectification of any errors should be kept. Tests should be conducted to show that backed up data is retrievable following a system breakdown.

The routine backing up of data should involve placing the data into a safe storage location, adequately separated from the primary storage location. This could be storage media held in a fireproof safe or on a server. The media used should be documented and justified for reliability.

Control Of Computer System Environments

Computer systems should lock out when not in use and may need to be located in controlled areas. Server rooms should have restricted access and maintain the conditions necessary to ensure the correct functioning of equipment (with control of temperature, firefighting measures, uninterruptible power supply, and so on).

5-Step Approach

One way to conduct a LIMS audit is to adopt the five-step approach.⁴ These are:

1. Conduct the initial review (planning the audit).
2. Review and assess internal controls.
3. Conduct compliance testing (test the internal controls).
4. Perform substantive testing (test the detailed data).
5. Reports (conclusions and findings).

The auditor(s) should reach an understanding with the client concerning the scope and limitations of the audit from the very beginning. This will facilitate accomplishment of the audit objectives in an effective and efficient manner. As part of audit preparation, the auditor should conduct a preliminary survey of the entity to plan how the audit should be conducted. The auditors gather information about the LIMS that is relevant to the audit plan, including: a preliminary understanding of how the LIMS’ functions are organized; identification of the computer hardware and software used by the entity; a preliminary understanding of each significant accounting application processed by computer; and identification of planned implementation of new applications or revisions to existing applications and applicable controls.

With LIMS there are two types of controls: general and application. General controls are those that cover the organization, management, and processing within the computer environment but are not tied to particular applications. They should be tested prior to application controls because if they are found to be ineffective the auditor will not be able to rely on application controls. General controls include such things as proper segregation of duties, disaster plan, file backup, use of labels, access control, procedures for acquiring and implementing new programs and equipment, and so on. Application controls relate to specific tasks performed by the system. They include input controls, processing controls, and output controls and should provide reasonable assurance that the initiating, recording, processing, and reporting of data are properly performed.

The system owner should perform compliance testing to determine whether the controls actually exist and function as intended. There are three general approaches to compliance testing: the test data approach, taking a laboratory example, where the auditor has test results processed through the client’s system and then compares the results to predetermined results; the integrated test facility approach, where dummy test results are processed along with real test results and compared to auditors’ predetermined results; and the parallel simulation approach, in which real test results are processed through the client's system and also through a parallel system set up by the auditor using the same programs, and the results are compared. Whichever of these test approaches is used, the results should tell the auditor if the controls exist and are functioning properly.

Significant Non-Compliances

Although the auditor may pick up on several areas of concern, arguably the most significant issues that an auditor could find are:

• The lack of a written detailed description of each system
• System log not kept up to date with controls over changes
• Weak security in place
• No audit trails in place or audit trails not active
• Lack of evidence for the quality assurance of the software development process
• Inadequate validation of the LIMS
• Improper data manipulation
• Adjustment of time clocks
• Backdating of information
• Creating records after the fact or without actually executing the procedure
• Excluding adverse information
• Sharing of passwords
• Discarding or destroying original records

The above list feeds into the area of data integrity. Preventing data integrity breaches can be addressed with three primary elements: personnel and training, good system validation, and maintaining security.

Summary

This article provides a framework for conducting a LIMS audit, providing advice for both the auditor and auditee. Focal points within the audit process include verifying how well LIMS handles electronic data exchanges, how an instrument's input and output data are managed, and how reliable the outputs are in terms of batch reports and trend reports.

This article has been adapted from chapter 2 of the book Digital Transformation and Regulatory Considerations for Biopharmaceutical and Healthcare Manufacturers, Volume 2, written by Tim Sandle and co-published by PDA and DHI. Copyright 2021. All rights reserved.

References

1. PDA (1999) Validation and Qualification of Computerized Laboratory Data Acquisition Systems, Parenteral Drug Association, Technical Report #18, Bethesda, MD, USA
2. Wingate, G. (1997) Validating Automated Manufacturing and Laboratory Applications: Putting Principles into Practice, Taylor and Francis, New York, USA, pp10-15
3. Skobelev, D.O., Zaytseva, T.M., Kozlov, A.D.,  Perepelitsa, V.L. and Makarova A. S. (2011). Laboratory information management systems in the work of the analytic laboratory. Measurement Techniques. 53 (10): 1182–1189
4. Conti, T.J. (1992) LIMS and quality audits of a quality control laboratory. Chemometrics and Intelligent Laboratory Systems, Laboratory Information Management, 17: 301–304

About The Author:

Tim Sandle, Ph.D., is a pharmaceutical professional with wide experience in microbiology and quality assurance. He is the author of more than 30 books relating to pharmaceuticals, healthcare, and life sciences, as well as over 170 peer-reviewed papers and some 500 technical articles. Sandle has presented at over 200 events and he currently works at Bio Products Laboratory Ltd. (BPL), and he is a visiting professor at the University of Manchester and University College London, as well as a consultant to the pharmaceutical industry. Visit his microbiology website at https://www.pharmamicroresources.com.