Critical Thinking In Root Cause Investigations: Are You Asking The Right Questions?

Critical thinking is one of those terms we hear about all the time but sometimes have difficulty simply defining. Here’s one definition to consider: Critical thinking is an unbiased evaluation of an argument (or statement) in order to fully understand the argument and its intent, conclusion, and the quality of evidence used to support it. According to a report by the consulting firm McKinsey, critical thinking skills in personnel are extremely important to management when making a hiring decision.¹

A root cause investigation is one particular situation where critical thinking should be applied; there are many other opportunities where it can be used as well. The focus of critical thinking is on the argument someone makes, such as when stating and defending why you claim that you have correctly identified the root cause of a deviation or quality event.

The term argument, as it is used in critical thinking, is not two people screaming at each other but rather a statement (written or spoken) that is meant to persuade or convince someone to a particular position or course of action. The word has its roots in the Latin word arguer, which means to “enlighten” or “prove.” In an investigation report, the reader wants you to convince or prove to them that you have identified the true cause(s) of the unwanted event.

When critically examining an argument, one needs to first dissect its parts. Consider this paragraph (adapted from a Wikipedia entry) and look for the issue, the conclusion the argument makes, and the reasons for that conclusion:

Aspirin, also known as acetylsalicylic acid (ASA), is one of the most frequently used medicines globally, with 40 to 120 billion dosage units made each year. It is an important, valuable drug because of its extensive effects on the human body. For example, it is effective in treating inflammatory conditions, headaches, and minor pain. For a person with pain or fever, they would generally start to sense relief within 30 minutes or so of taking the drug. As a preventive drug, low-dose aspirin is taken by those at high risk to reduce the likelihood of heart attacks, ischemic strokes, and blood clots.

• Issue – why this argument (or statement) is being made: That aspirin is an extremely important drug product.
• Conclusion – the message (sometimes directly stated, sometimes — like here — implied) that the writer wants you to accept: You might benefit from using aspirin.
• Reasons – the evidence or statements used to support the conclusion: it is effective in treating inflammatory conditions, headaches, and minor pain; it works quickly (“relief within 30 minutes”); it prevents some undesirable medical conditions.

Now examine this argument used in a hypothetical deviation investigation and identify its elements:

The cause of the dirty tablets that were submitted as a lab sample was that the manufacturing technician who took the samples dropped them on the ground, picked them off the floor, and tried blowing off any dirt before putting them back into the sample container but did not tell anyone.

• Issue – why this argument (or statement) is being made: Providing a root cause for the deviation (i.e., tablets that had “dirt” on them).
• Conclusion – the message that the writer wants you to accept: The tablets were dropped, became contaminated, picked up, put into the sample container…
• Reasons – the evidence or statements used to support the conclusion: [No evidence provided.]

When a key element is missing, the argument being made does not stand up to scrutiny. In this example the reader asks, “How do you know this?” “What data or information (like a statement from the technician, a witness who observed the action, or an analysis of the “dirt”) supports this conclusion?”

There are several useful questions that one should ask when confronted with a statement that will help you critically evaluate it.² These questions can be used when critically assessing an investigation report or more broadly, any statement or argument:

• Are there any words or phrases in the statement that are ambiguous and not clearly defined or specific? For instance, in the last example, the phrase “dirty tablets” might be better written as, “[product name] tablets with unidentified particulate matter on the exterior surface.”
• What assumptions are needed to help connect the reasons with the conclusion? In the case of the tablets, an obvious assumption is that “dirty” tablets are unwanted and therefore, this is a deviation from what should be.
• Are there any logical fallacies or weaknesses in the reasoning provided? In making an argument, the writer may use logical constructs or reasoning that is weak, such as in these following two examples: If the tablet argument above included the phrase, “The technician involved is totally incompetent and is always messing things up,” the writer is attacking the person. This type of inappropriate reasoning is called an ad hominem fallacy. Another logical fallacy is mistaking the association of two events for causation. If, for example, Pat, who had microbial contamination on his gown after an aseptic filling, is working on a media fill (i.e., aseptic process simulation) that fails, it does not mean he caused it. There may be many other factors – HVAC issues, other personnel, contaminated containers – that could be the cause. This is an example of the post hoc fallacy. Association is not causation.
• What is the quality of the evidence provided? This is one of the most important when it comes to technical and scientific arguments. How good is the data? How big was the sample? How were the samples taken? Is the evidence used cited back to reliable primary sources?
• What is the credibility of the author/speaker? What is the level of the writer’s or speaker’s experience? Are they an expert who has worked in the field for a long time? What organizations is the person affiliated with? Is the person providing the statement acknowledged by other experts as having current knowledge in the field? Has the article or statement been peer reviewed and published in a reputable journal? Are other reputable sources or experts in agreement?
• Could there be another reasonable explanation for the event? For example, could the sample container have been contaminated? Was the compression equipment compromised in some way? (When writing an investigation report, it is very useful to review what was examined even if there was nothing found. This is called “reporting negative information.”)
• Are statistics used properly? Is the term “average” used or is a more precise word, such as “mean,” “median,” or “mode”? When describing the difference between values, are the actual results given (e.g., 20 and 25) or the percentage difference (e.g., 25 percent change)? Are these meant to clarify or to mislead?
• If graphs or figures are present, do they distort the true meaning in some way? For example, are the axes clearly labeled? Are they linear, starting at “0,” or do they start in such a way to emphasize the difference between the categories (see Figure 1)?

Figure 1: Graphing the same data two ways – with a change of the scale on the y axis.

Critical thinking, particularly at first, does take some extra brain power and time, but the more you do it, the better you get. As you read your own work or listen to the arguments you make, you will start to improve them and make them stronger.

It is more important today than ever before, in our professional and personal lives, to apply critical thinking to what we hear and read. In some cases, writers and speakers may just be unaware or sloppy in how they are providing information or do not see the importance in making a solid case for what caused the unwanted event. In other situations, there is a definite intent to mislead.

As we write or review a root cause investigation, it is up to each of us to challenge the information we are given and see if it stands up to having a critical light shined on it.

References:

1. McKinsey (2020). Pharma Operations: Creating the workforce of the future. https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/pharma-operations-creating-the-workforce-of-the-future. Accessed 16 Jul 2020.
2. Brown, M.N., Keeley, S.M. (2015). Asking the right questions: A guide to critical thinking. 11^th Edition. Essex, England: Pearson.

About The Author:

James Vesper, Ph.D., MPH is a director at ValSource with a focus on learning, training, quality risk management, and error investigation. With more than 35 years of pharma experience, he started his career with Eli Lilly and Company and then started the consulting firm LearningPlus. Vesper has worked globally with pharma firms, regulatory agencies, and the World Health Organization. He has just completed his sixth book, GMP Root Cause Investigations and Corrective Actions: A Clear and Simple Guide, available from PDA/DHI. He can be contacted at jvesper@valsource.com.