Food quality managers, it is time we discussed the critical importance of validation studies in the quality lab. Although commonly overlooked, microbiological method validation studies are the linchpins of entire quality programs, and method validations done without rigor are crippling our industry’s ability to truly ensure the quality and safety of foods on a daily basis. This article discusses the purpose and importance of microbiological method validation studies and why the food industry should insist on validation study designs of maximum rigor and validity.
A microbiological method, for the purposes of this discussion, is any microbiology test or assay used in the food industry. It may be a test for indicator organisms such as Coliforms or yeast and mold, pathogens such as Salmonella or E. coli O157, or toxins secreted by microorganisms such as Staphylococcal enterotoxin.
A validation study is a one-time study that food safety risk managers complete in order to assure themselves that a new microbiological method produces accurate results that will enable them to effectively measure and manage food safety risk. A validation study is conducted in the actual lab where testing will be performed, with current laboratory analysts, with the specific formulations of foods that are tested regularly.
Food industry regulators and certifying bodies such as SQF expect food producers to use microbiological test methods that are proven fit for use on specific foods. If we are to draw inferences about the fitness of a new test method on specific foods, then we must study how that new test method compares to an accepted reference method, or “gold standard” method. Reference methods are those written in the Food and Drug Administration’s Bacteriological Analytical Manual, the United States Department of Agriculture’s Microbiological Laboratory Guidebook, or ISO methods. Regulators and experts agree that these methods represent the standard to which all other tests should measure up. Methods certified by the Association of Analytical Communities (AOAC) are not considered reference methods and must be validated as fit for use on foods that are appreciably different from the matrices studied. Likewise, AOAC Performance Tested Method (PTM) and Official Methods of Analysis (OMA) certificates are not substitutes for internal validation studies in any given food plant.
In my experience working with quality labs across the United States, I have seen several different validation study designs used to evaluate alternative, more rapid and cost-effective microbiological methods. Some common validation study designs are shown in Table 1. Multiple alternative tests are available, however an internal validation study is needed regardless of the test kit manufacturer. Rarely does a validation study include a comparison to agar plates, which are required for almost every microbiological reference method. Material costs, labor costs, and emergency situations typically prohibit food labs from conducting a rigorous validation study that can speak to the performance of a new method in relation to the current gold standard.
Table 1: Scientific Questions Inherent in Food Microbiology Method Validation Study Designs |
||
| Validation Study Design | Inherent Scientific Question | Does Study Explain Performance of New Test? |
| Test positive or spiked samples side by side on reference method and the new test | Does the test perform comparably to the reference method on my food? | Yes |
| Test positive or spiked samples on the new test | Regardless of accuracy, can the test detect certain or specific bacteria in my food? | No, but may be useful to understand workflow |
| Test any samples side by side on current AOAC certified method and new test | How do the new test’s results compare to my current AOAC certified method on my food? | No, but may be useful to understand workflow |
| Test any positive or negative samples on the new test | Will the new test’s workflow improve my lab’s efficiency? | No, but may be useful to understand workflow |
| This table presents several validation study designs common in the food industry and the scientific questions that are addressed by each design. | ||
It is in the best interests of food producers and the public’s health to conduct rigorous validation studies that give food safety risk managers good information to make correct risk management decisions. In theory, some percentage of unsolved epidemiological foodborne illness clusters must be due to incorrect risk management decisions that allowed contaminated products to reach the market. At the same time, some percentage of all food lot rejections and recalls must be made incorrectly. A portion of these events must be related to food matrix interference that yielded incorrect microbiological results and caused the wrong risk management decision. As they say, “Garbage in, garbage out.”
In addition, including a comparison to agar reference methods in your microbiological method validation study is critical, as it reduces your chances of making an incorrect risk management decision.
Look at things this way: Plants certified with a GFSI accredited quality scheme have already put in effort to ensure analytical equipment such as thermometers and scales are calibrated. Similarly, validating microbiological methods against a reference method is equally if not more important. Finished product microbiology results inform decisions made every day that affect your profits and losses, and those results are likely a primary metric you use to study the effectiveness of your prerequisite programs and preventative controls.
Consider a quality lab that is using an alternative microbiological method that has not been rigorously validated with the plant’s specific foods. Unknown to the lab, the test results every day are twice as variable as the reference agar method and are frequently inaccurate relative to the plant’s product specifications. A rigorous method validation would demonstrate that results on the current method vary widely, while the same samples are consistent with a reference method. This well-intentioned plant is unknowingly making incorrect risk management decisions not just multiple times per year, but multiple times per week, either accidentally releasing contaminated product, reworking product that is acceptable, or disposing of perfectly good product. For the millions of dollars the food producer invests in prerequisite programs, preventative controls, quality personnel, and testing, the plant is unable to optimize their food safety risk management simply due to an unknown and overlooked incompatibility of the microbiological method with the plant’s product.
In my estimation, the costs of rigorously validating a microbiological method on all of your food products outweigh the potential hidden costs that could result from method incompatibility. The business case justifying the costs of a validation study are strong and compelling. And learning how to apply current microbiological methods specific to your foods is not as hard as you might think, considering the large host of test manufacturers, third-party labs, consultants, food safety extensions, and industry groups available to regularly provide study design education and services.