Tag Archives: salmonella

Thomas R. Weschler, Founder and President, Strategic Consulting, Inc (SCI)

High False Positive Rates for Pathogen Food Safety Testing

By Thomas R. Weschler
No Comments
Thomas R. Weschler, Founder and President, Strategic Consulting, Inc (SCI)

This article looks at proficiency testing (PT) for pathogen analysis, and the recent finding by the the American Proficiency Institute (API) of a 6.6 percent false-negative rate on food safety PT samples (14-year average for the 1999-2012 period).

While at IAFP this year, I met with Heather Jordan, who directs food PT programs at API. The proficiency testing programs are used at many food labs in conjunction with lab accreditation programs. Proficiency testing is done at food plant labs (FPLs) and corporate labs, as well as at food contract testing labs (FCLs) as a way to demonstrate quality results in their food micro and chemistry testing.

More proficiency testing but less proficiency?

In fact, the use of PTs is increasing in food labs, which is probably tied in part to the push for lab accreditation by FSMA and non-government groups like GFSI. Yet it seems to me that the current use of PTs doesn’t go far enough to enable an FPL or FCL to demonstrate overall laboratory competency, and gain or maintain accreditation (ISO 17025).

In most labs, PTs are done just a few times a year. And really, they test the competency of the lab technician and protocols used in analyzing the PT samples. They are not a holistic measure of the lab and its ability to consistently generate quality results on every test run by every operator in the lab.

In a previous life I ran a group of environmental testing labs, which also are required to run PT samples during the year. From this experience, I know that lab personnel are aware that PTs are in-house: The sample-receiving group logs them in, and then alerts management. As a result, the best operators usually are assigned to run the PTs. This kid-glove treatment is not representative of day-to-day practices and processes. If we really want to validate and accredit the proficiency of an entire lab, shouldn’t every operator be tested on all protocols in use?

Plus, if labs know when they are running PT samples, and likely have their best operators running them, shouldn’t there be few, if any, false-negative or false-positive results? Surprisingly, that’s not what the API research found…

API study: Performance accuracy for food pathogens remains problematic

In a retrospective study, “Pathogen Detection in Food Microbiology Laboratories: An Analysis of Proficiency Test Performance,” API analyzed the results from 39,500 food proficiency tests conducted between 1999 and 2012 to see how U.S. labs are doing in detecting or ruling out contamination of four common food pathogens.

Over the 14-year period, “False negative results ranged from 3.3 percent to 14.0 percent for E. coli O157:H7; 1.9 percent to 10.6 percent for Salmonella spp; 3.4 percent to 11.0 percent for L. monocytogenes; and 0 percent to 19.8 percent for Campylobacter spp.” Most concerning is that while both false positive and false negative rates were down in the last year of the study, the cumulative false negative rate for the 14-year period was 6.6 percent.

As we know, false positive results (in which a sample that does not contain pathogens is incorrectly shown as positive) are a nuisance. But false negative test results—which fail to detect true pathogenic organisms in the sample—are not unacceptable.

Tom-Weschler-False-Negatives-Sep-2014

The cumulative average false positive rate was 3.1 percent, less than half of the false negative rate for the same period.

The objective of the study—and, I would think, of proficiency testing in general—is to demonstrate improvement in lab performance year over year. The results of the API report concluded to the contrary, however: “Performance accuracy for food pathogens remains problematic with the recent cumulative trend showing a slight decrease for false positive and false negative results.”

Clearly if false negatives happen in proficiency programs, they happen in the course of regular testing at food labs. I’m told that many FCLs and FPLs rely on other parts of their QA systems to make sure testing is being conducted properly. Even so, the documentation of ongoing and unacceptably high false negative rates in PT testing is a big concern for everyone. It also points to a number of follow-on questions:

  • Would the false negative and false positive results be even higher if every technician, rather than the best operator, performed the analysis?
  • PT samples are created in only a couple of sample matrices. Would results be even worse if performed on the myriad of sample matrices present in the food industry?
  • What are the performance results among all of the pathogen methods available? Are some methods better than others when measured in real world conditions? Do the more complex protocols of some pathogen diagnostic systems result in poorer PT performance results?
  • Would PT results and, even more important, lab proficiency improve if the frequency of PTs increased, and were required of every technician involved with real food samples?
  • How can proficiency testing be used to isolate problem areas, whether in the pathogen diagnostic method or the competency of lab operators and processes?
  • And finally, is the performance data different between food contract labs and food plant labs? And are all FCLs are equal, or are some more able to deliver quality results?
Bob Savage, President and Founder of the HACCP Consulting Group

FSQA: Creating HACCP Excellence

By Michael Biros
No Comments
Bob Savage, President and Founder of the HACCP Consulting Group

For a successful Food Safety and Quality Assurance program, there must be management commitment and measurable expectations set. Senior management has to be committed to the program. They are the foundation for everything in food safety. They have to provide resources to develop and implement the plans across different departments as well as provide for training and encourage communication, advises Robert A. Savage, President and Founder of The HACCP Consulting Group.

Sharing some lessons learned from decades of HACCP implementation experience, Savage spoke at a recent webinar on FSQA: Creating HACCP Excellence, presented by SafetyChain Software. We present excerpts below. 

A few years ago, there was a very serious Salmonella outbreak in peanut butter. It appears that company shopped around for negative salmonella results and then shipped the product. It’s a worst case scenario, but in this case, short term profitability at the expense of food safety resulted in the over 600 illnesses and a few deaths as well as the bankruptcy of the company, described Savage.

Role of GMPs in Creating and Minimizing CCPs

Without good GMPs, facilities tend to have more CCPs than necessary. There has to be a good balance between GMPs and CCPs. When companies understand the the relationship between HACCP, GMPs, and CCPs, typically the HACCP plan would not have more than 3 or 4 CCPs and everything else is covered by GMPs. 

Best practices for HACCP management must be committed from the beginning and throughout the process. GMPs should be in place prior to even beginning to revamp the HACCP plan. Multidisciplinary HACCP teams, including QC, QA, Lab, Sanitation, Product Development and Sales, experts, should contribute to the process in developing the plan. Having a multidisciplinary team helps with achieving buy-in or company-wide commitment to the plan. 

Companies have been pretty good with monitoring, but there’s still some confusion between verification and validation. Verification is a check of the checkers. When CCPs are identified and monitored, verification is making sure that the company says what it’s doing and is doing what it says. Validation asks if the company has the right CCPs and how can they prove it. 

Best Practices for Audit Prep

USDA regulated plants have routine inspections to verify what the companies do on an everyday basis. Separate from these routine inspections, USDA also performs food safety assessments which can take days or weeks to complete. Companies under USDA jurisdiction should do their own food safety inspection to prepare for these FSIS audits. FDA regulated plants may go months or years between inspections. These facilities should have third party audits such as SQF or other audits. 

GFSI schemes have taken hold in the US and around the world. The popular one in the U.S. is SQF. Companies that meet SQF standards should have no problem meeting new FDA FSMA regulations.