Skip Validation, You’re Asking for Problems

Running an unvalidated program or product is like betting your life’s savings on a horse because you overheard a “surefire tip” outside the racetrack, or driving around without any mirrors.

To put it less dramatically: Skipping validation is asking for problems. But what does validation mean, how much is necessary, and what’s the best way to include it in your plans?

In order to start understanding validation, we must first break it down into two main categories: Product validation and process validation. From there, it’s important to look at whether something has been broadly validated for general use, and whether it has been narrowly validated for use in your specific situation. That last question is where people often struggle: How can we ensure this product or process is validated for use in the way that we plan to use it?

Validating an on-site allergen test kit requires a few different layers of research and testing. Taking the time to carefully design and vet a validation process may seem tedious, and it may require some additional up-front costs—but in the long run, it’s the only way to ensure you are spending your money on a test kit that works. And if you’re using an allergen test kit that doesn’t actually detect allergens in your facility—best-case scenario, you’re wasting money and time. Worst-case scenario, you’re headed straight for a recall and you won’t see it coming until your customers get sick.

If you are buying a test to determine the absence or presence of allergens in your facility (specific or general), you’ll likely ask the kit manufacturer if the test kit has been validated. This validation can come in many forms, most commonly:

Third party validation (eg., AOAC)
Internally produced validation documents or whitepapers
Published studies

A product with more validation (third-party certifications, studies, whitepapers) isn’t necessarily better than a product with less. It may have simply been on the market longer or be produced by a company that allocates its funding differently. However, validation documents can be very comforting when reviewing a product, as they provide a starting point for your own research. When you are reviewing validation data, ask yourself a few questions:

Does this data cover products like mine?
- Are the ingredients similar (raw meat, ice cream, spices, etc.)?
- Are the preparation processes similar (heat, fermentation, etc.)?
Does this data cover an environment like mine?
- Will the tests be run the same way in my facility as in the data?
- Is the contamination being introduced in a way and amount that feels realistic to the risk factors I know about in my facility?
Does the data mention any complicating factors (and do I need to care about them)?
- Are there ingredients known to cross-react or cause false negatives?
- Are there processes known to change the LOD or cause false negatives?
If I am aware of limitations with other similar test kits, are those limitations addressed in the data for this test kit as well?

To give an example, let’s imagine you make premium ice cream and are reviewing allergen test kits that look for peanuts and almonds in product, in rinsewater and on surfaces. You’ll want to ask questions like:

How does the kit perform in a high-fat environment?
Does the validation data cover product, rinsewater and surfaces?
Are there ingredients in our facility that are called out as cross-reactive (or otherwise troublesome)?
Do our ingredients get exposed to temperatures, pH levels, or other processes that impact the LOD?

You might learn, for example, that one of the matrices tested in validation was ice cream. If so: Wonderful! That’s a vote of confidence and a great starting point. Or maybe you learn that the kit in question isn’t recommended for matrices that include an ingredient in your formulation. If so: That’s equally wonderful! Now you know you need a different solution. Or maybe the instructions on your current peanut test kit indicate that heavily roasted peanuts have a higher detection limit than raw peanuts, but this new test kit only has data for raw peanuts. If so: OK! You have more research to do, and that’s fine too.

In short: Pre-existing product validation data is a helpful starting point for determining whether or not an allergen test kit MIGHT work well in your facility—but it doesn’t eliminate the need for you to run your own internal validation study.

Once you’ve identified an allergen test kit that you want to use in your facility, you’ll want to prove that it can work to identify contamination in your specific environment. This is where a more narrowly tailored validation comes into play. Your test kit provider may have resources available to help you design an internal validation. Don’t be afraid to ask for help! A reputable test kit provider should care not just about making the sale, but also about making your food safer.

Before you even order a new test kit, you should have a good idea of how your validation process is going to work. It’s important to have both the study design and study outcome on file. Here are some possible additions for your internal validation study:

Validating that an allergen test kit can reliably prove your surfaces are clean of said allergen:

Test the surface prior to cleaning, after the allergen in question has been run. Do you see positive results? If not, then a negative result after cleaning is essentially meaningless.
Test the surface after cleaning. Do you see negative results? If not, it could mean a problem with your cleaning process—or a strange interference. Both require further research.
If your products encounter multiple surfaces (eg., stainless steel and also ceramic), test them all with before and after testing.

Validating that an allergen test kit can reliably prove your rinsewater is free of said allergen:

Test water from the beginning of the cleaning cycle as well as the end. Do you see a change in results, from positive to negative?
If you don’t ever see the allergen present in your rinsewater, you may want to “spike” a sample by adding a small amount of the product that contains the allergen into the rinsewater you’ve collected. Could it be that something in your cleaning protocol or some aspect of your matrix is affecting the detection limit?

Validating that an allergen test kit can reliably prove your ingredients or finished products are free of said allergen:

Test a product that you know contains the allergen but is otherwise similar. Keep in mind that some allergen test kits can be overloaded and can show false negatives if too much allergen is present in the sample—if you aren’t sure whether the test kit you are trialing has this limitation, ask your supplier. Do you see a positive?
Have you encountered batches of your product with accidental cross-contamination from the allergen in question? If so, and you have some of that batch archived, run a test on it. Would this kit have identified the problem?
Do you have a batch or lot of product that has been analyzed by a third-party lab? If so, do your results in-house match the lab’s results?
Run—or ask a lab to run—a spiked recovery. This is especially important if there is no pre-existing data on how the test kit works against your specific matrices.
- Some test kit manufacturers can provide this service for you—you would simply need to send them the product, and they can add various amounts of allergen into the product and confirm that the test kit shows positive results.
- Some kit manufacturers or other suppliers can send you standards that have known quantities of allergen in them. You can mix these into your product and run tests, and confirm that you get positive results when expected.
- You may want to simply do this on your own, by adding small quantities of the allergen into the sample and running tests. However, take care to be especially careful with your documentation in case questions arise down the line.
No matter how the spiked recovery is being run, consider these two factors:
- Be sure you’re including what could be a realistic amount of contamination—if you’re concerned about catching 25ppm of allergen, loading up your sample with 2000ppm won’t necessarily help you prove anything.
- The matrix of your allergen-containing foods is just as important as the matrix of your allergen-free foods. If your allergen has been fermented, roasted, pressurized, etc. —your spike needs to be processed in the same way. If you aren’t sure how to think about your matrices, this previous Allergen Alley post is a good starting place.

Once you’ve proven that the test kit in question can in fact show positive results when traces of allergen are present, you can confidently and comfortably incorporate it into your larger allergen control plan. If your matrices change, you’ll want to re-validate whatever’s new.

While it can be tempting to rely on a kit’s general validation, taking the extra step to validate your unique matrices is an essential part of a truly robust food safety plan. If you’re stumped for how to begin, contact your kit provider—after all, you share the same goals: Safe, allergen-free food for consumers who rely on you to keep themselves and their families healthy and well fed.

March 6, 2019

Ghee Whiz!

By Susanne Kuehne

Ghee, the glorious clarified butter leading a creamy buttery flavor to dishes and widely used in Asian cooking, is now also target for food fraud. Real ghee is based on pure butterfat extracted usually from cow’s milk, without impurities or additives. In Surajpole, India, an adulterated ghee operation was seized that used refined soybean oil and added butter flavor, which may have created a health hazard.

Resource

Unit producing spurious ghee raided in Surajpole. (February 25, 2019). Accessed February 28, 2019. Retrieved from https://timesofindia.indiatimes.com/city/jaipur/now-spurious-ghee-manufacturing-unitraided/articleshow/68142661.cms

March 5, 2019

Architect the Perfect Food Safety Team: How to Assess the Candidate

By Bob Pudlock

If there’s anything to take away from our three-part series on Architecting the Perfect Food Safety Team, it’s to be thoughtful and cognizant of what behavior and end results you’re looking for from your team.

When you enter the ASSESS phase, it’s important to arm yourself with questions that elicit responses that give you an indication of whether the person CAN perform well and who will thrive in your company’s culture.

Additionally, you want that behavior and end result to be synced and aligned against the broader organization’s mission.

Bob Pudlock will be moderating the panel discussion, “Food Safety Leadership: Earning Respect”, at the 2019 Food Safety Consortium | October 1–3 | Schaumburg, ILAs a director or VP, that’s an important distinction. A company that’s in M&A mode is much different than one that’s under pressure from a major customer to get in compliance with their supplier guidelines. The ideal candidate for one is not necessarily the best fit for the other.

Let’s say you’re a director or VP at a company who has just acquired a smaller company certified under a different GFSI scheme. For the next year to two, other integration efforts have been prioritized over folding the acquired company’s scheme into the parent company. For the foreseeable future, that means there will be a disconnect in some protocols, reporting and expectations between the parent and acquired company.

In this scenario, we brainstorm with our client and bring forward themes or dynamics that will be present.

Transition -> Change -> Ambiguity
Gray area -> Open loops not immediately resolved -> Discomfort
Acquisition / Integration -> power/influence/reporting structure transition -> Ambiguity

Two themes that come out of this brainstorm are “dealing with ambiguity” and managing discomfort associated with a GFSI certification or being downgraded as a priority while the company pushes other objectives forward.

For these themes, now we look to construct questions that elicit how one has behaved in past. We can also construct a hypothetical scenario to see how an individual would strategize and act moving forward.

As you look at potential hires into the organization at this stage, you’re going to be presented with a range of candidates that exhibit varying degrees of emotional flexibility.

Emotional flexibility is the ability to identify, assess and adjust responses to events, circumstances or triggers as they arise.

If a candidate in this scenario is rigid or exhibits a black and white “compliance or bust” mentality, that’s going to be a source of constant friction for the individual and those with whom they interact. During the integration efforts, that person is going to have a hard time calibrating their feeling of incompleteness or disconnect from the broader organization.

A candidate that exhibits a low degree of emotional flexibility will have a harder time “rolling with the punches” and will make those around them uncomfortable—they’ll push and work towards a set of expectations that is not consistent with the broader organization’s timeline. Now there is certainly an opportunity to manage that individual’s expectations, but the less we have to do that as leaders, the better—hence, the importance of accurately assessing and pegging the attributes early and often in the interview process.

So, how do we do that?

We advocate for what we call “layered” interview questions. In simple terms, it’s asking a question a number of different ways and in different contexts to elicit responses that offer an accurate prediction of how someone will react in the future.

The first question might be (one layer): How do you deal with ambiguity?

Another question would be: How have you dealt with a situation in the past where your boss was not on site but your plant manager took on day-to-day supervising and reporting? The hiring team can shade in specifics to make the scenario more realistic. The core of the question is to create an image of an ambiguous environment.

And yet another question would be (this is hypothetical): Let’s say you’re alone and have been put on the spot to solve this food safety problem. Your boss is not available for the next two days and you believe that there might be a disconnect between how the plant manager might solve the problem and how your boss would solve it. What would you do?

Asking a question three different ways will ferret out canned, stale answers. Additionally it will test a candidate’s ability to “connect the dots” between past experience and current/future challenges in the workplace. Asking these questions and observing the responses is a significant improvement over what’s typically done.

Taking it a step further will give you an even more accurate prediction of whether a candidate will thrive or perish in this environment. To test this, drill down further on the candidate’s responses to each layered question to the point where you determine what toll, if any, these situations took on them; in other words, how does ambiguity and discomfort in the workplace make them “feel”? Just because someone has experience in ambiguous climates or has managed “transitional” situations like a company’s M&A phase doesn’t mean they thrive on it. For some people, it’s a constant stressor, and if they’re not emotionally resilient, at some point they will break or wear down.

Drilling down to this level will give you a more accurate feel to how resilient and tolerant one will be in your company’s current and future culture. Additionally, you’ll determine whether or not their level of emotional flexibility will allow them to thrive or cause a constant stressor that will ultimately wear them down.

This is just one example of how to use a layered question. There may be three to four key themes that you want to dig into for each role in your organization. Pairing the layered questions (same question, multiple contexts) with eliciting feeling-based responses will give you an even more accurate predictor of who will thrive on your team. It will also isolate and disqualify individuals who have canned responses to interview questions and/or who can’t think on their feet, or struggle to tie past experiences to current state challenges.

This might “feel” like a lot of effort, but it’s an immense time and energy saver when you balance it against the aggravation, time and energy-suck you experience with a poor or mediocre hire.

Read on to Part III of this series: Architect the Perfect Food Safety Team: Capture Your Ideal Candidate and Set Them Up for Success

February 26, 2019

In Wine, There Is Not Always Truth

By Susanne Kuehne

Three arrests were made and at least 11,000 bottles of red wine labeled high-quality IGT Toscana wines have been seized in Italy for containing lower quality wines and fraudulent labeling, misrepresenting the wine’s geographic origin. The investigation was taken to the Europol level in conjunction with German and Italian law enforcement authorities.

Resource

Authorities shutdown international wine fraud operation; three in Italy arrested. Food Safety News. Accessed February 19, 2019.

February 20, 2019

Food Entrepreneurial Trends

By Matthew Botos

Over the last two decades I have had the utmost privilege to work with food manufacturers from every sector. From dried goods, meats, poultry, school lunches programs, etcetera. I have seen almost everything. The leading trend I have seen over the last 20 years is the steady rise of beverage companies. Everyone likes a different type of beverage: Water, juice, cold coffee, teas, energy drinks, kombucha… the list goes on. The two most important things a small company entering the market needs to remember is to make sure that first, you can comply with the regulators, and second, that the people you are selling your products to also understand their market (buyers and retailers).

The marketplace loves new and innovative products. We need, as an industry, to continue to support innovation in the beverage industry. Healthcare trends, nutraceuticals, supplements, and “fresh” are some of the many things consumers are looking for in a product. We often have questions such as, “I have the perfect recipe for a new beverage product—how do we get it out into the market?” The first question to ask yourself is: Do you understand your product and its intended use? And, second: Do you understand how the physical properties of this product impact how it will need to be transported as well as stored and used by the final consumer? Both of these questions come back to—and can be answered by—food safety. Companies can have the greatest product ideas; however, if you do not have the ability to make sure it has a safety factor during transportation, as well as the ability to communicate your food safety plan, an entrepreneur can jeopardize the future of their company before it is even allowed to begin.

I have used the analogy that food safety best practices are like a sport—the more you train, practice and focus on the “basics done well,” the better your plan will be on a day-to-day basis. Bottom line: The focus is making the food supply safer. Please take note the transition to new practices does not mean that an existing HACCP plan (Hazard Analysis and Critical Control Points) is invalid! As a matter of fact, HACCP and Good Manufacturing Practices (GMP’s) should be looked at as the foundation of FSMA’s Preventive Controls for Human Food. Local departments of public health still rely on HACCP as their main line of defense for the food safety industry. We have seen so many small processors and restaurants that have inspections where HACCP is still the focus, even though Preventive Controls has some more advanced techniques for protection. Both HACCP and Preventive Controls focus on making sure you have good sanitation practices, employee training, and have done a hazard analysis for biological, chemical and physical hazards. I believe the lines are blurring a bit—companies, academics and regulators don’t often understand the differences between the two and, to be honest, the differences are not that substantial from a fundamental level. Ultimately, companies are responsible for their own food safety best practices.

As we trend toward new and innovative beverage concepts, we need to be partners with both our regulators and our customers. There is so much information across the United States and companies that are in emerging markets will only grow and develop if they have proper food safety plans as a foundation. Emerging companies should get connected early and learn from other companies and organizations, and become more proactively involved in the food safety element of their product.

I believe 2019 will be an incredible year for the rise of food and beverage startups. We have already seen much growth in the knowledge of food supply due to easily accessible information about products. Keeping a transparent conversation with regulatory personnel and customers is key to the success of any food or beverage company.

February 19, 2019

Bee Careful What You Eat

By Susanne Kuehne

Food Fraud, honey — These types of records can be found in the Food Fraud Database
Image credit: Susanne Kuehne

One of China’s most famous health brands has been banned from making honey and issued a steep fine in China after selling expired honey. For a long time, the brand’s “Premium” honey was a supposedly safe alternative in China compared to “fake” honey, mixed with sugar syrup.

Resource

Executives of TCM company in trouble over honey
Wen, X. (February 13, 2019). Executives of TCM company in trouble over honey. Accessed February 13, 2019. Retrieved from http://www.chinadaily.com.cn/a/201902/13/WS5c635b26a3106c65c34e8fba.html

February 12, 2019

How Food Fraud Happens

By Karen Everstine, Ph.D.

The food industry has been hard at work over the past few years implementing food fraud mitigation plans in response to Global Food Safety Initiative (GFSI) certification program requirements. GFSI defines food fraud as:

“A collective term encompassing the deliberate and intentional substitution, addition, tampering or misrepresentation of food, food ingredients or food packaging, labelling, product information or false or misleading statements made about a product for economic gain that could impact consumer health.” (GFSI Benchmarking Requirements, 2017)

GFSI then further defines the terminology of food fraud by citing seven categories (shown in the following diagram).

GFSI, Food Fraud — Used with permission from GFSI

In the Food Fraud Database, we categorize food fraud records using the following terminology (with examples):

Dilution/substitution
- Substitution of an entire fish fillet or partial dilution of olive oil with another oil
Artificial enhancement
- Addition of melamine to artificially increase the apparent protein content of milk or the addition of coloring agents to spices
Use of undeclared, unapproved, or banned biocides
- The use of chloramphenicol in honeybee populations (where not permitted) or the addition of hydrogen peroxide to milk
Removal of authentic constituents
- The sale of “spent” spice powder (used in the production of an oleoresin) as a whole spice powder
Misrepresentation of nutritional value
- Infant formula that does not contain the required nutritional content
Fraudulent labeling claims
- Misrepresentation of label attributes related to production method (organic, kosher, halal, etc.)
Formulation of an entirely fraudulent product (using multiple adulterants and methods)
- The sale of “100% apple juice” that consists of sugar, water, malic acid, flavor, and color
Other
- This includes counterfeits, theft, overruns, etc.

Harmonization of food fraud terminology is frequently discussed, so I thought it might be useful to provide information on how our definitions relate to the GFSI terminology:

GFSI category “Dilution”: This category maps directly to our category dilution/substitution. The reason we combine these into one category is that the intent is the same: To replace the weight or volume of a product. This can occur either through partial or full substitution of a liquid product, a granulated product, or swapping an entire intact product such as a fish filet. One of the GFSI examples for substitution is “sunflower oil partially substituted with mineral oil”, which could just as accurately be described as dilution.

GFSI category “Substitution”: As noted above, this category maps directly to our category dilution/substitution. However, we would not consider the use of hydrolyzed leather protein in milk (one of the cited examples) to be dilution/substitution because it is not used to replace weight or volume. We would view that as artificial enhancement of the protein content of milk.

GFSI category “Concealment”: We do not include a category focused on concealment because all food fraud involves concealing some aspect of the true contents of the food. One of the examples cited in this category is “poultry injected with hormones to conceal disease.” The use of antibiotics, anti-fungal agents or other substances to reduce bacterial load or mask deterioration would be classified, in our system, as the use of undeclared, unapproved or banned biocides. The use of coloring agents on fruit to improve appearance would also be classified as artificial enhancement.

GFSI category “Mislabeling”: Since all food fraud is, to some extent, mislabeling, we reserve the use of the term fraudulent labeling claims to those label attributes that describe production processes (organic, kosher, etc.). With the exception of falsification of expiration dates, the other examples cited would not be classified by us as mislabeling. The sale of Japanese star anise, which is potentially toxic, as Chinese star anise (a different species) is dilution/substitution and a health risk to consumers. The sale of cooking oil that has been recovered from waste streams and illegally produced is also a form of substitution that poses a potential health risk to consumers.

GFSI category “Unapproved enhancements”: This GFSI category aligns nicely with our category artificial enhancement, and both examples cited are nicely illustrative of the concept, which involves the fraudulent addition of a substance specifically for its function (not as a replacement for weight or volume).

GFSI Category “Gray market production/theft/diversion”: The production and sale of food products through unregulated channels would all be classified in our category called other. Because these forms of food fraud involve the sale of food outside of regulatory control, prevention measures will generally be substantially different from the prevention of fraud within legitimate supply chains.

GFSI Category Counterfeiting: This GFSI category is similar to the gray market production/theft/diversion category in that it involves intellectual property infringement and production outside of regulatory control. It would similarly be classified in our other category.

February 8, 2019

How to Strengthen Your Audit Score

By Shawna Wagner

It is one week before your scheduled audit. Are you running around from department to department crossing the T’s and dotting the I’s, or are you cool, calm and collected, as it is just another audit? More than likely you have been in both cases, but how does one finally get to the collected stage? It is not about being nervous—even as a lead auditor I get nervous (I am human, too)— it is about being ready!

Believe it or not, I can generally gage if the audit is going to flow well within the first hour of arrival. My questions from audit to audit of course change, but overall, I am asking for the same materials and results. If the team is getting baffled by questions in the beginning stages of the audit, I know I am in for a long ride. Why does this happen? If the situation has gone down hill rather quickly, I find that whomever at the plant is responsible for the implementation of the audit standard, has never taken the time to read or review it. That, my friends, is one of the most critical mistakes you can make. I know these standards are not world-class bestselling books that you are dying to binge read by the fireplace with a glass of wine—if this was your view, you would be an auditor already. HA! The standards are not always black and white as far as requirements, but just taking the time to read the standard will enhance your understanding of where the auditor is coming from.

Now you have read the standard, you understand it and are comfortable. You have a 200,000 square-foot facility, 200 employees and four different HACCP plans to watch over. How could you possibly score an excellent, “A”, pass, whatever the top score may be defined as? I am sure you have heard the expression, it is not all about you, and that holds true for audits. So many times I have witnessed that the quality manager (or equivalent) try to undertake the entire audit from start to finish and answer all the questions. This is not a best-practice approach. A good example here would be that many audits now talk about annual objectives and key performance indicators. As an auditor, I expect that your plant manager (or equivalent key person responsible) come talk about these with their senior management. Too often I see the quality department commit to answering these questions, but as an auditor, I am interested in your system and how it works throughout your entire facility. I do not just want to know about the quality department. Get all departments involved. They, too, then get firsthand experience of understanding what the audit requirements are, instead of just reading endless emails of what needs to be accomplished.

Teamwork and understanding are now established—great! How strong are your programs? There are a few programs I want to touch on here, including your internal audit program and procedures. I can not write this article and tell you “X” is the solution, as there is not a one-size-fits-all approach. If you are spending two hours on an annual internal audit for a facility like the one mentioned above, it is possible that your program is lacking robustness. When conducting an internal audit and a requirement is in compliance, do you write what made it in compliance, what you reviewed, and with whom you spoke? This is a great way for you to compare what you see in your plant versus what an auditor sees in your plant, especially if you built your internal audit from an audit standard. How about your sanitation program? I came from a background of meat inspection, so maybe I am a bit biased, but your sanitation is encompassed with other pre-requisite programs in the facility. If your facility has untidy locker rooms, cafeterias and bathrooms, what motivates the employees to be tidy on the production floor? Sometimes it is the small things that go a very long way. In talking about sanitation, I have observed that many site managers never go in to watch your pre-operational inspections that may take place very early in the morning. Just because it is not on first shift, does not mean it is not important, and it also does not mean that it will not be audited. Likely, it will be audited!

Lastly, take all nonconformities that you have found either through internal or external audits, respond to them in a timely fashion, and get the root causes and preventative actions in place! Nothing pains me more then to see sites repeat a nonconformance, just to possibly get a major conformity. It’s like shooting yourself in the foot and your audit score takes an unnecessary plunge that could have been avoided.

Audits are not a fad and are not going away. It does take way more time to get to the cool, calm and collected stage for audits, but in the end, being audit ready all the time looks good on you and your site!

February 6, 2019