Everyone knows a “checklist” when they see it: a systematized tool that lists things, components, steps or criteria whose presence or quantitative amount are essential to the performance of a specific task. The order of items in a checklist may or may not be critical in terms of the sequence of tasks which need to occur.

Checklists serve the wonderful purpose of identifying the important and critical steps needed to manufacture fresh food, fly a plane, perform a surgery or run a nuclear plant. They serve the purpose of helping to make sure that no important step is forgotten, and all critical steps are performed in the right order. Having a high “checklist intelligence” means that checklists are used proactively and dynamically, and that they drive continuous improvement in practices and procedures. And this occurs regardless of personal or organizational biases.

Let’s review some of the published literature on checklists.

A popular book on checklists is “The Checklist Manifesto” (Picador, 2009) by surgeon Atul Gawande. As noted in the book, checklists are very useful when there is a lot to get right, that is, when there is a high degree of complexity for certain actions. For example, commercial airplanes have “become too much airplane for one person to fly.” Hence, the industry uses a number of checklists, especially pre-flight, to address possible risks before the airplane takes to the air. Food manufacturing is similar, especially given its impact on public health.

A distinct disadvantage is that checklists can drive a tyranny of the urgent, i.e., simply checking a box to be done with it.

A distinct advantage of checklists is that they can be built by the “wisdom of a group” of experienced people, and therefore do not rely solely on a single individual’s memory or experience base. A distinct disadvantage is that checklists can drive a tyranny of the urgent, i.e., simply checking a box to be done with it.

Western Michigan University (2017) has an Evaluation Checklist Project with a number of excellent resources showing how to develop checklists for evaluating programs and projects. These include a “checklist for formatting checklists” and scholarly presentations on the logic behind checklists. Their suggestions can easily be re-applied to the food industry.

There are many published articles which address bias in decision making. For example, Ely et al. (2011)[1] studied the use of checklists to reduce diagnostic errors in hospitals, clinics and emergency rooms. Of note, the authors delve into cognitive processes to identify the inherent biases and reliance on intuition that often drive decision-making. They remind checklist developers to take into account “Type 1” thinking processes which are fast, reflexive and intuitive (and usually subconscious) and “Type 2” processes which are analytic, slow and deliberate (and usually take very focused attention).

Application of Checklists in the Food Industry

Checklists are widely used in the food industry. The USDA (2014) has a label submission checklist that helps companies avoid common labeling mistakes, and clarifies what is needed. The agency also has a guideline checklist for the cooking of meat and poultry products.

In 2020, FDA published an Employee Health and Food Safety Checklist in response to the COVID-19 pandemic. In 2001, the agency developed the Allergy Inspection Guide, a checklist for inspection of food companies which manufacture products potentially susceptible to contamination by allergenic ingredients, and now has a draft guidance/checklist for evaluating the public health importance of allergens.

In the food manufacturing setting, companies often conduct their FSMA-related GMP audits by having employees walk around the plant using a checklist of equipment, documents and practices to look for. Companies making prepared foods have checklists that operators must follow to ensure that proper cooking and cooling procedures have been followed (these are also called SOPs, or Standard Operating Procedures, which are, in essence, checklists). Similarly, sanitation teams follow strict SOPs/checklists to ensure the right sanitizers are used in the right concentrations and for the right durations. Line changeovers often use checklists to prevent allergen cross-contamination. The same is true for pre-production equipment assembly. And product development/chef teams use checklists to ensure that the right ingredients are used, with proper consideration given to allergens, glutens, GMOs and organic product needs.

Finally, some of the most widely used checklists in the food industry are standards, including those developed by the Global Food Safety Initiative (GFSI), Safe Quality Foods (SQF) and the British Retail Consortium Global Standard (BRCGS).

Checklists as an Indicator of Food Safety Audit Maturity

Companies going through the GFSI certification process (e.g., SQF) often follow a three-phase audit maturation process that highlights how checklists can help or hinder food safty. In the first phase, the company is new to the process and therefore may not have systems in place to handle all of the requirements inherent to the standard. Thus, the company may “shotgun” their approach based on where they think they have gaps (by their own evaluation and/or with the help of third-party consultants). In this phase, the SQF Code may not be looked at in its totality nor in its intent, and certain requirements may be looked at as more important than others (with the insidious side effect of some requirements being missed).

In the second phase of the growth curve, the company recognizes that the food safety requirements are laid out in a very organized and helpful manner: the SQF Code. They realize that if they can match each requirement in the code with practices and procedures, then they can essentially use the code as a checklist. Many companies in this phase build their programs and their audit readiness exactly in the order of the code, and solely to meet the specific requirements detailed in each section of the Code. This ensures that when the SQF auditor comes in, the company will have addressed each and every requirement. This approach serves those companies well who are still in the learning phase of building a strong food safety plan and food safety culture, and generally helps most companies “pass” their food safety audit.

The right culture drives the right entries on the checklists. Not the other way around.

All is well until there is a food safety incident, trade withdrawal or public recall, which can happen in spite of a company checking every box on the SQF “checklist.” A major negative event, or even the recognition that such an event could happen, can therefore rightly push a company into the third phase of using the SQF Code.
In the third phase, a company uses and views checklists as valuable tools (and likely still structures its audit readiness in the same order as the SQF Code). However, the company has critically realized that it needs to go beyond checklists to drive the right food safety culture in the organization. Additional practices, procedures, documentation and systems are put in place to drive the right culture. These in turn make sure that the checklists get checked. Said another way, the right culture drives the right entries on the checklists. Not the other way around.

A Checklist for Checklists

Let’s consider creating a checklist for checklists. Each of the following provides perspective on the value, and the warnings, of using checklists to drive and improve an organization’s food safety culture and therefore its “checklist intelligence.”

Checklists Can Speak the Truth. If the results from a completed checklist are pointing out significant issues, then at the basic level the checklist is working. This is not a time to alter the checklist, which can happen in low-maturity organizations as a way to hide an issue, or an excuse to fill out the checklist incompletely. Rather, complete results should be heralded as validation that the checklist is performing as it should.

Learn From Failures. Something going wrong despite the use of a checklist is a good clue that the wrong things are being checked or that something is missing. This should be discussed broadly and cross-functionally and drive a root-cause analysis, which can markedly point out what got missed, which in turn allows for continual improvement of the checklist.

There Is No “One-Off.” All experienced auditors have heard “this is the first time that this has happened.” Or “there are many unusual things going on at the same time, and this caused the issue; it won’t happen again.” There is no one-off! A root cause analysis should be performed. Checklists must be able to help the organization identify and diagnose root causes.

Check the Checker. Is the person filling out the checklist being driven by the checklist to look for the right food safety behaviors, or is the person merely checking the boxes since that’s the job? Perhaps more insidious, employees might follow a checklist quite diligently—observing just those tasks which are on the list—yet miss faulty or risky behaviors. This may not be the fault of the checklist, but it is certainly the fault of the organization and its training. Relying solely on a checklist can still allow egregious and unwanted behaviors. If the employees are trained only to follow the checklist and make sure it gets filled out, significant untoward behaviors get missed. In this regard, checklists become shackles.

Check the Documents. Critical to some checklists are documents which are meant to substantiate that a particular task on the checklist was taken care of. The utility of these documents is only as good as the value of having them on the checklist to begin with. Time must be taken to identify which procedures or cooking logs, for example, need to be checked as part of a checklist. This is independent of having these documented as part of the organization’s food safety plan.

Honor the System. Checklists are just that: lists. They are not roadmaps, graphs or linkages to knowledge bases. They are static, rather than dynamic systems that drive action and resolution of issues. In general, checklists can be ill-equipped to capture systemic behaviors and the culture of an organization. This is especially true if the checklists are from a third-party and/or have not been adapted to specific organizations and facilities. Hence, checklists should be used for what they can bring—no more, no less.

The Law of Unintended Consequences. An oft-quoted phrase is “you get what you measure.” And this is certainly in play for checklists. If the item on the checklist is wrong, or is directing the wrong behavior, measuring it regularly could serve the unwanted purpose of instilling that behavior as “correct.”

Defeating the Checklist

By now you realize that checklists in the food industry can serve as a crutch or as a divining-rod for continuous improvement of food safety practices and procedures. Following are some indicators that a checklist is not working or is not as effective as it should be.

Too burdensome. A very common checklist used in food manufacturing is the “GMP audit checklist.” This is typically a long list of behaviors and practices which the organization believes it should be engaging in to meet the GMP regulations and produce safe food. Most organizations commit to conducting such audit checklists as part of their promises to the auditing organization. The list gets longer and gets spread across more functions, and all of this work becomes quite burdensome. When it is time for the GFSI audit, missing or incomplete checklists may get pencil-whipped, leading the auditor to believe that the company has been using the checklists regularly.

Pencil-Whipping. As much as putting false entries on a form is unethical, and usually illegal, it can still occur under the right stressors or employee attitudes. Simply checking the boxes on a checklist does no one any favors and can provide a false sense of security.

Complacency. Organizations that rely on the data from checklists could develop a false sense of security and become complacent about corrective actions. Although not necessarily unethical or illegal, someone checking a box as “complete” just because it always has been in the past is misleading (if not outright wrong) if the checker really did not check. Understanding this risk can help define the items in the checklist, including those things needed to ensure that the checklist checker is focused and paying attention.

Pressure to “get back to work” can be one of the quickest means to defeat a checklist.

Inaccurate documents. Practices and procedures change over time, and often the documents that go along with them do not get updated on the production floor. Continual vigilance is needed to ensure that the most up to date documents are aligned with current practices and the details on the checklist. In fact, one of the items on a checklist might be checking the issue dates of key documents being used by operators.

Stress. Pressure to “get back to work” can be one of the quickest means to defeat a checklist. This could be due to senior management’s communications, a team’s own leadership or individuals believing they need to hurry up so that they can resume their “real job.”

The End-Game: A Game of Checkers

To win at the game of checkers (or draughts), there are a number of strategies which experts often espouse, most of which apply to checklists in the food industry.

Control the center: Focus on the stuff that counts, not the stuff on the edges.

Play offense, not defense: Attack the issues that strive to undermine the food safety program.

The goal is to get to the end of the board: The checklist must be completed in its entirety.

Be willing to sacrifice: If an item on the checklist is not working, take it off.

Advance as a group: Don’t just leave checklists to one group (e.g., QA); build and use them based on input from experts from all functions.

Realizing the value of checklists requires the right culture, rules and execution as well as recognition that checklists are tools to maximize risk identification and risk management. Building your organization’s “checklist intelligence” will help in the development of the checklists, the effectiveness of those checking the checklists and in increasing the assurance of those checking the checkers.

The game never ends, which means that with the right strategy you can win all the time.

References:

[1] Ely, John, Graber, Mark, and Croskeey, Pat (2011): Checklists to Reduce Diagnostic Errors, Academic Medicine, 86:307.