It is an exciting time to be in the food industry. Consumers are ever more aware of what they are eating and more demanding of quality. And the vital need to reduce global food waste is transforming how we produce and consume food. This is driving innovation all the way along the supply chain, from gate to plate.

One of the biggest areas of opportunity for the industry to increase automation and improve food safety is in the processing plant. The challenges processors have faced in the last 12 months have accelerated the focus on optimizing resources and the drive for more adoption of new technology.

Foreign material contamination is a growing issue in the meat industry and new types of detection systems are emerging to help address this challenge. As Casey Gallimore, director of regulatory and scientific affairs at the North American Meat Institute, highlighted in a recent webinar, 2019 was a record year for the number of recalls related to foreign object contamination, which totaled 27% of all FSIS recalls in that year.

“There are a number of potential reasons why recalls due to foreign object contamination have increased over the years: Greater regulatory focus, more discerning consumers, [and] more automation in plants. But one important reason for this trend is that we have a lot of new technology to help detect more, [but] we are not necessarily using it to its full potential,” said Gallimore. “As an industry, we have a strong track record of working together to provide industry-wide solutions to industry-wide problems. And I believe that education is key to understanding how different detection systems—often used together—can increase the safety and quality of our food.”

Types of Detection Systems

Processors use many different detection systems to find foreign materials in their products. Equipment such as x-rays and metal detectors, which have been used for many years, are not effective against many of today’s contaminants: Plastics, rubber, cardboard and glass. And even the most well trained inspectors are affected by fatigue, distraction, discomfort and many other factors. A multi-hurdle approach is imperative, and new technologies like vision systems need to be considered.

Vision systems, such as cameras, multi-spectral, and hyperspectral imaging systems can find objects, such as low-density plastics, that may have been missed by other detection methods. Yet, depending on the system, their performance and capabilities can vary widely.

Camera-based systems are the most similar to the human eye. These systems are good for distinguishing objects of varying size and shape, albeit in two-dimensions rather than three. But they become less effective in situations with low contrast between the background and the object being detected. Clear plastics are a good example of this.

Multi-spectral systems are able to see more colors, including wavelengths outside of the visible spectrum. However, multispectral systems are set up to use only specific wavelengths, which are selected based on the materials that the system is expected to detect. That means that multispectral systems can identify some chemical as well as visual properties of materials, based on those specific wavelengths. It also means that other materials, which the system has not been designed to find, will likely not be detected by a multispectral system.

Another relatively new type of vision system uses hyperspectral imaging. These systems use chemistry to detect differences in the materials being inspected and therefore recognize a broad range of different contaminants. They are especially good at seeing objects that cameras or human inspectors may miss and at identifying the specific contaminant that’s been detected. The same system can assess quality metrics such as composition and identify product flaws such as woody breast in chicken. Hyperspectral systems also gather tremendous amounts of chemistry data about the products they are monitoring and can use artificial intelligence and machine learning to get a more holistic picture of what is happening in the plant over time, and how to prevent future contamination issues. This might include identifying issues with a specific supplier, training or other process challenges on one line (or in one shift), or machinery in the plant that is causing ongoing contamination problems.

Many processors are considering implementing new inspection systems, and are struggling to understand how to compare the expected performance of different systems. One relatively simple methodology that can be used to evaluate system performance is, despite its simplicity, called a “Confusion Matrix”.

The Confusion Matrix

A confusion matrix is often used in machine learning. It compares the expected outcome of an event with the actual outcome in order to understand the reliability of a test.

Figure 1 shows four possible outcomes for any kind of test.

	Actual (True Condition)
Predicted (Measured Outcome)		Positive (P)	Negative (N)
	Positive Detection	True Positives (TP)	False Positives (FP)
	Negative Detection	False Negatives (FN)	True Negatives (TN)
		P = TP + FN	N = FP + TN
Figure 1. Confusion Matrix

But what does a confusion matrix tell us, and how can it help us assess a detection system?

The matrix shows us that a detection system may incorrectly register a positive or negative detection event—known as a ‘False Positive’ or ‘False Negative’.

As an example, say we are testing for a disease such as COVID-19. We want to know how often our system will give us a True Positive (detecting COVID when it *IS* present) versus a False Positive (detecting COVID when it *IS NOT* present).

Let’s apply this to processing. If you are using an x-ray to detect foreign objects, a small piece of plastic or wood would pass through unnoticed. This is a False Negative. By contrast, a system that uses hyperspectral imaging would easily identify that same piece of plastic or wood, because it has a different chemical signature from the product you’re processing. This is a True Positive.

A high rate of false negatives—failing to identify existing foreign materials—can mean contaminated product ends up in the hands of consumers.

The other side of the coin is false positives, meaning that the detector believes foreign material to be present when in fact it is not. A high rate of False Positives can lead to significant and unnecessary product wastage, or in time lost investigating an incident that didn’t actually occur (see Figure 2).

The secret to a good detection system lies in carefully balancing the rates of true positives and false positives by adjusting the sensitivity of a system.

This is where testing comes in. By adjusting a system and testing under different conditions, and then plotting these outcomes on the confusion matrix, you get an accurate picture of the system’s performance.

Effectiveness of a Detector

Detection is not just the act of seeing. It is the act of making a decision based on what you have seen, by understanding whether something of importance has occurred. Many factors influence the effectiveness of any detection system.

Resolution. This is the smallest size of object that can possibly be detected. For example, when you look at a photograph, the resolution affects how closely you can zoom in on an image before it becomes blurry.

Signal to noise ratio. This measures the electronic “noise” of the detector and compares it with the “background noise” that may interfere with the signals received by the detector. Too much background noise makes it harder to identify a foreign object.

Speed of acquisition. This measures how fast the detector can process the signals it receives. Motion limits what you can see. As line speeds increase, this impacts what detectors are able to pick up.

Material being detected. The type of material being detected and its properties will have a significant impact on the likelihood of detection. As previously mentioned, for example, x-rays are unlikely to detect low-density materials such as cardboard, resulting in a high number of False Negatives.

Presentation or location of material being detected. Materials that are underneath another object, that are presented on an angle, are too similar to the product being inspected, or are partially obstructed may be more difficult for some detectors to find. This also presents a risk of False Negatives.

Complexity of the product under inspection. Product composition and appearance vary. For example, just like the human eye, finding a small object on a uniformly illuminated and uniform color background like a white kitchen floor is much easier than finding the same small object on a complex background like industrial carpet. Coarsely ground meat might be more difficult to detect than uniform back fat layers, for example.

Environment. Conditions such as temperature and humidity will have a significant effect on detection.

Detection Curves

To understand system performance even better, we can use a detection curve. This plots out the likelihood of detection against different variables (e.g., object size) and allows us to objectively compare how these different factors impact the performance of each system.

Figure 3 shows how this looks when plotted as a curve, with object size on the x-axis (horizontal) and the probability of detection (a True Positive from the Confusion Matrix) on the y-axis (vertical). It shows three examples of possible detection curves, depending on the detector being used.

A detection curve tells you both the smallest and largest object that a detector will find and the probability that it will be found.

In the example presented by Figure 3, Detector 3 can see essentially 100% of large and very large objects, as can Detector 2. But Detector 3 is also more likely than the other two systems in the example to see microscopic objects. Based on this detection curve it would likely be the best option if the goal were to detect as many foreign objects as possible, of all sizes.

Of course, the performance of a detector is determined by multiple measures, not just size,

Detection capability can be improved for most detection systems, but typically comes at a significant cost: Increasing sensitivity will increase the number of false positives, resulting in increased product rejection. This is why looking at the detection curve together with the false-positive/false-negative rates for any detection system gives us a clear picture of its performance and is invaluable for food processing plants when selecting a system.

Using the confusion matrix and a detection curve, processors can compare different detection systems on an apples-to-apples basis. They can easily see whether a system can identify small, tiny or microscopic objects and, crucially, how often it will identify them.

Every detection method—X -ray, metal detection, vision systems, manual inspection—presents a trade-off between actual (correct) detection, rejection of good product (false positive) and missed detections (false negative). This simple way to compare differences means processors can make the right decision for the specific needs of their plant, based on easily gathered information. For all of us data geeks out there, that sounds like the Holy Grail.