FST Soapbox

Effective Testing: Developing Rigorous, Reliable and Relatable Questions

By Ibidun Layi-Ojo
No Comments
April 24, 2018


• Categories: Food Manufacturing, FSMA, FST Soapbox

Success Factor 2: Develop rigorous, reliable and relatable exam questions (items) that are developed, tested and continuously evaluated to correlate with market needs and trends.

My previous column in Food Safety Tech outlined the single most important factor in ensuring that all employees have the proven ability to keep the public safe from foodborne illness: Education. Only rigorous, continually evaluated exams, designed for a company’s particular industry segment, can give employers the assurance that employees have the skills they need to make food safe.

Constructing and administering those exams starts with partnering with the right food safety assessment provider. Once that provider has been chosen, the next step is to develop questions—and ultimately an exam that exemplifies the three R’s: Rigorous, reliable and relatable.

Rigorousness begins with the process by which questions are created. This process must be a step-by-step effort to ensure that the final exam asks the right questions, based on the industry segment and the skills needed to be measured, and that the questions meet or exceed current industry standards. The ultimate aim is to give employees the opportunity to demonstrate their knowledge, which results in a sense of empowerment that makes them effective stewards of food safety every day.

To meet these goals, a company must work closely with its food safety assessment provider throughout the test development process, which begins with an analysis of the job (or jobs) for which the exam is being created (i.e., what are employees’ important tasks for which performance must be measured?). This analysis informs the development of precise specifications for the exam, and with those specifications established, the food safety assessment provider can begin collaborations with subject matter experts to formulate questions for the exam. Every question on the exam should dovetail with needs and trends in the marketplace, with emphasis on the client’s position in the marketplace.

The next step in the process—item review (question validation) —is key to making sure the exam is comprehensive. In effect, this is a ‘test of the test’ and should address the following:

• Does the exam ask all the right questions?
• Are the questions free of ambiguity that could lead to an inaccurate measurement of knowledge?
• Are the questions in line with current industry standards?

Once every question has been subjected to validation, a passing score for the exam is set concurrent with best practice guidelines for making scoring decisions. Next the food safety assessment provider and the client collaborate on the best way to administer the examination (e.g., whether on paper or online, taken at work or home).
Only then is the test ready to be given, scored and analyzed.

It might seem, at this point, that the exam-creation cycle has been completed. On the contrary, the cycle must be a continuous process, with results from the initial test administration serving as a baseline for ongoing test maintenance and fine-tuning.

This continuity is critical, because standards and practices for food safety are always evolving. FSMA gave the FDA broad authority to prevent contamination of food in every step of the supply chain. In the seven years since then, regulations at the federal, state and local levels have been constantly amended and updated across the entire spectrum of the food industry, from growers, manufacturers and processors to grocers, retailers and even culinary schools. Only ongoing test maintenance—including the development and validation of new test items—can ensure that exams stay in lockstep with the FDA food code and safety guidelines.

Exam questions also must be aligned with the accreditation guidelines of the American National Standards Institute (ANSI), the private, nonprofit organization that, since 1918, has been the overseer of U.S. standards for consumer protection.

Developing and maintaining accurate, reliable food-safety exam content is complex and challenging, requiring a commitment to continuous validation and “testing of the test” to meet the needs of the marketplace and the requirements of federal, state and local regulators. Partnering with the right food safety assessment provider is crucial in meeting those needs and requirements, protecting the public, and ensuring a company’s reputation for providing safe, wholesome food.

Look for Part 3 of this series to learn more about how to create food safety exams that factor in a best-practices approach to properly assess the workforce.