Visual Inspection Reference Samples: Check Inspector Agreement

A visual reference sample needs an approved classification, a defined use condition and a traceable identity. Before releasing the set, check whether inspectors classify the same examples consistently with themselves, with one another and with the approved reference decisions.

An experienced operator notices a defect that others miss. Preserve the observation, then ask quality to establish the acceptance rule. A photograph or a part on a shelf becomes useful only when the next inspector knows what it demonstrates and where that judgement applies.

Define the feature and the approved reference

The quality engineer owns the sample set. Use this workflow when visual ratings differ, a new cosmetic requirement arrives or an inspection aid changes. Start with one product family and one characteristic. Bring the current customer or engineering criteria, known sample identities, the viewing conditions and the person authorised to decide the reference classification.

Include clear acceptable and unacceptable examples and, where approved, examples close to the decision boundary. Do not let a production team's majority vote establish the standard. If the responsible technical authority cannot classify a sample, record that uncertainty and resolve it before using the sample to judge inspectors.

Compare repeated ratings without revealing the answer

Minitab's attribute agreement guidance distinguishes consistency within an appraiser, between appraisers and against a known standard. These are separate questions. Two inspectors can agree and both be wrong.

For a planned study, quality defines the sample selection, repeats, order, conditions and acceptance criteria before testing. Keep the reference answers hidden during the exercise and vary presentation order so remembering a sequence does not replace inspection. Preserve every original rating rather than editing the data after discussion.

Illustrative arithmetic only: three inspectors each rate ten examples twice, producing 3 × 10 × 2 = 60 judgements. If 54 match the approved reference, the overall observed agreement is 90%. That single figure does not establish that each inspector is consistent, and this small example is not a qualification plan.

Suppose all six mismatches concern one near-boundary sample. Review that sample, its approved rule and the viewing conditions before concluding that all three people need retraining. If one inspector changes a rating between repeats, record the within-person disagreement separately. Quality should investigate false acceptance as well as false rejection; their consequences can differ.

Resolve disagreement without weakening the requirement

Use the evidence to decide whether the wording, sample, conditions or training need correction. Do not relax an engineering requirement to make the agreement score improve. If a feature requires a specified measurement, a visual aid remains supplementary; it cannot replace the required method.

A scratched or faded reference can change what people learn. Withdraw it visibly, identify any copies and review inspections potentially affected under the normal quality process. When the approved requirement changes, link the replacement set to the previous version and keep the reason for withdrawal.

Release a usable inspection aid

Before release, every sample must have an authorised classification, readable identity, permitted scope and known custodian. Complete the agreed study and resolve critical disagreements. Ask an inspector to locate the current set and explain the action for an uncertain part. The expected response is the defined hold or escalation path, not a forced guess.

Our Optiwise quality workflow supports configurable inspection forms, parameters, form versions and role-based quality tasks. Link the reference-set identity and revision to the inspection record. This keeps the approved aid connected to the work without treating a software record as technical approval.

Reference-sample questions

Can photographs replace physical samples?

Only when the responsible quality authority accepts that method for the characteristic. Lighting, scale, display and surface appearance can change what a photograph communicates.

Does 90% agreement mean the method is acceptable?

Not by itself. Use the predefined study criteria, error types, sample design and risk of the characteristic. The example above demonstrates counting, not an acceptance threshold.

Bring one controlled inspection aid to a demo and map its revision and result records with us.