I wrote many times that QC inspections before shipment are a good way of keeping quality problems where they belong: in the factory.

It is a good tool. But pointing to its limits is important. Misinterpreting its reliability is very easy.

Here is an example. Some buyers think along these lines:

I followed what the ISO 2859-1 standard advises by default: normal severity, level II.
I placed an upper limit (AQL) of 1% on major defects.
The inspector picked a number of cartons that is considered sufficient in the QC industry (at least the square root of the total number of cartons), and then picked some samples randomly inside these cartons.
100% of cartons were packed and were nicely lined up, so the cartons were picked in good conditions and it was possible to check that the whole quantity was presented.
Therefore, I can be confident that neither I, nor my customers, will ever find more than 1% of defects in that batch.

Unfortunately, that conclusion is DEAD WRONG for several reasons.

Reason 1: the findings of a random inspection are not 100% accurate

Page 38 of the ISO 2859-1 standard shows the consumer’s risk for normal inspections. If you set the AQL limit at 1%, and if the sample size is 200 pieces, you might still receive up to 4.59% of defective goods!

Why does the standard allow for such a difference between the objective and reality’s worst case? Because there is a risk that the sample be worse than the average of the batch. The producer’s risk of getting a batch rejected even though it should be accepted is capped at 5%. Obviously the standard is more favorable to the supplier than it is to the buyer.

Reason 2: statistics give a rosy picture… that doesn’t correspond to reality

The consumer risk is calculated based on one dubious assumption: data follow a hypergeometric distribution, and can be approximated with a binomial distribution. That, in itself, assumes that the defects are somewhat randomly spread in the batch.

In reality, there are often clusters of defective pieces. For example, the battery cover is good for the first 95% of a batch. But, for the last 5%, the manufacturer notices they are missing the right cover. So they use old covers that were put aside a few months before (because the color was off).

In this case, why doesn’t the distribution of defects follow a binomial law? Because placing the cover is the last step before packaging. these 5% of defective products will be packed in the same cartons (rather than being spread out here and there among all cartons). Let’s say, for simplicity, that 5% of the cartons are full of these defective goods.

How is the probability that some of these cartons (containing faulty covers) will be picked for inspection? Not very high.

[By the way, make sure the factory writes numbers on cartons before the inspection, and ask the inspector to write what cartons he pieked in his report. Making every party accountable is always a good idea.]

Reason 3: you, or your customers, will probably not follow the same sampling plan as the inspector in the factory

Let’s say you deliver 20,000 pieces to your customer’s DC (Distribution Center). Then they send 3,000 pcs to a regional DC. The regional DC opens a couple of cartons, for a quick inspection. And you are out of luck: one of these two cartons is full of defective covers. Ouch!

This example is not extreme. It is close to real-life situations that were described to me by several clients.

The problem is, the customer will tell you “the battery cover is unacceptable on all your products; we are sending this batch back to you.” And it’s hard to change a customer’s mind with a rational explanation, in such a situation.

Reason 4: human errors happen

Inspectors are not machines. Inattention and laziness, poor training, pressure (or more) from the manufacturer, all come into play and can significantly alter the reliability of findings.

I described this in How your inspectors can fail to notice quality problems.

Reason 5: the factory might play games after the inspector is gone

When off-the-shelf products are checked, the factory can easily replace the (good) products that were presented by a substandard batch. Or they might want to decrease slightly the quantity of pieces in each carton.

Who knows what they might do, if they are fundamentally dishonest and short-term thinkers?

Conclusion

If having 3% of defects in a batch can incur very high costs (likely to wipe your margin out and cause a loss in customer confidence), random inspections are NOT appropriate.

I described 100%, piece-by-piece, inspections before. As a buyer, I would feel much more confident having a team of inspectors working together, sorting defects out, and sealing cartons before they go.

If checking every piece is a bit too expensive, a dynamic sampling plan might make sense: start with a random sample, and then go up to 100% if a high proportion of defects is found (but look only for those types of defects).

Some buyers even require a 200% inspection when they MUST have a very low proportion of defects. It all depends on what you can afford to pay, and how much a few defects might cost your business.

In the long term, the best solution is simply to work with good manufacturers… Or to help them improve their quality.

–

PS: if you liked this article, you can subscribe so that you receive future articles by email. Just click here. Thanks!