I wrote many times that QC inspections before shipment are a good way of keeping quality problems where they belong: in the factory.
It is a good tool. But pointing to its limits is important. Misinterpreting its reliability is very easy.
Here is an example. Some buyers think along these lines:
I followed what the ISO 2859-1 standard advises by default: normal severity, level II.
I placed an upper limit (AQL) of 1% on major defects.
The inspector picked a number of cartons that is considered sufficient in the QC industry (at least the square root of the total number of cartons), and then picked some samples randomly inside these cartons.
100% of cartons were packed and were nicely lined up, so the cartons were picked in good conditions and it was possible to check that the whole quantity was presented.
Therefore, I can be confident that neither I, nor my customers, will ever find more than 1% of defects in that batch.
Unfortunately, that conclusion is DEAD WRONG for several reasons.
Reason 1: the findings of a random inspection are not 100% accurate
Page 38 of the ISO 2859-1 standard shows the consumer’s risk for normal inspections. If you set the AQL limit at 1%, and if the sample size is 200 pieces, you might still receive up to 4.59% of defective goods!
Why does the standard allow for such a difference between the objective and reality’s worst case? Because there is a risk that the sample be worse than the average of the batch. The producer’s risk of getting a batch rejected even though it should be accepted is capped at 5%. Obviously the standard is more favorable to the supplier than it is to the buyer.
Reason 2: statistics give a rosy picture… that doesn’t correspond to reality
The consumer risk is calculated based on one dubious assumption: data follow a hypergeometric distribution, and can be approximated with a binomial distribution. That, in itself, assumes that the defects are somewhat randomly spread in the batch.
In reality, there are often clusters of defective pieces. For example, the battery cover is good for the first 95% of a batch. But, for the last 5%, the manufacturer notices they are missing the right cover. So they use old covers that were put aside a few months before (because the color was off).
In this case, why doesn’t the distribution of defects follow a binomial law? Because placing the cover is the last step before packaging. these 5% of defective products will be packed in the same cartons (rather than being spread out here and there among all cartons). Let’s say, for simplicity, that 5% of the cartons are full of these defective goods.
How is the probability that some of these cartons (containing faulty covers) will be picked for inspection? Not very high.
[By the way, make sure the factory writes numbers on cartons before the inspection, and ask the inspector to write what cartons he pieked in his report. Making every party accountable is always a good idea.]
Reason 3: you, or your customers, will probably not follow the same sampling plan as the inspector in the factory
Let’s say you deliver 20,000 pieces to your customer’s DC (Distribution Center). Then they send 3,000 pcs to a regional DC. The regional DC opens a couple of cartons, for a quick inspection. And you are out of luck: one of these two cartons is full of defective covers. Ouch!
This example is not extreme. It is close to real-life situations that were described to me by several clients.
The problem is, the customer will tell you “the battery cover is unacceptable on all your products; we are sending this batch back to you.” And it’s hard to change a customer’s mind with a rational explanation, in such a situation.
Reason 4: human errors happen
Inspectors are not machines. Inattention and laziness, poor training, pressure (or more) from the manufacturer, all come into play and can significantly alter the reliability of findings.
I described this in How your inspectors can fail to notice quality problems.
Reason 5: the factory might play games after the inspector is gone
When off-the-shelf products are checked, the factory can easily replace the (good) products that were presented by a substandard batch. Or they might want to decrease slightly the quantity of pieces in each carton.
Who knows what they might do, if they are fundamentally dishonest and short-term thinkers?
If having 3% of defects in a batch can incur very high costs (likely to wipe your margin out and cause a loss in customer confidence), random inspections are NOT appropriate.
I described 100%, piece-by-piece, inspections before. As a buyer, I would feel much more confident having a team of inspectors working together, sorting defects out, and sealing cartons before they go.
If checking every piece is a bit too expensive, a dynamic sampling plan might make sense: start with a random sample, and then go up to 100% if a high proportion of defects is found (but look only for those types of defects).
Some buyers even require a 200% inspection when they MUST have a very low proportion of defects. It all depends on what you can afford to pay, and how much a few defects might cost your business.
In the long term, the best solution is simply to work with good manufacturers… Or to help them improve their quality.
PS: if you liked this article, you can subscribe so that you receive future articles by email. Just click here. Thanks!
Etienne Charlier says
Renaud, good post to bust some common mistaken views on sampling. It is all a probability game and this is something that most people misunderstand. Even when you are familiar with it it is sometimes hard to keep focused on all aspects. It is too easy to forget that sampling always leads to 2 numbers: in the MIL 105 case, an AQL (such as the 1% defect) AND a probability of still accepting a batch with exceeding defect rate. I have seen this probability called OC Curve sometimes.
The OC curve shows the “chances” of accepting a batch in function of the defect rate. The curve on this link below shows that although the chance goes down a lot around AQL,it is far from zero for defect rates up to 10%. But this is mainly due to the small batch size (200). If the batch size were 2000 the slope would be much steeper reducing the risk of accepting batches with high defect rates.
Batch 200: http://www.sqconline.com/inc/ocgif.php?m=50&c1=1&AQL=11
Batch 2000: http://www.sqconline.com/inc/ocgif.php?m=125&c1=3&AQL=11
Renaud Anjoran says
Yes, we share the same view on this. Thanks for adding to the article.
The fact is that random sampling is NOT very easy to understand, and its implications are not intuitive at all for the average buyer.