How to figure out what reliability testing you need?

Let’s explore how to figure out what reliability testing you need for your product, the questions reliability engineers will ask you, and the factors that influence the kinds of testing to be done and its cost.

Prefer listening to reading?

Listen to the audio here.

What is product reliability?

Product reliability is where the product lasts for an appropriate amount of time without breaking or developing any problems. For example, this may be its warranty period or a reasonable lifetime that customers would expect. (02:01)

Quality is not the same as reliability.

A lot of consumers believe that they have high-quality products, but they really mean that they’re high reliability because they value that it lasts and doesn’t break down. Quality is related to the product’s design and build…if the product is built exactly to specification then it can be considered to be ‘good quality,’ however, this doesn’t guarantee that it will be reliable. You can see evidence of this in some luxury cars, as they are good quality but are not always as well-known for reliability as, say a Toyota. This could be down to their engineering being exact, so as soon as something goes slightly out of tolerance, the car will start to suffer from problems and may break down or at least show a warning. (04:41)

What questions will a reliability engineer ask you in order to proceed?

What industry is the product in?
Different industries have very specific reliability needs, so it’s critical to know where it fits. For example, in the medical industry, there is zero tolerance for failures as patients may die, whereas in consumer devices no one will be harmed if there’s a problem. The reliability tests, equipment, and methodology may all differ wildly between industries, so your test engineer needs to understand your product’s niche. Some testing will take longer than others, too, so time is a consideration. For example, when testing medical devices, the engineer may have to spend some time with the patient and observe how they will use the device before any testing can be planned and done. There is also a cost element, as reliability testing is not always ‘cheap’ and the test equipment alone may cost millions of dollars, so that’s reflected in your costs, as well as the staff and other resources committed to the testing.
What kind of product is it?
Depending on the kind of product, we will need to find out which standards apply, the types of tests to be performed, the typical testing environment/s, and test equipment required. Then we can start creating the test plan and then do the tests. (07:06)

Defining your reliability needs for the product

Reliability needs will depend on the product, so let’s look at some examples to illustrate typical needs and tests. A mobile phone can be expected to survive drops from hand height, around 1 Meter, without the screen breaking, therefore a drop test will be needed. Medical devices must provide excellent safety for the patient, so connectivity is key and the patient’s room environment needs to be taken into account – the temperature may vary and the medical device, which is probably very expensive, could be at risk of being dropped by accident, both influencing the testing to be done. (14:29)

What if your reliability demands are either too extreme or not strict enough?

Many people who have a product idea don’t have experience in reliability testing so sometimes they feel that the reliability requirements must be extreme in order to give peace of mind that the product will be reliable. But by testing the product under too extreme an environment, for example, you may be guaranteeing reliability at too high a level to be realistic which also costs too much. For instance, testing a bottle at 200C is a very high temperature, double the boiling temperature for water. In fact, your bottle may only need to be tested to, say, 85C because it is not intended to hold boiling liquids at all, let alone liquid up to 200 degrees! A reliability engineer will consult with you, understand your needs, and may suggest where testing needs to be less stringent which will also probably save time and money. (16:54)

Examples of common reliability testing requirements for a typical consumer product (a smartwatch).

To get an idea of what kinds of testing requirements will be required for common consumer electronics, a typical niche for many importers and ourselves, here are some examples:

Functionality Testing: Do functions and features work as intended? This may be items like includes testing the display, buttons, connectivity, sensors, and GPS.
Battery Life Testing: Battery performance in different environments and use case scenarios are tested.
Durability & Reliability Tests: Is the device durable and likely to last as long as required? Drop test, vibration, water resistance, dust resistance, temperature and humidity testing, and more, will be done.
Environmental Testing: Accelerated life testing, high and low temperature and cycle testing, HALT, and HASS testing can be used to assess the watch’s performance under different rather extreme environments.

But aside from the tests, you need to understand why they’re being done so you can analyze their results and make a deduction about if the product reaches your reliability requirements. Any failures need to be explored and, if they’re serious enough, must be fixed.

Failures are not always a bad thing, either, as they can show that the product is durable and reliable enough if the failure has occurred when the product is subjected to stresses or environments that you consider to be far outside of an expected use case scenario. (18:45)

Reliability does have a cost, but is it a worthwhile investment?

You need to find a balance between doing too much or too little design for reliability and reliability testing as it can become costly. Over-engineer devices to be rugged and reliable and they may be blocky and unattractive as well as costing a lot to test to their limits and beyond. The components, too, need to be considered, as more durable components that resist shock, vibration, dust, etc, will cost more, such as gorilla glass for a display or rugged electronics. In terms of staff costs, experienced design engineers who understand how to design reliable products also won’t be cheap.

To illustrate why spending money on reliability is a good investment, there’s an example of a $40 screw that caused some debate in Congress in the USA. How could the US taxpayer be expected to pay $40 for a single screw…what an outrage! But this is an illustration of the cost of reliability because these screws were made for use in space. Therefore they had to be developed from scratch and be durable against very high and low heat, withstand radiation, not break under extreme stress, and so on. The screen testing would also require hundreds or even thousands of samples, as well as being very thorough, both of which are costly. What the Congressman didn’t take into account is that $40 for the screw is a small price to pay for the assurance that it will be reliable and won’t break resulting in the spacecraft being destroyed at massive cost and loss of life.

When considering your product, if it’s a consumer product especially, failure probably won’t lead to loss of life, but it can lead to returns, bad reviews, reputational damage, and a loss of business. So the investment into quality is still worthwhile. (22:11)

For many consumer products, focus on your product being at least reliable enough to last beyond its warranty period, but without over-designing and testing it to the point of overkill.

By understanding your product’s industry or niche, type, and the reliability level your customers will expect, you can design and test the product to be reliable enough. You do not want to receive product returns under warranty and have to pay out for repairs or replacements, that’s very expensive, so for many consumer products, they need to last the lifetime of the warranty at least, but without you making them so rugged that the cost of doing so becomes prohibitive. (28:45)

Don't miss a post