Renaud is joined by Andrew Amirnovin, Sofeast’s product reliability head, and they’re looking at how Apple tests iPhone reliability and durability. Recently Marques Brownlee shared some videos from within Apple’s testing lab and interviews with their head of hardware engineering and the insights were interesting. It turns out that Andrew also visited Apple’s reliability test lab for iPhones (years ago) and he shares his own insights about that, and the pair discuss what you can learn from the way Apple do things.
If you prefer listening to reading…
The advent of the iPhone in 2006.
Around 2006 when NOKIA suffered from some legal issues in California, Apple hired many of its design and engineering team to work on the iPhone 1. Andrew at this time also left NOKIA but joined AT&T instead. During his time there, he developed a special mobile device testing procedure named “DIRNT” that was similar to HALT testing (vibration testing, etc) but without fast temperature changes or requiring a specialist and expensive machine. After this procedure was developed all of AT&T’s suppliers had to adhere to the requirements, and in the videos recently shot in Apple’s lab it looks like they still use similar test procedures to this day.
The tests are done during the development and validation of a new phone to ensure that it will be reliable and can withstand being splashed with water of the vibrations it may encounter in, say, a moving car. Back in ’07 Apple was a newcomer to the phone market and AT&T exclusively distributed the iPhone 1, so, unlike today, Apple was told what kind of testing needed to be done on the iPhone by the carrier in order for it to be distributed and seen as reliable, and Andrew’s tests were a part of that. (01:08)
Worries about reliability: What did Andrew learn at their testing lab?
Steve Jobs insisted that the iPhone 1 had a large glass display which was a first in the industry and made it a real risk of being unreliable and being complained about by customers. AT&T worried that their ‘flagship product’ could hurt their business if it didn’t meet their minimum requirements, and Apple’s reliability team would have been nervous about the new phone failing, too.
Unlike RIM, the makers of the Blackberry, who pushed back against AT&T’s new reliability testing requirements, Apple agreed with it, took it on board, and invested over half a million dollars into the right test equipment in their lab for testing iPhone 1’s reliability. Eventually, RIM came around, too, and they found the test procedures so effective at ensuring quality that they wanted to use it for all products going to any other carriers, too (which the boss of AT&T at the time was not that happy about). (08:48)
What can we learn from the tests we see in Brownlee’s videos from Apple’s labs?
We see Apple using a pretty standard water IP test machine to spray the iPhone in different ways, and this is normal for any products requiring about IP64 or 65 water and dustproof ratings up to the point where they need to be submerged underwater where a different machine is required. We see raindrops, splashes, and high-pressure jets of water being used. There is a really large hose spraying the phone, too, maybe a fire hose. That could be a test to failure to see what happens.
The most fragile part of an iPhone, the glass screen, isn’t focused on much and this may be indicative that users have been conditioned to normalize that weakness, as Apple and other brands know that most users will place the phone in a protective case and add screen protectors etc as soon as they purchase a phone. It seems that Apple has been more focused on features like battery life and weight of late which are no doubt important for users. (13:40)
What kind of drop test were they doing?
There was a robot arm dropping the phone at strange angles from a relatively low height. This may be similar to a test developed by Motorola in the past which simulated the phone being dropped from around a foot or so onto a countertop to test ruggedness. Andrew felt that Motorola’s test wasn’t that important, because if a phone was engineered to be reliable after drops from 1 meter, it was almost certainly going to be reliable after drops from, say, 10cm. There may be some value in this test, however, when it comes to seeing how internal components cope with repeated small drops and also that it simulates a user awkwardly dropping a phone from different angles (which is more realistic as drops are never intentional, and the phone tends to slip out of someone’s hand and then drop randomly).
A tumble test is likely a cheaper way to achieve the same results as this special robot, although Apple are not short of money… (22:00)
What do the different drop surfaces used mean?
We see different drop surfaces used by Apple for the iPhones, including slabs of marble, granite, and other surfaces. But why does it make a difference to use these?
Different surfaces are more forgiving than others. When he worked at LG Andrew tested dropping phones on linoleum, hardwood, steel, marble, concrete, asphalt, and others. Lino and wood lead to few failures, but once the phones started hitting steel and concrete the failures shot up as the screens would crack very often. Clearly, Apple follows the same kind of procedure here. Users can at least then be warned not to drop the phone on specific surfaces or from certain heights in the manuals (if they read them). (26:07)
Apple’s head of hardware engineering wants to make a product that ‘never fails.’ Realistic?
All reliability engineers have this goal in the back of their minds, but is it realistic? In theory, it’s possible, especially with new advances in Gorilla Glass etc, but at what cost? The product may be too expensive for users. Most consumers are happy with an iPhone today, albeit with a fragile glass screen, because they love the look, feel, and UX of the device. The tradeoff of needing a case, insurance, etc. is now seen as acceptable by users, so why would Apple prioritize more ruggedness over other features in their design process? (29:10)
Is repairability better than durability?
It depends. Apple’s attitude seems to be that repairability is a noble concept, but if a phone is durable enough to start with then users will not need to repair it. In the past phone companies made a lot of money by insisting that users could only send them the devices for repair, especially if they wanted to maintain their warranties. But these days the ‘Right to Repair’ movement is forcing a rethink. Manufacturers can either make a product that is disposable once it fails or one that’s repairable and now they’re starting to give users repair options (even Apple). At least Apple can make money from selling parts.
A notable backlash was farmers against John Deere who made their tractors more computerized and harder to repair. This dispute has been rumbling on, with farmers unhappy that they can’t repair their machines easily leading to inconvenience and delays. So there are cases of some products where making them easier to repair and supplying spare components needs to be integrated into their design in order to avoid user complaints later on when the products are in the field (literally in the case of tractors). (31:49)