From Prototype to Production: What You Need to Know, Part 1

PART 1: DON’T MAKE THIS FATAL MISTAKE

Low-cost electronics modules and “how-to” design guides for hobbyists have made it easy to pop together working prototypes. That’s fine for hobbyists, but if you are planning on selling your creation to the masses, you need to be sure you understanding the following:

There is a HUGE difference between
a prototype and a production-ready design

If your prototype design was generated by experienced senior engineers, then they are likely to be aware of the many additional challenges that must be overcome in moving that design into production.

However, if your prototype design was based on cut/paste “reference designs,” pre-packaged modules, or hobbyist schematics, then you may not even be aware that there is a difficult path forward. In fact, you may make this fatal assumption: The prototype works, therefore let’s build a million of them and get rich!

Unfortunately, that fatal assumption will probably not lead you to wealth, but instead will create excruciating anxiety as you watch your new product crash when it exhibits one or more of the following problems:

• intermittent performance
• inexplicable shutdowns
• excessive power drain (e.g. frequent battery replacement or recharging)
• errors or even total failure due to normal variations in power source or environmental factors such as temperature and humidity
• failure to properly operate over the device’s warranty period
• overheating
• breakage when being normally shipped and handled
• customer frustration due to a poor user interface
• errors when operating near other electronic devices
• other electronic devices malfunctioning when near your device
• failure due to common levels of electrostatic discharge

and this biggie:

• customer injury or death

In future newsletters we’ll provide some tips on how to minimize the risks listed above. In the meantime, if you think that you need some guidance in moving from prototype to production, please contact me. We enjoy helping startup firms achieve their dreams.

-Ed Walker

Baloney Alert Special Report: Derating Guidelines

DACI Newsletter Classics
Valuable lessons from our 1st Qtr 2006 Newsletter

Many engineering departments don’t allow design engineers to operate parts anywhere near specification limits. This policy is spelled out in a Derating Guidelines document that mandates greatly reduced stress levels. For example, a typical derating for resistor power is 50%, which means that the design engineer can’t apply more than 1/4W to a 1/2W resistor. This Special Report asks the simple question: Why not?

Why Can’t Designers Use Maximum Ratings?

Is it because design managers are paranoid? Have they found that when a vendor claims a resistor will work to 1/2W that the resistor will only work to 1/4W? Or how about semiconductor junction temperatures? A typical power diode for example might be rated at 150C, but the designer often must adhere to a derating rule that prohibits a junction temperature greater than 90C. Does this mean that design managers consider power diode vendors to be just as untrustworthy as resistor vendors?

Or could it be that design managers don’t have faith in their design engineers? Are derating factors just a tactful way of applying fudge factors to account for the math errors made by those lazy and sloppy designers?

Although the above hypotheses may be applicable for a few managers, most often the stated rationale for deratings is “improved reliability.”

But do derating guidelines actually deliver this benefit? Please read on.

Costly Overdesign due to Excessive Caution

First let’s assume that vendors provide components that, the great majority of the time, will meet their specifications. (If not, in our efficient capitalist system they would be justly faced with lawsuits and loss of business and bankruptcy.)

Next, let’s assume that design engineers do a pretty good job in predicting worst case application stresses, taking into consideration the effects of tolerances, including aging. (If not they would rapidly be seeking employment in other fields.)

What about transient conditions? Again, competent design engineers will consider peak transient stresses as well as steady-state stresses, and select components accordingly. Or more likely, they will include clamps and filters and other forms of protection to ensure that maximum transient stresses are safely limited.

Okay, what’s left? Not much, other than uncertainties in the analytical models. But such uncertainties are minimized by a comparison of predicted results to prototype test results (a step that all good design engineers insist on).

So here’s the question: How much smaller than unity does a stress ratio need to be to account for such small remaining uncertainties?

A Typical Real World Example

For example, consider a resistor that has a predicted maximum power dissipation of 0.7W and a rating of 1W. Its stress ratio SR is

SR = Predicted / Rated = 0.7 / 1 = 0.7

With worst case tolerances and transients and testing considered, the designer — applying judgment based on experience — believes that a 10% uncertainty is ample, and that the maximum allowable SR should be 0.9. Therefore the designer is pleased with a ratio of 0.7. Plus, the designer knows that the 0.7 maximum is out in the tail of the distribution and that on average the stress will be much lower.

But the designer checks the department’s sacred Derating Guidelines and finds that the allowable stress ratio is only 0.5. Will the designer have a good laugh and leave the design alone and move on to more important tasks? Of course not!

Faced with an upcoming Design Review where some junior reliability engineer would gleefully jump all over this supposedly egregious violation of good design practice, the designer will avoid such embarrassment and spend extra time updating the design to a 2W resistor. Although this will make the design larger/heavier and more costly, with no measurable improvement in quality or reliability, the Derating Gods must not be offended.

Derating Cookbooks Can Cause Design Poisoning

Using a Derating Guidelines cookbook that was developed without support of science-based reasoning can easily result in design poisoning. For example, arbitrary temperature deratings can result in the use of much heavier heat sinks than are really necessary, and such added weight is almost universally a bad thing in a product. The useless extra weight not only takes up valuable space and costs more, but it requires more energy to transport (a particularly unfortunate result for fuel-sensitive aircraft applications). Worse, if the mandated derating forces the designer to shift from simple convection cooling to the use of fans or a liquid cooling pump, then overall reliability will very likely be significantly reduced.

Another major problem with the cookbook approach is that it discourages thinking. An engineering cookbook by definition is supposed to be a tried and true collection of guidelines and rules to assist the designer. Therefore the designer is discouraged from thinking about the aspects of the design that are covered by the cookbook. In theory this is not too bad if the cookbook is regularly updated using a science-based review process.

But has anyone ever read any science-based report that supports the typical cobweb-encrusted Derating Guidelines document [1] used by many engineering firms? And if by chance such a report is stumbled upon in the engineering department’s dusty attic, has it been updated to keep pace with technology?

---

Note 1: Subtitle: “This is the way grandpa did it and by God this is the way we’re going to do it!”

---

Oversimplification Misses Key Concerns

Returning to our resistor power example, savvy designers know that resistor power is really an approximate proxy for resistor temperature. For low-wattage resistor applications this is reasonable because resistor temperature rise is negligible. But as resistor dissipation increases, particularly with today’s ever-shrinking packages, resistor temperature can become a serious concern. One does not want a resistor to desolder itself from the PWA, even if the resistor is operated within its allowable stress derating. But do the derating guidelines for your department address resistor temperatures? Or temperature-related solder degradation issues?

A similar point can be made for capacitor ripple currents, which are also approximate proxies for capacitor core temperatures. Do your derating guidelines mention capacitor core temperatures?

And whereas decades ago the effects of electromigration due to integrated circuit current densities were a valid concern (which may have justified an associated temperature derating), advances in processes over the years have made such deratings obsolete — almost. Today there is renewed concern with electromigration due to advanced miniaturization.

So, like Dad’s old suspenders that were out of fashion for thirty years and then came back into vogue, our old neglected Derating Guidelines might just be partially right every few decades, if only by accident.

Question Authority

The hallmark of science is testability. If someone makes an assertion they are obligated to prove the assertion in a manner that can be replicated by independent observers. If you suspect that your engineering department’s deratings policy is archaic, why not challenge it? Ask the powers-that-be to defend the policy with objective evidence. If they can’t provide such evidence then it’s time for a change.

By helping modernize your deratings policy you can save your company time and money, plus simultaneously improve product reliability — an impressive outcome.

-Ed Walker

A Must-Have Book for Every Test Engineer

The Test Engineer’s Measurement Handbook / How to Design Tests for 1st-Pass Success
by Van Brollini

Van Brollini’s new book is an essential addition to the test engineer’s library, as well as the library of any product manager.

The Handbook contains practical advice that is based on Mr. Brollini’s extensive experience with test development, including unique insights that I have not seen elsewhere, insights that will provide the test engineer with a quantum leap in productivity.

The test engineer will also appreciate the fact that Brollini’s methods — clearly presented as a series of rules, tips, and straightforward equations — are practical and cost-effective, illustrated by real-world examples throughout.

The Handbook’s teachings can be applied with basic math and spreadsheet tools, although Brollini does recommend Design Master™ for best efficiency, particularly for more advanced applications.

(I have known Van for many years, as he was one of the first engineers to adopt our Design Master software. From time to time he has offered suggestions for improvements, which were incorporated into the software.)

The Test Engineer’s Measurement Handbook is available through the DACI website.

-Ed Walker

Lithium Batteries “An Unacceptable Risk” to Aircraft

Excerpts from “Lithium Battery Shipments On Passenger Planes ‘An Unacceptable Risk,’ Say Aircraft Makers,” by  Suman Varandani, 10 March 2015 International Business Times:

“Aircraft makers and pilot unions are calling for a ban on the transport of lithium battery shipments aboard passenger planes following fears of fires that could prove difficult for aircraft fire protection systems to contain. An industry paper obtained by The Associated Press (AP) cited recent tests conducted by the Federal Aviation Administration (FAA) that showed overheating of the batteries could result in explosions.”

“The FAA tests revealed that batteries emit explosive gases when overheated, and that aircraft fire protection systems ‘are unable to suppress or extinguish a fire involving significant quantities of lithium batteries, resulting in reduced time available for safe flight and landing of an aircraft to a diversion airport,’ according to AP. ‘Therefore, continuing to allow the carriage of lithium batteries within today’s transport category aircraft cargo compartments is an unacceptable risk to the air transport industry.'”

Note that charging of the lithium batteries is not required to initiate a fire; a fire can start with the batteries sitting unconnected in their shipping containers. Triggers can include a manufacturing fault (short) in a single battery cell, a short caused by mechanical puncture during rough handling or travel, shorted terminals due to improper packaging, external overheating, or battery gas emissions that can be ignited by electrostatic discharge.

-Ed Walker

Update 3-21-15: I guess the word is spreading about lithium batteries and aircraft safety. A reader sent in this pix of a warning label that was on his recently purchased lithium batteries:

You Want To Be A Consultant? Rule #4: Have a Thick Skin

An assignment a few years ago required some very specialized design work. We provided a detailed report with many specific recommendations, many of which were ignored. When we politely pointed this out to the client, we were told, “Just because we pay you for advice, that doesn’t mean we’ll follow it.”(1)

Fair enough. Managers sometimes have to make difficult tradeoffs, with technical advice being only one of many parameters that must be considered. Consultants should therefore not be offended when a manager decides to assign a lower priority to their recommendations — we don’t see the big picture; the manager does.

In another example, a colleague of mine — arguably one of the top experts for guiding the preparation of winning proposals for military contracts — arrived to head up a hot time-sensitive proposal effort, only to spend two days sitting in a waiting room. Whether the client was suddenly engaged in an emergency task or whether they were being woefully inefficient doesn’t matter; this was the client’s prerogative, and nothing to get upset about. (My colleague was paid handsomely for sitting.)

Bottom line: although consultants should expect to be treated with professionalism and respect, they should not expect to be given any special privileges or accommodations, and they certainly should never demand such treatment. We are there to do a job with minimal hand-holding, not to be treated like visiting royalty, and our egos should be accordingly prepared.(2)

-Ed Walker

Note 1: In this example, the subsequent eruption of significant technical problems indicated that ignoring our advice was not a wise decision.
Note 2: For example, a consultant should not assume that an office will be provided, or even a desk. Be prepared to grab a table in the cafeteria, or to use the desk of someone who’s on vacation.

Malaysian Flight 370: Lithium Battery Fire Is A Reasonable Hypothesis

The cargo fire hypothesized by Canadian pilot Chris Goodfellow to explain the disappearance of Malaysian Flight 370 (see “Malaysian Flight 370: Canadian pilot’s analysis goes viral“) is a reasonable one.

According to Malaysian officials, the plane was carrying 440 pounds of lithium batteries. Lithium batteries, sitting inert (not being charged or discharged), were identified as the cause of the fire and resultant 2010 crash of a UPS 747 flight at Dubai. Ironically, even though “improper storage” in that case was determined to be the cause of the fire, I have never read any explanation of how improper storage can ignite a lithium battery. It appears more likely that lithium batteries, under certain conditions not completely understood (e.g. a combination of battery construction and chemistry, heat, vibration, and/or shock) can spontaneously ignite, albeit very rarely.

In addition to pilot Goodfellow’s comments, an added interesting point is that Flight 370 also gained very high altitude shortly after communications ceased. It could be that the pilots, upon becoming aware of the fire at that time, tried to quickly elevate the plane to quell the fire by starving it of oxygen. This might have been an excellent maneuver for most fires, but lithium batteries, once ignited, create their own oxygen and will continue to burn at high altitude.

Bottom Line: Until the cause of the disappearance of Flight 370 is positively determined, the possibility of a lithium battery fire is a reasonable hypothesis, and worth investigating.

-Ed Walker

 

Attention Battery and Supercapacitor Makers: Your Hybrid Vehicles Market Just Got Smaller

Innovative thinking includes taking old established technologies and applying them to modern applications. Peugot has recently proved this point by using compressed air instead of the battery (and/or supercapacitor) typically used in hybrid vehicles for stop-start energy savings.

For details please see “The car that runs on AIR: Peugot reveals plans for hybrid set to hit the streets next year“

News Bite: Boeing Continues to Gamble with Lithium Batteries in the Dreamliner

Please see “JAL reports problem with 787 battery on Helsinki-Tokyo flight,” 9 Nov 2013 Reuters, and “Battery Problems on Boeing’s 787 Dreamliner Are Back” by Meghan Foley, 11 Nov 2013 WallStCheatSheet

As mentioned previously (e.g., “Boeing’s Fix for its Flaming Lithium Batteries: Is There A Fatal Flaw?“), until Boeing digs down to the root cause of their lithium battery problems, they — and those who fly on the Dreamliner —  will continue to be exposed to undefined risk.

(Note: Boeing provided a report, “Certification of 787 Battery Solution,” back in April, which lists “improvements” and “enhancements,” but makes no mention of a root cause.)

Toyota Unintended Acceleration: It Wasn’t Just the Floor Mats

(Photo from “Oklahoma Jury Finds Toyota Liable For Sudden Acceleration Fault; Awards $3M In Damages,” by Arjun Kashyap, 25 Oct 2013)

Back in 2009 this newsletter started to express skepticism about Toyota’s insistence that unintended acceleration in their vehicles — in some cases resulting in fatalities — was due to a floor mat that could cause the accelerator pedal to stick. (Those earlier posts are listed at the bottom of this post.)

The reasons for the skepticism were (a) the number of reported cases was relatively high compared to other non-Toyota models, (b) there was at least one case where it was clear that a stuck pedal could not be the cause, and (c) the investigation by the National Highway Traffic Safety Administration (NHTSA) appeared to have some major flaws. Also, DACI’s own experience in investigating several low-probability events has convinced us that customers who report such problems tend to be too easily dismissed, rather than receiving the respectful assumption that they are honest and observant and reporting the problems carefully; i.e. the sudden acceleration stories related by some Toyota owners were not consistent with the floor mat hypothesis. Finally, it didn’t help Toyota’s credibility when Toyota employees were caught congratulating themselves on how they had slowed and limited the accident investigation.

Despite the above reservations, the official report by the NHTSA was that the root cause was the floor mat. Well, after several years it turns out that a detailed analysis of the electronics, as brought out during a recent trial, has confirmed that it was not just the floor mat. Key trial results are detailed in “Toyota Case: Vehicle Testing Confirms Fatal Flaws,” by Junko Yoshida in the 31 October 2013 EETimes. Here’s an excerpted summary of the problems identified:

•  Software bugs that specifically can cause memory corruption

•  Unmaintainable code complexity in Toyota’s software

•  A multifunction kitchen-sink Task X designed to execute everything from throttle control to cruise control and many of the fail-safes

•  That all Task X functions, including fail-safes, are designed to run on the main CPU in the Camry’s electronic control module

•  That the brake override that is supposed to save the day when there is an unintended acceleration is also in Task X

•  The use of an operating system in which there is no protection against hardware or software faults

•  A number of other problems

The deficiencies in the throttle design are shocking, because good rules exist for the design of safety-critical electronics (e.g., Chapter 4, “Safety Analyses,” in The Design Analysis Handbook).

The Toyota case makes one wonder how many other possibly-catastrophic flaws are lurking within the cars we drive, or in other electronics-guided machinery, due to poorly-designed safety-related systems.

—

Prior Toyota Unintended Acceleration Posts:

10 Nov 2009
Toyota Unintended Acceleration Causing Deaths And Injuries

20 Jan 2010
Toyota Sudden Acceleration Update: It’s Not Just Floor Mats

3 Feb 2010
Stop Driving Recalled Toyotas

5 Feb 2010
Toyota’s “Drive By Wire” Throttle System Suspected As Crash Cause

21 Feb 2010
Toyota Joins The Gallery Of Shame

9 Mar 2010
Customers Claim “Fixed” Toyotas Are Still Accelerating

9 Feb 2011
Toyota Unintended Acceleration: “No Electronics-Based Cause”: Not True and Misleading

20 Mar 2012
Toyota Sudden Acceleration: An Example Of How Not To Do A Failure Analysis

DACI Newsletter Extra: More Lithium Battery Troubles?

A Tesla Model S electric vehicle, which uses lithium batteries, burst into flames after supposedly running over a road obstruction. The state trooper who investigated the event did not find evidence of an obstruction.

The car’s battery pack is located in the floor, so it is possible that an obstruction ruptured the pack. But without confirmation, it is also possible that the battery pack erupted in flames due to an internal defect, similar to the Boeing incidents. Low-probability incidents by their very nature will occur rarely, but when they do occur they can be deadly when associated with high energy storage system, such as battery packs.

Photo above from the USA Today report: http://www.usatoday.com/story/money/cars/2013/10/02/tesla-fire-stock-falls-analyst-downgrade/2911345/