Reliability Modelling
Series Systems
Parallel Systems
Bayesian Testing
Design Verification
Design for Six Sigma roadmap
HoQ1￫Boundary￫HoQ2￫P-diagram￫DFMEA￫PFMEA￫SCIF￫Control plan
HoQ1￫Boundary￫HoQ2￫P-diagram￫DFMEA￫PFMEA￫SCIF￫Control plan
HoQ1￫Boundary￫HoQ2￫P-diagram￫DFMEA￫PFMEA￫SCIF￫Control plan
HoQ1￫Boundary￫HoQ2￫P-diagram￫DFMEA￫PFMEA￫SCIF￫Control plan
System validation plan
System verification plan
Sub-system verification plan
Component verification plan
Time line
System
Sub-system
Component
System
Sub-system
Slide 6
What do I Verify & Validate?
Slide 7
Pool is 50 metres long
500 metres
DfSS Process
Boundarydiagram
Parameter diagram
DesignFMEA
SCIF
HoQ #1
ProcessFMEA
SCIFManufacturingcontrol plan
Field performance
Project goals
System boundary diagram
Slide 9
Anglemechanism
Velocitymechanism
Support
Barrel
Distance(475 – 525 m)
Angle43o – 47o
Velocity (m/s)75.5 – 79.2
Gravitationalacceleration(m/sec2)9.81 – 9.82
Sound(100 – 120 dB)
Height(> 90 m)
Soundmechanism
Pool
Side ShowBob
( ) g
2sinvd
2θ
=( )2g
sinvh
22θ
=
Velocity mechanism boundary diagram
Slide 10
SupportInner barrel
wall
Spring constant
Weight
Frictioncoefficient
Distancecompressed
v = f(spring constant,friction coefficient,weight, distancecompressed)
Rollers Plunger Spring
Compressionlever
Compressiongauge
Side ShowBob
Velocity (m/s)75.5 – 79.2
Spring boundary diagram
Slide 11
Spring constant
Wire diameter
Free length
Number of active windings
Young’s modulus
Force =f(Young’s modulus, wire diameter, free length, number of active windings, Poisson ratio, outer diameter)
Poisson ratio
Outer diameter
Bogey test
• The duration for a Bogey test is equal to the reliability requirements being demonstrated.• For example, if a test is designed to demonstrate that a component has specific reliability at
100 hours means the test duration is100 hours, then the test is designated as a Bogey test.
• Example 1: If 95% reliability is required at 200,000 kilometres of service, then the units being tested will be removed from testing when they fail or when they complete the equivalent of 200,000 kilometres of testing. The sample size required to demonstrate reliability of r with a confidence level of c is:
Bogey: a numerical standard of performance set up as a mark to be aimed at especially in competition.
ln(r)
c)ln(1N
−=
Slide 12
Calculation of test sample size
Reliability Confidence Sample Size
99% 95% 299
99% 90% 229
99% 50% 69
95% 95% 59
95% 90% 45
95% 80% 31
90% 90% 22
90% 80% 16
Example 2: A windshield wiper motor must demonstrate 99% reliability with 90% confidence at 2 million cycles of operation. How many motors must be tested to 2 million cycles with no failures to meet these requirements:
Following table shows relationship between sample size, confidence interval & reliability
229ln(r)
c)ln(1N =
−=
Slide 13
Do tested parts represent the population?
• Variation from multiple production operators
• Variation from multiple lots of raw materials
• Variation from tool wear
• Variation from machine maintenance
• Variation from seasonal climatic changes
• Variation from supplier changes
Slide 14
Does the test represent actual use?
• Number of cycles
• Environment
• Variability in the part itself
Slide 15
What are the design requirements
• Application environment is harsh and highly variable• Vehicles must operate reliably in artic conditions and in desert conditions
• Driving profiles range from the 16 year-old male to the 90 year-old female
• An airliner may fly long haul ocean routes for 20 years
• Identical model flies short-range routes resulting in many more take-offs
• Combining this variety into a realistic test is difficult• Consider specific tests aimed at particular failure causes or failure modes
• Ensure component requirements are properly linked to the system requirements
Slide 16
95th percentile customer
• 95th percentile of what?• Cycles
• Cold temperature
• Hot temperature
• Salt
• Automobile engine• Starts
• Run time
Slide 17
Percentile vs. cycles
0
100
200
300
400
500
600
700
800
900
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Percentile
Bra
ke A
pp
lica
tio
ns
(Th
ou
san
ds)
Slide 18
Randomise loads
Slide 19
0.0
0.5
1.0
1.5
2.0
2.5
3.0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Deceleration (g-force)
Pro
ba
bil
ity
De
nsi
ty
Average Customer
95th Percentile Customer
Bayesian Weibull
• Bogey test is inefficient. By extending the test duration beyond the required life, the total time on test can often be reduced. When the test duration is extended, it is necessary to make assumptions concerning the shape of distribution of the time to fail.
• In Bayesian testing this is done by assuming a Weibull distribution for time to fail and by assuming a shape parameter
• Assume β is known:• β is dependent on the physics of failure• Year to year re-qualification
• Programme to programme
• Old design to new design
• Time raised to β is an exponential random variable• Reduces testing requirements
Slide 20
Effect of shape parameter on Time-to-FailWeibull slope β (shape) = 3.6
4.03.63.22.82.42.01.61.20.80.40.0
400
300
200
100
0
Bogeys
Fre
qu
en
cy
Shape 3.6
Scale 2
N 2500
Weibull
Histogram of Time-to-Fail
Slide 21
Effect of shape parameter on Time-to-FailWeibull slope β (shape) = 1
80706050403020100
900
800
700
600
500
400
300
200
100
0
Bogeys
Fre
qu
en
cy
Mean 15
N 2500
Histogram of Time-to-FailWeibull Shape Parameter = 1
Slide 22
Effect of shape parameter on Time-to-FailWeibull slope β (shape) = 1.8
14.212.811.29.68.06.44.83.21.60.0
350
300
250
200
150
100
50
0
Bogeys
Fre
qu
en
cy
Shape 1.8
Scale 5
N 2500
Weibull
Histogram of Time-to-Fail
Slide 23
Effect of shape parameter on Time-to-FailWeibull slope β (shape) = 8
1.91.71.51.31.10.90.70.5
600
500
400
300
200
100
0
Bogeys
Fre
qu
en
cy
Shape 8
Scale 1.4
N 2500
Weibull
Histogram of Time-to-Fail
Slide 24
Bayesian test design• Tests are designed to demonstrate a specific reliability at a specific
time (R95/C90 at 100,000 miles).
• To have 95% reliability at 100,000 miles the mean must be• greater than 1,500,000 miles if b = 1.0
• greater than 330,000 miles if b = 1.8
• greater than 220,000 miles if b = 3.6
• greater than 140,000 miles if b = 8.0
Slide 25
Bayesian test design
• Supported and encouraged by Automotive Industry.
• Bogey Testing is the most inefficient method of testing.• If 95% reliability is required with 90% confidence at 100 hours, there is no less efficient
method than designing a test for a duration of100 hours.
• Use extended testing.
Slide 26
Sample size to achieve R95/C90 at 1 Bogeyb: shape parameter 1 Bogey 2 Bogeys 3 Bogeys
1.0 45 22 15
1.2 45 20 12
1.5 45 16 9
2.0 45 11 5
3.5 45 4 1
7.0 45 1 1
Slide 27
Sample size to achieve R90/C90 at 1 Bogeyb: shape parameter 1 Bogey 2 Bogeys 3 Bogeys
1.0 22 11 8
1.2 22 10 6
1.5 22 8 5
2.0 22 6 3
3.5 22 2 1
7.0 22 1 1
Slide 28
Establishing the shape parameter (b)
• Can be based on physics of failure
• Should be based on (at least) 7 failures
• Can also be based on experience
• There should be a policy to build an internal database
Slide 29
Other benefits
• Failure modes will be known
• Testing can be accelerated
• Designs can be compared
• Component failures can be catalogued
• Degradation testing may be possible
• Cost savings because we understand the limits of the design
Slide 30
Example test duration
• R95/C90 at 1.5 million miles means
• to have 95% reliability at 100,000 (i.e. L5 = 1.2 million miles), with 90% confidence.
• If the Weibull slope b = 7.04 and the sample size = 8. What is the test duration?
• If 1 unit fails at time = 1.82 million miles does the product fail to demonstrate the required reliability?
Slide 31
Example test duration
Slide 32
• One unit fails after 1.82 million cycles• How long do the remaining 7 units have to survive to meet the original
test requirements?
Example
• Assume: The requirements are not met.
• Remember that the width of the confidence limits decrease as the sample size and test duration increase.
• You must make a decision:• Does the product fail to meet the requirements with this small sample size, or
• Is the product not durable enough
Slide 33
Example
• There are several options• Continue testing the surviving items
• Test additional items
• Re-design and re-test
• For extended test plans (beyond 1 lifetime) it is common to obtain failures• Failures beyond 1 lifetime do not necessarily indicate a poor design
<BayesianTestingTemplate.xls>
Slide 34
• Does a basketball player have a 90% free throw percentage?
• Verify free throw percentage with 85% confidence
• Acceptance test requires• 18 consecutive successful free throws
• If the player’s free throw percentage is exactly 90%• The probability of passing the test is
Probability of passing a bogey test
15.09.018
==p
• Demonstrate a reliability of 95% at 10 years with a confidence level of 90%
• System has a time to fail distribution for a system with a Weibull shape parameter of 2.5
• Four systems have to be tested for 26.3 years without failure
Probability of passing a Bayesian test
( )
( )
( ) ( ) 1.0563.0Test Passing
563.03.26
81.3295.0ln
10
4
81.32
3.26
5.2/1
5.2
==
=
=−
=
=
−
P
eR
• 45 units with 95% reliability at 100 time units• β = 2
• θ = 441.54
• To demonstrate 95% reliability with 90% confidence• 45 units surviving to 100 time units
• 11 units surviving to 202 time units
• 5 units surviving to 300 time units
• Compute the probability of passing each of the 3 tests above
Zero failure test plan problems
<ProbabilityOfPassingTest.xls>
Probability of passing a test – β = 2
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
5 15 25 35 45 55 65 75 85
True L5 Life (Years)
Pro
ba
bil
ity
of
Pa
ss
ing
Te
st
Probability of passing test is1 – confidence
Probability of passing a test
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
5 15 25 35 45 55 65 75 85
True L5 Life (Years)
Pro
bab
ilit
y o
f P
assin
g T
est
b = 1.5
b = 2.5
b = 3.5
b = 5
Confidence interval width
0 1 2 3 4 5 6 7
1
2
3
Nu
mb
er
of
Fa
ilu
res
Confidence Interval Width (# of Bogeys)
b = 1.5
b = 2.5
• Statistically designed test can be defeated by selecting the parts in a non-random manner
• Select parts that are close to the nominal dimension
• The variability is reduced
• Positive bias is introduced because the parts close to nominal will perform better than those further than nominal
• Combination of reduced variability and positive bias greatly improves the probability of passing the test
• Select parts at worst case tolerance combinations
• The variability is reduced
• Negative bias is introduced because the worst-case parts will perform worse than those randomly selected
• Combination of reduced variability and negative bias greatly reduces the probability of passing the test
Sensitivity to a random sample
• Desire to determine free throw percentage for all male high school basketball players in Detroit
• Select the best free throw shooter from each team?• Similar to building prototype parts to nominal
• Select the worst free throw shooter from each team?• Similar to building prototype parts to worst case
• In either case a statistical test is nonsense
Sensitivity to a random sample
• Auction price for clocks along
• clock age
• number of bidders
• population is normally distributed• mean = 1327
• standard deviation = 393
• population L5 = 681
• But what if the sample is not random?
• Five oldest clocks are selected
• 90% confidence interval for L5 is 831 to 1641
• true L5 of 681 falls outside this range
• selecting the oldest clocks is similar to making 5 prototype parts with a new tool
Sensitivity to a random sample
Sensitivity to a random sample
0 500 1000 1500 2000 2500
Price
Pro
ba
bil
ity D
en
sit
y
Age Greater Than 180
Entire Population
Test at Worst Case tolerance?
• What is worst case• Nominal may be worst case
• Launching projectile at 45°
• Can you produce worst case?Number of
CharacteristicsNumber of Tolerance
Combinations
2 4
3 8
4 16
5 32
10 1,024
20 1,048,576
50 1.13×1015
100 1.27×1030
Slide 45
0%
5%
10%
15%
20%
25%
20% 25% 30% 35% 40% 45% 50%
Pro
ba
bil
ity
of
All
Ch
ara
cte
rist
ics
in W
ors
t C
ase
Re
gio
n
Percentage of Tolerance Defined as Worst Case
2 Characteristics 3 Characteristics 4 Characteristics 5 Characteristics 6 Characteristics 7 Characteristics
Assumes Cpk = 1.33
Test design
• Use statistics as baseline
• Recommend sample size of 4 to 8• Select samples as close to worst case as possible
• More samples if testing is inexpensive and sources of variation can be incorporated
• Less if testing is expensive
• Always ask if calculations can be substituted
• If one or more samples fail prior to Bogey test is failed
• If all samples exceed Bogey• If one or two failures test is passed
• If more than two failures
• Compute reliability confidence limits and use an guideline
• Compute shape with confidence limits and use as guideline
Slide 48
Strategy
• Build confidence using a bottom up approach
• Short tests aimed at key failure modes or causes based on the P-diagram & FMEA
• Understand equations• Test unknown areas
• Use models – FEA, SPICE, CATIA, etc.
• Use standard, validated components
• Focus on System Interaction
Slide 49
