If the actual percentage of defects in the lot was higher, the probability would be lower

Acceptance Testing

Andrew J. Collins, Ph.D.
www.drandrewjcollins.com

Frank Batten College of
Engineering & Technology
Engineering Management and Systems Engineering Department
CC – City Of Dream by Ray in Manila

1

Quality Control
Quality Assurance
Improvements to production
Quality Control
Focus on testing and reporting
Includes Acceptance Testing

Dates back to 1930s when manufactures began “systematizing” their processes
Acceptance Sampling is a method in Quality Control

www.drandrewjcollins.com
2

History of Quality Control
History
1930s – Statisticial Quality Control
Acceptance Testing
1956 – Total Quality Control (TQC)
Every department involved
Similar, Company Wide Quality Control (CWQC)
1985 – Total Quality Management (TQM)
1986 – Six Sigma

www.drandrewjcollins.com
3

Books
Montgomery, D. C. (2013). Introduction to statistical quality control (7th Edition): John Wiley & Sons.
www.drandrewjcollins.com
4

Parts of Quality Control
Controls Inspection
Well managed process
Performance Criteria
Competency
Soft Elements
Integrity
Motivation
Confidence
www.drandrewjcollins.com
5

Terms
Lot
Collect of homogeneous items to be tested
Attributes
Quality that is being tested
E.g. reliability, safety, taste

www.drandrewjcollins.com
6

Acceptance Sampling
Acceptance sampling is when a sample of the lot is taken and tested for defects
Approaches
Zero tolerance
Any defects result in rejecting the lot
When no defects is important, i.e., mediciance
Lot Sentencing
If number of defects in the sample exceeds a certain number, then reject the lot
We focus on lot sentencing
www.drandrewjcollins.com
7

Why Acceptance Sampling?
When testing is destructive Do not want too too much
Food testing
Perishable goods
Safety equipment
Exhaustive testing too costly
Time and/or money
Know production has a high quality level
www.drandrewjcollins.com
8

Lot Sentencing
General approach
Determine the risk levels (> zero)
Based on multiple lots
Determine thresholds
Sample lot
Sentence lot
Accept or reject whole lot

www.drandrewjcollins.com
9

Requirements
Homogeneous lots
Come from same production line
Large number of items in a lot
Minimal Risk from handling
Random Sampling
Stratify – divide lot into homogeneous subgroups and draw sample evenly across these groups
Salting – When Vendor places the best at the top or bottom of lot
www.drandrewjcollins.com
10

Types of Plans
Single Sampling
A single sample of ‘n’ units
Acceptance number ‘c’ defects
Less or equal to this, accept lot o/w reject
Double Sampling
Repeat process twice
Multiple Sampling
Repeat process on 2+ samples
Sequential Sampling
Test sampled units one at a time till reject or accept
www.drandrewjcollins.com
11

Single Sampling Plan
A sample plan is determined by:
sample size, n,
Defect acceptance number, c

That is, we will sample ‘n’ form the lot and if observe greater than ‘c’ defects, will reject the lot

How do we determine with ‘n’ and ‘c’ are?
www.drandrewjcollins.com
12

Probability of Observation
Percentage of defects in a lot: p
What is the probability of observing ‘c’ or few defects ‘d’ in a sample of ‘n’?

We can use the binomial distribution
Assuming replacement
Though no replacement, if n << lot size the an ok assumption www.drandrewjcollins.com 13 Binomial # occurrences after multiple Bernoulli events (i.e. how many heads after ’n’ tosses of a coin). X ~ b(n, p) Probability of Observation Using the Binomial Distribution, we get the formula: If the actual percentage of defects in the lot was higher, the probability would be lower Assuming c << n That is, the would higher More defects means we expect to observe more defects in our sample www.drandrewjcollins.com 15 Consumer Tolerances The purchaser (consumer) might say: “I do not a lot with 10% or more defects” But everything is possible, so what percentage of lots is the consumer willing to accept more than 10% or more defects “I will only except 10% or more defects in 1% of lots” www.drandrewjcollins.com 16 Consumer Tolerances “I will only except 10% or more defects in 1% of lots” Lot Tolerance Percent Defective (LTPD) The highest percent defective that the consumer will tolerate Denoted by Lot Tolerance Percent Defective (LTPD) The highest percent defective that the consumer will tolerate Denoted by p2 10% in our case Consumer’s risk How often can the lot break this rule Probability denoted by  1% in our case www.drandrewjcollins.com 17 Consumer Mathematics Consider a lot with minimum unacceptable level ‘p2’. What is the chance it will pass our test? So, to satisfy the consumer’s risk: www.drandrewjcollins.com 18 Consumer Mathematics So to satisfy the consumer, we need: So taking the worst case: Remember, a higher defect level will result in a lower probability www.drandrewjcollins.com 19 Producer Tolerances The producer might say: “I cannot guarantee my lots are defect free but I will guarantee that they have less than 1% defects most of the time” What do you mean “most of the time”? “I will guarantee that they have less than 1% defects 95% of the time” www.drandrewjcollins.com 20 Producer Tolerances “I will guarantee that they have less than 1% defects 95% of the time” Acceptence Quality Level (AQL) Denoted by p1 1% in our case Producer’s risk Rejecting the lot when it is within tolerance levels of defects Probability denoted by  5% in our case (100 – 95) www.drandrewjcollins.com 21 Producer Mathematics What is chance of rejecting a lot that is acceptable? So, to satisfy the producer’s risk: Note, worst allowable case is when you are unnecessary rejecting lots so when probability of reject is maximized which is at the equality www.drandrewjcollins.com 22 Producer Mathematics So to satisfy the producer, we need: So taking the worst case: www.drandrewjcollins.com 23 Determining the plan Now we have two equations and two unknows! To determine the plan, we need to solve these two simulatinoeous equation for ‘c’ and ‘n’ Luckily, there is are couple of simple ways to do this Using Microsoft Excel Solver Using Nomograph www.drandrewjcollins.com 24 Example “Producer will guarantee that they have less than 1% defects 95% of the time” “Consumer will only except 10% or more defects in 1% of lots” www.drandrewjcollins.com 25 Microsoft Excel Solver See MS Excel sheet www.drandrewjcollins.com 26 Microsoft Excel Solver C = 2 N = 81 P1 = 0.01 P2 = 0.1 Implies Alpha is 4.8% So except an 1% error 95.2% time Beta is 0.9% So except a 10% error 0.9% www.drandrewjcollins.com 27 Nomograph www.drandrewjcollins.com 28 Adapted from Montgomery (1991) Nomograph Nomographs provide a tool to derive more quickly the required testing parameters. How they work? Draw two lines Draw a line between the AQL (1%) is marked on the left-hand side and the complement of the producer’s risk (95%) is marked on the right-hand side. Draw a line between the LTPD (10%) is marked on the left-hand side and the consumer’s risk (1%) is marked on the right-hand side Where the lines cross shows the ‘n’ and ‘c’ values Round up values This increases the producer risk www.drandrewjcollins.com 29 Nomograph www.drandrewjcollins.com 30 Example The monograph implies: n = 90 c = 2 Our example result implies n = 81 c = 2 Why different? www.drandrewjcollins.com 31 Nomographs What if lines do not cross? This means that you can not derive a testing plan that satisfies both the consumer and the producer Would suggest that alter producer’s values to make them cross www.drandrewjcollins.com 32 Raspberry Example I will accept 20% defects in my raspberry pack about 20% of the time The producer guarantee 5% defects 90% time Sampling plan: Test 21 raspberries, if find 2 defects then reject www.drandrewjcollins.com 33 Risk Levels and Error Types A quick discussion on Producer’s risk Relates to Type I errors Incorrectly rejecting the null hypothesis Rejecting the lot when it is within tolerance levels Consumer’s risk Relates to Type II errors Incorrectly accepting the null hypothesis Test accepts a lot that contains an unacceptable level of defects www.drandrewjcollins.com 34 Assumptions We are assuming that our inspections are error free www.drandrewjcollins.com 35 Sequential Sampling www.drandrewjcollins.com 36 Sequential Sampling It is a bit annoying and wasteful to test a whole sample to determine if accept or reject Especially if you know that the lot is probably good Sequential Sampling allows you to test a sample item one at a time If then total number of defects goes above some threshold then reject If then total number of defects goes below some threshold then reject Threshold determined by an equation www.drandrewjcollins.com 37 Sequential Sampling Example www.drandrewjcollins.com 38 Sequential Sampling Equations www.drandrewjcollins.com 39 Discussion www.drandrewjcollins.com 40 Standards Lot-by-lot sampling MIL STD 105E ANSI / ASQC 21.4 Variables Sampling Method MIL STD 414 ANSI / ASQC 21.9 www.drandrewjcollins.com 41 Alternatives No inspection Inspect all items Inspect some www.drandrewjcollins.com 42 Novel Uses Acceptance Sampling to aid in verification of computational simulation models Erika Frydenlund, Andrew J. Collins, Christopher J. Lynch, and R. Michael Robinson mimeo www.drandrewjcollins.com 43 ( ) ()1 {0,1,2,...,} nx x n PXxpp p xn - æö ==- ç÷ èø Î ! !()! n n x xnx æö = ç÷ - èø () var()(1) EXnp Xnpp = =-