Breakthrough Performance in Design for Successful Systems

Course Group: 
Capability Development
Course Outline: 

This five day course (being run over 4 long days for March 2018), extends on the advanced modern techniques for test design and analysis taught to systems engineers and project managers initially in foundational design of experiments (DOE) or design for six sigma (DFSS) either through the University of NSW’s masters subject (ZEIT 8034) or Air Academy Associates’ (AAA) Operational DOE course. Students who have been applying the basic test design and test analysis techniques to screen, model or validate system performance with packages like DOE PRO XL, SPC XL and rdExpert, will be introduced to more advanced functionality offered by Quantum XL and DFSS techniques to improve their test designing and analysis in the following areas:

  • Methods to build quality in from conceptual design through to service (house of quality)
  • How to optimise designs more fully, including operational models.
  • How to extract and use probabilistic transfer functions more fully
  • Expected value analysis (EVA) using Monte Carlo routines to estimate more precise confidence intervals of outputs
  • Using EVA to set more robust design inputs and allocate cost-effective tolerances for design or operations
  • Logistics regression techniques to better evaluate and predict from testing when outputs are binary like pass/fail, defeat/no defeat, decoy/no decoy, penetrate/no penetrate as used in electronic warfare, operational analysis and cybersecurity testing
  • Historical (big) data analysis techniques.
  • Case studies and practical workshops on the above.

Students who complete this course will be eligible in the following year to submit to a short examination and review of three case studies of their work by AAA in order to be Green Belt Industry accredited in DFSS techniques. This accreditation is well regarded in international industry.

Duration:  5 day course being conducted over 4 long days

Delivery Mode: Classroom

Location: Canberra (ADFA)

In-house: All states and neighbouring countries, contact the Professional Education Course Unit for more information. Recommended for groups of 10 or more


What you will receive

  • Comprehensive set of course notes
  • UNSW Canberra and AAA certificate of attendance
  • Morning tea, lunch and afternoon tea

What you will need to bring

  • The two textbooks from the foundations course in DOE
  • A laptop with Quantum XL and rdExpert loaded and working (DOE Pro XL and SPC XL would be an advantage as well, but Quantum XL will suffice)
  • Any example test design and analysis you want to check


“Excellent instructor for very tough subject, good interaction with class, great training materials, books, visuals, etc.”

“Overall, one of the very best training courses I have ever had.  Top notch instructor, presentation, and knowledge skills.”

“I really enjoyed the team interactions and the excellent instructor.  I have been to a number of courses and this has been my favorite in 10 years!”

“First course I have ever taken where exercises truly added to the understanding.  The most clear and meaningful presentation on process improvement I have seen to date.”



This course is extremely valuable for acquisition and sustainment managers and systems engineers who do testing, evaluating or research on performance of systems. The course has been designed for professionals to apply test design and analysis using relatively simple modern packages that help assess performance of systems probabilistically (i.e., real systems). Attendees do need to have completed the foundation course by AAA or UNSW and practically applied those basic techniques to at least one system in their workplace or hobby.

Participants are encouraged to bring along a piece of their own work-related test design and analysis to help improve.




Introduction to Design for Six Sigma (DFSS)

  • The Identify, Design, Optimize, Validate (IDOV) Methodology
  • DFSS Goals and Benefits
  • Key Ingredients for success
  • Quantifying Savings and Benefits


The DFSS Scorecard

  • Keeping Score using the DFSS Scorecard
  • Scorecard Elements and Construction
    • Parts
    • Process
    • Performance
    • Software
  • Methods for Computing DPU
  • Hands-on practice completing a DFSS Scorecard using DFSS Software


Identify:  The DFSS Project and Team

  • DFSS projects and studies
  • Key elements of the business case
  • Initiating and Scoping the project
  • Project charters
  • The DFSS team
  • Stakeholder analysis


Identify:  The Voice of the Customer (VOC)

  • The Identify phase of IDOV
  • Kano’s Model
  • Determining Customer Wants and Needs
  • Sampling Methods
  • Identifying Critical to Customer Requirements
  • Prioritizing Customer Requirements
  • Introduction to Quality Function Deployment (QFD) and the House of Quality
  • Hands-on Practice Completing House of Quality #1 to identify Measurable CTCs (critical to customer measures)


Design:  Concept Generation and Selection, Product Design, and Requirements Flowdown

  • The Design phase of IDOV
  • Assigning Specs and Formulating Design Concepts
  • FAST diagrams
  • Overview of Triz and Axiomatic Design
  • Comparing Alternate Design Concepts using Pugh Concept Selection
  • Risk Analysis and Management
  • Building House of Quality #2



Design:  Transfer Functions

  • Importance of Transfer Functions for IDOV
  • Methods for Obtaining Transfer Functions
    • Design of Experiments (DOE)
    • Simulation
    • Historical Data Analysis
  • Using Multiple Transfer Functions to optimize performance
  • Multiple Response Optimization experiment using the Statapult®
  • Logistic Regression Analysis
  • Historical (Big) Data Analysis and Predictive Analytics



Optimize:  Expected Value Analysis

  • Meaning of expected value
  • Expected Value Analysis (EVA) for Quantifying the Distribution of the Output
  • Comparing Methods and Strategies for EVA
    • Root Sum Square (RSS)
    • Monte Carlo Simulation
  • Using DFSS software for EVA Analysis
  • Determining distribution of inputs
  • Working with Normal and Non-Normal Input and Output Distributions


Optimize:  Robust Design

  • Understanding how input means affect output performance
  • Noise factors and P-diagrams
  • Robust Design: Finding Optimal Mean Settings for the Inputs to Minimize Variation in an Output
  • Methods for Robust Design
  • Hands-on Robust Design experiment using the Statapult®
  • Computer Based Robust Design (Parameter Design) using DFSS software
  • Robust design examples and exercises


Optimize:  Tolerance Allocation

  • Quantifying the Sensitivity of the Output DPM to Changes in the Input Variable’s Standard Deviation
  • Tolerance Allocation Examples and Exercises
  • Setting Tolerances
  • Methods to Reduce the Standard Deviation of Inputs


Design for X-ability and Scorecard Updates

  • DFX Concepts – beyond just designing for performance
  • DFX principles and examples
  • Introduction to Design for Reliability (DFR)
    • Designing with Reliability in Mind
    • Common life data distributions and analysis
  • Mistake Proofing and Error Proofing
  • Capability prediction
  • Design scorecard updating and analysis


Validate:  The Achievement of Breakthrough Performance

  • Sensitivity Analysis
  • Quantifying the impact of mean and standard deviation shifts in design inputs
  • Developing prototypes, test cases and pilots
  • Introduction to high throughput testing for validating performance
  • Performing gap analysis if results don’t confirm


DFSS Case Studies and Examples

Design Exercise:  Applying the IDOV Methodology

  • Hands-on practice with the IDOV process
  • Teams Design a Product to Meet the Customer Needs, using the IDOV process; Teams optimize their design using the IDOV tools and ultimately build and test their designs to validate performance




Mark Kiemele

Mark J. Kiemele, President and Co-founder of Air Academy Associates, has more than 30 years of teaching, consulting, and coaching experience.  Having trained, consulted, or mentored more than 30,000 leaders, scientists, engineers, managers, trainers, practitioners, and college students from more than 20 countries, he is world-renowned for his Knowledge Based KISS (Keep It Simple Statistically) approach to engaging practitioners in applying performance improvement methods.  His support has been requested by an impressive list of global clients, including Xerox, Sony, Microsoft, GE, GlaxoSmithKline, Raytheon, Lockheed-Martin, Northrop-Grumman, Woodward, Samsung, PerkinElmer, Danaher, Corning, EMC, Kaiser Aluminum, Apogee, Schlumberger, Bose, Heritage Valley Health System, John Deere, Valeant, Sandia Laboratories, General Dynamics Land Systems, Aptuit, Pittsburgh Glass Works, Energizer, and Army Materiel Command.

Mark earned a B.S. and M.S. in Mathematics from North Dakota State University and a Ph.D. in Computer Science from Texas A&M University. Dr. Kiemele has been involved in the origin and evolution of Six Sigma, as he trained the first Six Sigma Black Belts at the Six Sigma Research Institute at Motorola and has helped deploy and implement Six Sigma at more than 80 companies worldwide. During his time in the U.S. Air Force, Mark supported the design, development and testing of various weapon systems, including the Maverick and Cruise Missile systems, and was a professor at the U.S. Air Force Academy.  He currently serves on the National Defense Industrial Association’s Test and Evaluation Executive Committee.

In addition to many published papers and articles, he has co-authored the books Basic Statistics: Tools for Continuous Improvement; Knowledge Based Management; Applied Modeling and Simulation: an Integrated Approach to Development and Operation; Network Modeling, Simulation, and Analysis; Lean Six Sigma: A Tools Guide; and Design for Six Sigma: The Tool Guide for Practitioners.  He is also the editor of the text Understanding Industrial Designed Experiments.

No dates? Or unable to attend dates shown? Submit an Expression of Interest below to be notified of upcoming courses.


13 March 2018 - 16 March 2018