Skip to main content Skip to course navigation

Attribute Measurement System Analysis – Kappa and AAA

Learning Objective

By the end of this lesson, the learner should be able to:

  • Explain the purpose of Attribute Measurement System Analysis.
  • Understand categorical inspection judgments such as pass/fail and defect/no defect.
  • Explain the purpose of Kappa.
  • Explain Attribute Agreement Analysis (AAA).
  • Understand the importance of a reference standard.
  • Apply attribute measurement concepts to an inspection situation.

1. Introduction

Not every measurement produces a numerical value.

In many processes, inspectors make categorical judgments, such as:

  • Pass / Fail
  • Defect / No Defect
  • Accept / Reject
  • Correct / Incorrect

Attribute Measurement System Analysis (Attribute MSA) evaluates the consistency of these judgments.

The objective is to determine whether different appraisers, or the same appraiser, are making classifications consistently.

2. Understanding Attribute Measurement

In an attribute measurement system, the result is a category rather than a continuous numerical measurement.

For example:

A quality inspector examines a component and records:

Pass — acceptable product

or

Fail — defective product

The reliability of these classifications is important because inconsistent inspection decisions can affect the quality data used by the improvement team.

3. Kappa

Kappa measures agreement beyond what would be expected by chance, under the applicable interpretation.

Simple percentage agreement tells us how often classifications agree.

Kappa provides additional information by considering the possibility that some agreement could occur simply by chance.

Therefore, a Black Belt should not rely only on percentage agreement when evaluating an attribute measurement system.

4. Attribute Agreement Analysis (AAA)

Attribute Agreement Analysis (AAA) evaluates consistency:

  • Among appraisers
  • Between appraisers and a reference standard

For example, several inspectors classify the same set of products as defective or non-defective.

AAA can be used to examine whether their classifications are consistent and whether they agree with the established standard.

5. The Reference Standard

A standard provides the reference used to judge classification consistency.

For example, if a product has already been objectively classified as defective according to the applicable inspection criteria, that classification can serve as the reference against which appraiser decisions are compared.

Clear and consistently understood criteria are therefore important to attribute inspection.

6. Tools & Techniques

The source identifies the following tools:

  • Kappa coefficient
  • Attribute Agreement Analysis (AAA)
  • Cross-tabulation
  • Minitab

These tools help the Black Belt investigate whether categorical measurement results are consistent and meaningful.

7. Application Example

Product Inspection

A manufacturing company uses several inspectors to classify products as:

Defective or Non-defective.

The Black Belt uses Attribute Agreement Analysis to examine:

  • Agreement among inspectors
  • Agreement with the reference standard
  • Consistency of the classification process

Kappa can provide additional information about agreement beyond simple percentage agreement.

8. Case Study

Food Processing Plant

A food processing plant conducted an Attribute Agreement Analysis.

Initially, the analysis showed 70% agreement.

Investigation identified the need for better calibration and clearer defect criteria.

After:

  • Calibration training
  • Clearer defect criteria

agreement increased to 95%.

Black Belt Learning

The case demonstrates that inconsistent attribute inspection can be addressed through improved understanding, calibration and standardization of classification criteria.

9. Black Belt Perspective

A Black Belt should ask:

“Are inspectors consistently classifying the same condition in the same way?”

The improvement sequence is:

Define Classification Criteria → Evaluate Agreement → Identify Inconsistency → Calibrate/Standardize → Re-evaluate

The purpose is to establish confidence in categorical inspection data before using it for further process decisions.

10. Lesson Practice

  1. Why is agreement important in attribute inspection?
  2. What does Kappa add beyond simple percentage agreement?
  3. Why should defect criteria be clear?
  4. What is the purpose of a reference standard?
  5. If several inspectors classify the same product differently, what should the Black Belt investigate?

11. Key Learning Points

  • Attribute MSA evaluates categorical judgments.
  • Examples include pass/fail and defect/no defect.
  • Kappa evaluates agreement beyond chance under the applicable interpretation.
  • AAA evaluates consistency among appraisers and against a standard.
  • A reference standard provides a basis for evaluating classification consistency.
  • Calibration and clear inspection criteria can improve agreement.
  • Kappa, AAA, cross-tabulation and Minitab are relevant tools.

12. Lesson Conclusion

Attribute Measurement System Analysis helps the Black Belt determine whether categorical inspection results are sufficiently consistent for reliable decision-making.

The essential concepts are:

Attribute Data → Agreement → Kappa / AAA → Calibration & Standardization → Reliable Inspection