ISO/IEC TR 24027:2021

TECHNICAL ISO/IEC TR
REPORT 24027
First edition
2021-11
Information technology — Artificial
intelligence (AI) — Bias in AI systems
and AI aided decision making
Technologie de l'information — Intelligence artificielle (IA) —
Tendance dans les systèmes de l'IA et dans la prise de décision assistée
par l'IA
Reference number
© ISO/IEC 2021
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
© ISO/IEC 2021 – All rights reserved

Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
3.1 Artificial intelligence . 1
3.2 Bias . 2
4 Abbreviations . 3
5 Overview of bias and fairness . 3
5.1 General . 3
5.2 Overview of bias . 3
5.3 Overview of fairness. 5
6 Sources of unwanted bias in AI systems . 6
6.1 General . 6
6.2 Human cognitive biases. 7
6.2.1 General . 7
6.2.2 Automation bias . 7
6.2.3 Group attribution bias . 8
6.2.4 Implicit bias . . 8
6.2.5 Confirmation bias . 8
6.2.6 In-group bias . 8
6.2.7 Out-group homogeneity bias . 8
6.2.8 Societal bias . 9
6.2.9 Rule-based system design . . 9
6.2.10 Requirements bias . 10
6.3 Data bias . 10
6.3.1 General . 10
6.3.2 Statistical bias. 10
6.3.3 Data labels and labelling process . 11
6.3.4 Non-representative sampling . 11
6.3.5 Missing features and labels . 11
6.3.6 Data processing .12
6.3.7 Simpson's paradox .12
6.3.8 Data aggregation . 12
6.3.9 Distributed training . 12
6.3.10 Other sources of data bias .12
6.4 Bias introduced by engineering decisions .12
6.4.1 General .12
6.4.2 Feature engineering .12
6.4.3 Algorithm selection .13
6.4.4 Hyperparameter tuning. 13
6.4.5 Informativeness . 14
6.4.6 Model bias . 14
6.4.7 Model interaction . 14
7 Assessment of bias and fairness in AI systems .14
7.1 General . 14
7.2 Confusion matrix . 15
7.3 Equalized odds . 16
7.4 Equality of opportunity . 16
7.5 Demographic parity . 17
7.6 Predictive equality . 17
7.7 Other metrics . 17
iii
© ISO/IEC 2021 – All rights reserved

8 Treatment of unwanted bias throughout an AI system life cycle .17
8.1 General . 17
8.2 Inception . 17
8.2.1 General . 17
8.2.2 External requirements. 18
8.2.3 Internal requirements . 19
8.2.4 Trans-disciplinary experts . 19
8.2.5 Identification of stakeholders . 19
8.2.6 Selection and documentation of data sources . 20
8.2.7 External change . 20
8.2.8 Acceptance criteria . 21
8.3 Design and development . 21
8.3.1 General . 21
8.3.2 Data representation and labelling . 21
8.3.3 Training and tuning .22
8.3.4 Adversarial methods to mitigate bias . 23
8.3.5 Unwanted bias in rule-based systems . 24
8.4 Verification and validation . 24
8.4.1 General . 24
8.4.2 Static analysis of training data and data preparation . 25
8.4.3 Sample checks of labels .
...