ISO/TR 16250:2013

TECHNICAL ISO/TR
REPORT 16250
First edition
2013-07-15
Road vehicles — Objective rating
metrics for dynamic systems
Véhicules routiers — Mesures pour l’évaluation objective des
systèmes dynamiques
Reference number
©
ISO 2013
© ISO 2013
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2013 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Terms and definitions . 1
3 Symbols and abbreviated terms . 1
3.1 General abbreviated terms . 1
3.2 General symbols and subscripts . 2
3.3 CORA . 2
3.4 EARTH and EEARTH . 3
3.5 Model reliability metric . 4
3.6 Bayesian confidence metric . 5
3.7 Overall ISO rating . 5
4 General requirements to the data . 5
5 CORA metric . 6
5.1 Corridor rating . 6
5.2 Cross-correlation rating . 8
5.3 Step-by-step procedure .10
6 EARTH metric .11
6.1 EARTH phase score .12
6.2 EARTH magnitude score .13
6.3 EARTH slope score .14
6.4 Overall EARTH score .15
6.5 Step-by-step procedure .15
7 Model reliability metric .16
8 Bayesian confidence metric .16
9 ISO metric .18
9.1 CORA corridor method .18
9.2 EEARTH method .18
9.3 Calculation of the overall ISO rating .23
9.4 Meaning of the objective rating score .24
10 Pre-processing of the data .24
10.1 Sampling rate.25
10.2 Filtering .25
10.3 Interval of evaluation .25
11 Limitations .26
11.1 Type of signals .26
11.2 Metrics validation .26
11.3 Meaning of the results .26
11.4 Multiple responses .27
Annex A (informative) Child restraint example .28
Annex B (informative) Sled test example .46
Annex C (informative) Case studies .51
Bibliography .65
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2. www.iso.org/directives
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received. www.iso.org/patents
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
The committee responsible for this document is ISO/TC 22, Road vehicles, Subcommittee SC 10, Impact
test procedures, and SC 12, Passive safety crash protection systems.
iv © ISO 2013 – All rights reserved

Introduction
Computer-Aided Engineering (CAE) has become a vital tool for product development in the automobile
industry. Various computer programs and models are developed to simulate dynamic systems. To
maximize the use of these models, their validity and predictive capabilities need to be assessed
quantitatively. Model validation is the process of comparing CAE model outputs with test measurements
in order to assess the validity or predictive capabilities of the CAE model for its intended usage. The
fundamental concepts and terminology of model validation have been established mainly by standard
[6]
committees including the United States Department of Energy (DOE), the American Institute of
[1]
Aeronautics and Astronautics (AIAA), the Defense Modeling and Simulation Office (DMSO) of the US
[5]
Department of Defense (DOD), the American Society of Mechanical Engineers Standards Committee
[2]
(ASME) on verification and validation of Computational Solid Mechanics, Computational Fluid
[3] [4][22][23]
Dynamics and Heat Transfer, and various other professional societies.
One of the critical tasks to achieve quantitative assessment of models is to develop a validation metric
that has the desirable metric properties to quantify the discrepancy between functional or time history
[7][19][20]
responses from both physical test and simulation result of a dynamic system. Developing
quantitative model validation methods has attracted considerable researchers’ interest in recent years.
[12][13][14][18][20][21][26][28][29][32]
However, the primary consideration in the selection of an effective
metric should be based on the application requirements. In general, the validation metric is a quantitative
measurement of the degree of agreement between the physical test and simulation result.
In this Technical Report, four state-of-the-art objective rating metrics are investigated and they are:
[10][30][31]
CORrelation and Analysis (CORA) metric, Error Assessment of Response Time Histories
[28][34] [18][27][35] [14][16][36]
(EARTH) metric, model reliability metric, and Bayesian confidence metric.
Multiple dynamic system examples for both tests and CAE models are used to show their advantages
and limitations. Further enhancements of the CORA corridor rating and the development of an Enhanced
Error Assessment of Response Time Histories (EEARTH) metric are proposed to improve the robustness
of these metrics. A new combined objective rating metric is developed to standardize the calculation
of the correlation between two time history signals of dynamic systems. Multiple vehicle safety case
studies are used to demonstrate the effectiveness and usefulness of the proposed metric for an ISO
Technical Report.
TECHNICAL REPORT ISO/TR 16250:2013(E)
Road vehicles — Objective rating metrics for dynamic
systems
1 Scope
This Technical Report specifies a method to calculate the level of correlation between two non-ambiguous
signals. The focus of the methods described in this Technical Report is on the comparison of time-history
signals or functional responses obtained in all kinds of tests of the passive safety of vehicles and the
corresponding numerical si
...