ISO 25062:2025
(Main)Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Common Industry Format (CIF) for reporting usability evaluations
Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Common Industry Format (CIF) for reporting usability evaluations
This document describes the common industry format (CIF) for reporting usability evaluations. It provides a classification of evaluation approaches and the specifications for the content items (content elements) to be included in an evaluation report based on the selected evaluation approach(es). The intended users of the usability evaluation reports are identified, as well as the situations in which the usability evaluation report can be applied. The usability evaluation reports in this document are applicable to software and hardware systems and products or services used for predefined tasks (excluding generic products, such as a display screen or a keyboard). The content elements are intended to be used as part of system-level documentation resulting from development processes such as those in ISO 9241-210 and the ISO/IEC JTC 1/SC 7 process standards (e.g. ISO/IEC/IEEE 15288 on systems lifecycle management, ISO/IEC/IEEE 12207 on software lifecycle management and ISO/IEC 33001 on process capability assessment).
Ingénierie des systèmes et logiciels — Exigences et évaluation de la qualité des systèmes et logiciels (SQuaRE) — Format industriel commun pour le reporting des évaluations d'utilisabilité
General Information
Relations
Standards Content (Sample)
International
Standard
ISO 25062
First edition
Systems and software
2025-01
engineering — Systems and
software Quality Requirements and
Evaluation (SQuaRE) — Common
Industry Format (CIF) for reporting
usability evaluations
Ingénierie des systèmes et logiciels — Exigences et évaluation de
la qualité des systèmes et logiciels (SQuaRE) — Format industriel
commun pour le reporting des évaluations d'utilisabilité
Reference number
© ISO 2025
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Outcomes and types of usability evaluations . 5
4.1 Outcomes of a usability evaluation .5
4.2 Types of usability evaluation .6
4.3 Assessing conformity of the object of evaluation against specified evaluation criteria .6
5 Conformance . 8
6 Overview of content elements within an evaluation report . 9
7 Description of content elements within an evaluation report . 9
7.1 Executive summary .9
7.2 Description of the object of evaluation .10
7.3 Purpose of the usability evaluation .10
7.4 Evaluation methodology .11
7.4.1 General .11
7.4.2 Type(s) of evaluation used .11
7.4.3 Evaluator(s) .11
7.4.4 Evaluation participants . .11
7.4.5 Tasks used for evaluation . 12
7.4.6 Evaluation environment . 13
7.4.7 Data collected during the evaluation .14
7.4.8 Additional content for conformity assessment (if used) . 15
7.5 Data analysis and results .16
7.5.1 Data analysis .16
7.5.2 Reported results .17
7.6 Recommendations .19
7.7 Conclusions .19
7.8 Appendix .19
7.8.1 General .19
7.8.2 Evaluation protocol .19
7.8.3 Sequence of organizational activities for conducting the evaluation . 20
7.8.4 Independent variables . 20
7.8.5 Predefined evaluation criteria . 20
7.8.6 General instructions given to the participants .21
7.8.7 Specific instructions on tasks .21
7.8.8 Additional content on ethics and intellectual property .21
Annex A (informative) Checklist of content elements for a usability evaluation report .22
Annex B (informative) Evaluation report outline .26
Bibliography .29
iii
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out through
ISO technical committees. Each member body interested in a subject for which a technical committee
has been established has the right to be represented on that committee. International organizations,
governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely
with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of ISO document should be noted. This document was drafted in accordance with the editorial rules of the
ISO/IEC Directives, Part 2 (see www.iso.org/directives).
ISO draws attention to the possibility that the implementation of this document may involve the use of (a)
patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed patent
rights in respect thereof. As of the date of publication of this document, ISO had not received notice of (a)
patent(s) which may be required to implement this document. However, implementers are cautioned that
this may not represent the latest information, which may be obtained from the patent database available at
www.iso.org/patents. ISO shall not be held responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 159, Ergonomics, Subcommittee SC 4,
Ergonomics of human-system interaction.
This first edition of ISO 25062 cancels and replaces the first edition ISO/IEC 25062:2006, which has been
technically revised.
The main change is as follows:
— the scope of the document has been significantly expanded to go beyond usability test reports and deal
with all types of usability evaluations.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
iv
Introduction
This document provides a framework and consistent terminology for reporting the results of usability
evaluations of an interactive system. It is intended to assist those who perform usability evaluations in
documenting and communicating the results of usability evaluations as part of the system development
lifecycle.
NOTE ISO/IEC 25040 specifies a framework for quality evaluation.
The human-centred design approach of ISO 9241-210 is well established and focuses specifically on
making systems usable. Usability can be achieved by applying human-centred design throughout
the system development lifecycle. It is important that all the relevant types of information related to
usability (information items) are identified and communicated as part of a human-centred approach. The
identification and communication of relevant types of information related to usability enables the design
and testing of the usability of a system.
The information items for reporting the results of usability evaluations of an interactive system can be
integrated in any process models. For the purpose of establishing process models, ISO/IEC/IEEE 24774
and ISO/IEC TS 33061 specify the format and conformity requirements for process models, respectively.
In addition, ISO/IEC/IEEE 15289 defines the types and content of information items developed and used in
process models for system and software life cycle management. ISO/IEC TS 33060 and ISO/IEC TS 33061
define work products, including information items, for the purpose of process capability assessment. Process
models and associated information items for human-centred design of interactive systems are contained in
ISO 9241-210 and ISO TS 18152, respectively.
The common industry format (CIF) for usability documents are part of the SQuaRE (Systems and software
Quality Requirements and Evaluation) group of standards developed by ISO/TC 159, and described
in ISO/IEC 25000, Systems and software engineering — Systems and software Quality Requirements and
Evaluation (SQuaRE) — Guide to SQuaRE.
The CIF documents use definitions that are consistent with the ISO 9241 series on ergonomics of human-
system interaction, as this is the terminology that is normally used for this subject matter.
ISO TR 25060 gives an overview of the CIF standards, which include documents covering the following
information items:
— reporting usability evaluations (ISO 25062);
— context of use description (ISO/IEC 25063);
— user needs report (ISO/IEC 25064);
— user requirements specification (ISO 25065).
Table 1 presents an overview of the structure and the contents of the SQuaRE standards.
Table 1 — Organization of the SQuaRE series
SQuaRE architecture and sub-projects
ISO/IEC 25030 covering quality ISO/IEC 25010, ISO/IEC 25011, ISO/IEC 25012, ISO/IEC 25040 covering quali-
requirements and ISO/IEC 25019 covering quality models ty evaluation
ISO/IEC 25001 covering quality management
ISO/IEC 25020, ISO/IEC 25021, ISO/IEC 25022,
ISO/IEC 25023, ISO/IEC 25024 and ISO/
IEC 25025 covering quality measurement
Future standards are planned, covering the following, outside of the current architecture:
— requirements for quality of ready to use software products (RUSP);
— the common industry format (CIF) for usability division.
v
While this document specifies the minimum content of the various types of usability evaluation reports,
ISO 9241-220 introduces the human-centred design processes including:
— identifying the context of use;
— identifying user needs;
— specifying the user requirements;
— specifying the user-system interaction;
— producing and refining user interface design solutions;
— evaluating user-centred design.
Table 2 illustrates the interdependence of these information items with the human-centred design
processes described in ISO 9241-220, as well as the corresponding system life cycle processes described in
ISO/IEC/IEEE 15288.
Table 2 — Relationship of CIF documents to ISO 9241-220 and ISO/IEC/IEEE 15288
Human-centred design (HCD) CIF International Standards System lifecycle processes
processes
ISO/IEC/IEEE 15288:2023
ISO 9241-220:2019
9.4.3 — Identify the context of ISO/IEC 25063: Common Industry Format (CIF) for 6.4.2 b) 1) — Define context of
use usability: Context of use description use
9.4.4.2 — Identify user needs ISO/IEC 25064: Common Industry Format (CIF) for 6.4.2 b) 2) — Identify stake-
usability: User needs report holder needs
9.4.4.3 — Specify the user re- ISO 25065: Common Industry Format (CIF) for Usa- 6.4.3 — System requirements
quirements bility: User requirements specification definition process
9.4.5.2 — Specify the user-sys- ISO 25067: Common Industry Format (CIF) for 6.4.4 — Architecture definition
tem interaction Usability: User interaction and user interface speci- process
fication
9.4.5.3 — Produce and refine (6.4.5) Design definition pro-
user interface design solutions cess
9.4.6 — User-centred Evalua- ISO 25062: Common Industry Format (CIF) for 6.4.9 — Verification process
tion Usability: Reporting usability evaluations (this
6.4.11 — Validation process
document)
vi
International Standard ISO 25062:2025(en)
Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE) — Common
Industry Format (CIF) for reporting usability evaluations
1 Scope
This document describes the common industry format (CIF) for reporting usability evaluations. It provides
a classification of evaluation approaches and the specifications for the content items (content elements) to
be included in an evaluation report based on the selected evaluation approach(es). The intended users of the
usability evaluation reports are identified, as well as the situations in which the usability evaluation report
can be applied.
The usability evaluation reports in this document are applicable to software and hardware systems and
products or services used for predefined tasks (excluding generic products, such as a display screen or a
keyboard). The content elements are intended to be used as part of system-level documentation resulting
from development processes such as those in ISO 9241-210 and the ISO/IEC JTC 1/SC 7 process standards
(e.g. ISO/IEC/IEEE 15288 on systems lifecycle management, ISO/IEC/IEEE 12207 on software lifecycle
management and ISO/IEC 33001 on process capability assessment).
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
accessibility
extent to which products, systems, services, environments and facilities can be used by people from a
population with the widest range of user (3.19) needs, characteristics and capabilities to achieve identified
goals (3.6) in identified contexts of use
Note 1 to entry: Context of use (3.3) includes direct use or use supported by assistive technologies.
[SOURCE: ISO 9241-112:2017, 3.15]
3.2
conformity assessment
demonstration that specified requirements (3.9) are fulfilled
[SOURCE: ISO/IEC 17000:2020, 4.1, modified — Notes to entry deleted.]
3.3
context of use
combination of users (3.19), goals (3.6), tasks (3.13), resources, and environment
Note 1 to entry: The “environment” in a context of use includes the technical, physical, social, cultural and
organizational environments.
[SOURCE: ISO 9241-11:2018, 3.1.15]
3.4
effectiveness
accuracy and completeness with which users (3.19) achieve specified goals (3.6)
[SOURCE: ISO 9241-11:2018, 3.1.12]
3.5
efficiency
resources used in relation to the results achieved
Note 1 to entry: Typical resources include time, human effort, costs and materials.
[SOURCE: ISO 9241-11:2018, 3.1.13]
3.6
goal
intended outcome
[SOURCE: ISO 9241-11:2018, 3.1.10]
3.7
harm from use
negative consequences regarding health, safety, finances or the environment that result from use of the system
Note 1 to entry: The negative consequences can be for the user (3.19) or for any other stakeholder (3.11).
Note 2 to entry: Although avoidance of harm from use, i.e. eliminating any exposure of risk that poses a potential harm,
cannot be achieved completely, designing an interactive system (3.12) can aim at mitigating risks to an acceptable
minimum.
[SOURCE: ISO 9241-220:2019, 3.10]
3.8
information item
separately identifiable body of information that is produced and stored for human use during a system or
software life cycle
Note 1 to entry: A document produced to meet information requirements (3.9) can be an information item, part of an
information item, or a combination of several information items.
Note 2 to entry: An information item can be produced in several versions during a project or system life cycle.
[SOURCE: ISO/IEC/IEEE 15289:2019, 3.1.12]
3.9
requirement
condition or capability that must be met or possessed by a system, system component, product, or service to
satisfy an agreement, standard, specification, or other formally imposed documents
Note 1 to entry: Requirements provide value when delivered, satisfied, or met.
Note 2 to entry: Requirements include the quantified and documented needs, wants, and expectations of the sponsor,
customer, and other stakeholders (3.11).
[SOURCE: ISO/IEC/IEEE 24765:2017, 3.3431, definition 2, modified — Notes to entry added.]
3.10
satisfaction
extent to which the user's (3.19) physical, cognitive and emotional responses that result from the use of a
system, product or service meet the user’s needs and expectations
Note 1 to entry: Satisfaction includes the extent to which the user experience that results from actual use meets the
user’s needs and expectations.
Note 2 to entry: Anticipated use can influence satisfaction with actual use.
[SOURCE: ISO 9241-11:2018, 3.1.14]
3.11
stakeholder
individual or organization having a right, share, claim, or interest in a system or in its possession of
characteristics that meet their needs and expectations
EXAMPLE End users (3.19), end user organizations, supporters, developers, customers, producers, trainers,
maintainers, disposers, acquirers, suppliers, regulatory bodies, and people influenced positively or negatively by a system.
Note 1 to entry: Some stakeholders can have interests that oppose each other or oppose the system.
[SOURCE: ISO/IEC/IEEE 15288:2023, 3.44]
3.12
interactive system
combination of hardware and/or software and/or services and/or people that users (3.19) interact with in
order to achieve specific goals (3.6)
Note 1 to entry: This includes, where appropriate, packaging, user documentation, online and human help, support
and training.
Note 2 to entry: This definition emphasizes that the user interacts with the system. An interactive system provides
feedback to user input and initiates further actions within the system or by other systems as required.
[SOURCE: ISO 25065:2019, 3.2.1]
3.13
task
set of activities undertaken in order to achieve a specific goal (3.6)
[SOURCE: ISO 9241-11:2018, 3.1.11, modified — Notes to entry deleted.]
3.14
usability
extent to which a system, product or service can be used by specified users (3.19) to achieve specified goals
(3.6) with effectiveness (3.4), efficiency (3.5) and satisfaction (3.10) in a specified context of use (3.3)
[SOURCE: ISO 9241-11:2018, 3.1.1, modified — Notes to entry deleted.]
3.15
usability defect
attribute of the interactive system (3.12) that leads to a usability problem (3.17)
Note 1 to entry: Usability defects are typically identified during inspections.
Note 2 to entry: Usability defects can originate from incorrect or missing interaction capabilities as well as poor task
(3.13) support.
Note 3 to entry: Typical usability defects include the following:
— additional unnecessary steps not required as part of completing a task;
— misleading information;
— insufficient and/or poor information on the user interface;
— unexpected system responses;
— limitations in navigation;
— inefficient use error (3.18) recovery mechanisms;
— physical characteristics of the user interface that are not suitable for the physical characteristics of the user.
3.16
usability finding
identified usability defect (3.15) and/or usability problem (3.17) or positive usability-related (3.14) attribute
Note 1 to entry: Deviations of attributes of the interactive system (3.12) from specified criteria such as user (3.19)
requirements (3.9), principles, design guidelines or established conventions are also usability findings.
Note 2 to entry: Usability findings can lead to the identification of new user requirements (3.21).
3.17
usability problem
situation during use, resulting in poor effectiveness (3.4), efficiency (3.5) or satisfaction (3.10)
Note 1 to entry: Usability problems can be either directly observed during qualitative and/or quantitative usability
(3.14) tests or are identified from analysis of use.
Note 2 to entry: Usability problems can lead to one or more use errors (3.18).
Note 3 to entry: Usability problems are sometimes referred to as use difficulties.
3.18
use error
user (3.19) action or lack of user action while using the interactive system (3.12) that leads to a different
result than that intended by the manufacturer or expected by the user
[SOURCE: IEC 62366-1:2015, 3.21, modified — "Medical device" replaced by "interactive system", notes to
entry deleted.]
3.19
user
person who interacts with a system, product or service
Note 1 to entry: Users include people who operate a system, people who use the output provided by a system and
people who conduct support tasks (3.13) using the system (including maintenance and training).
Note 2 to entry: According to ISO/IEC 25010, "user" is defined as “individual or group that interacts with a system or
benefits from a system during its utilization”.
Note 3 to entry: Primary and secondary users interact with a system, and primary and indirect users can benefit from
a system. This definition includes a broader understanding of individuals and organizations that act as users.
[SOURCE: ISO 9241-11:2018, 3.1.7, modified — Notes 2 and 3 to entry added.]
3.20
user need
prerequisite identified as necessary for a user (3.19), or a set of users, to achieve an intended outcome,
implied or stated within a specific context of use (3.3)
EXAMPLE 1 A presenter (user) needs to know how much time is left (prerequisite) in order to complete the
presentation in time (intended outcome) during a presentation with a fixed time limit (context of use).
EXAMPLE 2 An account manager (user) needs to know the number of invoices received and their amounts
(prerequisite), in order to complete the daily accounting log (intended outcome) as part of monitoring the cash flow
(context of use).
Note 1 to entry: A user need is independent of any proposed solution for that need.
Note 2 to entry: User needs are identified based on various approaches including interviews with users, observations,
surveys, evaluations, expert analysis, etc.
Note 3 to entry: User needs often represent gaps (or discrepancies) between what should be and what is.
Note 4 to entry: User needs are transformed into user requirements (3.21) considering the context of use, user
priorities, trade-offs with other system requirements (3.9) and constraints.
[SOURCE: ISO/IEC 25064:2013, 4.19]
3.21
user requirements
set of requirements (3.9) for use that provide the basis for design and evaluation of interactive systems (3.12)
to meet identified user needs (3.20)
Note 1 to entry: User requirements are derived from user needs and capabilities in order to allow the user (3.19) to
make use of the system in an effective, efficient, safe and satisfying manner.
Note 2 to entry: User requirements are not requirements on the users.
Note 3 to entry: User requirements include user-system interaction requirements (3.22) and use-related quality
requirements (3.23).
Note 4 to entry: In software engineering terms, user requirements include both "functional" and "non-functional"
requirements derived from user needs and capabilities.
[SOURCE: ISO 9241-220:2019, 3.46]
3.22
user-system interaction requirements
user requirements (3.21) that specify interactions (including: recognizing information, making inputs,
making selections, and receiving outputs) required by the users (3.19) to achieve the goals (3.6)
Note 1 to entry: User-system interaction requirements (3.9) are generally stated in qualitative terms.
[SOURCE: ISO 25065:2019, 3.1.11, modified — Note to entry added.]
3.23
use-related quality requirements
user requirements (3.21) that specify the intended outcomes of use of the interactive system (3.12) and
associated quality criteria
Note 1 to entry: Use-related quality requirements (3.9) are generally stated in quantitative terms.
[SOURCE: ISO 25065:2019, 3.1.12, modified — Note to entry added.]
4 Outcomes and types of usability evaluations
4.1 Outcomes of a usability evaluation
Usability evaluations can produce a variety of outcomes, including:
— determining whether specified user requirements have been implemented;
— actual usability findings (negative and/or positive);
— identifying usability defects that cause usability problems;
— provision of performance data;
— compilation of subjective attitudes of users about the usability of an interactive system;
— provision of the basis for a procurement decision.
The content of a usability evaluation report vary based on the goals of the evaluation. The evaluation report
should provide sufficient information to assess the validity of the evaluation results.
4.2 Types of usability evaluation
Usability evaluation is a systematic process using one of the following types of evaluation approaches. The
content of an evaluation report depends on the type of evaluation approach used.
a) Inspection to identify:
— usability defects, i.e. deviations of the object of evaluation from specified criteria such as user
requirements, principles, design guidelines or established conventions;
— potential usability problems and their impacts in terms of harm from use.
b) Usability tests including:
— qualitative usability tests observing user behaviour to identify actual usability problems;
— quantitative usability tests measuring user performance and response (e.g. time taken to perform a
task, number of use errors, skin conductance or eye pupil dilation).
NOTE 1 Usability tests can be carried out in a simulated use environment or conducted in a “real life”
setting.
c) User surveys including:
— eliciting problems, opinions and perceptions and responses from users (i.e. qualitative user surveys);
— measuring level of user satisfaction (e.g. rating scale values for perceived satisfaction, effectiveness,
efficiency, aesthetics) (i.e. quantitative user surveys);
— other user reported data (e.g. data collected from an individual in conjunction with observation data).
NOTE 2 Collection of information about participants, such as demographic data, does not constitute a
user survey, but can be used to support interpretation.
A usability evaluation report contains information about one or more types of the evaluations listed above.
EXAMPLE 1 A usability test report describes problems encountered by users when carrying out tasks (type of
information is “observing user behaviour”). A quantitative usability test report contains measures of effectiveness,
efficiency and satisfaction (types of information are “measuring user performance” and “user survey”).
When reporting findings in usability evaluation reports, usability defects shall be differentiated from their
consequences. While usability defects are typically inappropriate attributes of the interactive system, their
consequences describe the negative effect on the user that is either likely to occur or has been observed or
reported.
EXAMPLE 2 A usability defect can be the fact that within a web form, required entry fields are not marked as
such. The consequences can possibly be that users fail to fill in required entry fields and therefore make use errors
repeatedly.
4.3 Assessing conformity of the object of evaluation against specified evaluation criteria
Evaluation report data can be used for different purposes. One purpose is to show that the object of
evaluation meets specified requirements, also referred to as conformance criteria. A conformity assessment
of the object of evaluation against specified criteria is defined in ISO/IEC 17000 as a “demonstration that
specified requirements relating to a product, process, system, person or body are fulfilled”. Assessment
of conformity consists of comparing the evaluation results with pre-defined conformance criteria. The
conformance criteria can be defined within a project or by a third party (e.g. a regulatory body). A rigorous
evaluation is required to produce data that can be used for a conformity assessment. When a conformity
assessment is used, it shall be documented in conformance with the requirements of this document.
NOTE 1 A formal conformity assessment involves a defined “conformity assessment scheme”. The formal scheme
provides:
a) legal defensibility;
b) evidence of contractual compliance; and
c) consistency of application and comparability of results across assessors and organizations.
Conformity assessment schemes are implemented at an international, regional, national and sub-national level.
The conformity assessment can be included in a usability evaluation report or can be issued as a separate
“conformity assessment report”. Table 3 shows the different types of conformance criteria that can be
specified as the basis for a conformity assessment. There can be various sets of specified conformance
criteria for one conformity assessment if the underlying evaluation consisted of more than one type of
evaluation (e.g. inspection plus user observation plus user survey).
Table 3 — Types of usability evaluations and corresponding conformance criteria
Type of usability evaluation Types of specified evaluation criteria to be evaluated against
Inspection — User-system interaction requirements
e.g. “with the system, the user shall be able to select flights by duration” or
“with the system, the user shall be able to select items from the last pur-
chase for the next purchase.”
and/or
— Principles
e.g. “use error robustness.”
and/or
— Guidelines and regulations
e.g. “required entry fields shall be distinct from optional entry fields” or
“the system shall be compatible with screen readers.”
and/or
— Design conventions
e.g. “the edit-button is always located at the top-right corner of the form.”
Qualitative test — User-system interaction requirements
e.g. “with the system, the user shall be able to select flights by duration” or
“with the system, the user shall be able to select items from the last pur-
chase for the next purchase.”
Quantitative test — Use-related quality requirements
e.g. “95 % of all users shall be able to successfully complete the authoriza-
tion process without assistance.”
User survey — Specified scores for user-reported subjective measures of effectiveness, effi-
ciency, satisfaction
e.g."3,5 on a scale ranging from 1 (min) to 5 (max)."
and/or
— Reported severity of usability-problems
e.g. “if a reported usability problem is judged as unacceptable by a user in
the survey group, then the object of evaluation fails the conformity assess-
ment.”
Principles and guidelines that can be used as conformance criteria are published in various sources including
the ISO 9241 series. These principles and guidelines often apply across operating systems and development
environments (e.g. “Colour should not be used as the only means to code information” or “Required entry
fields should be distinct from optional entry fields”).
Recommendations related to user interaction can be found in the following parts of the ISO 9241 series:
— ISO 9241-13 — User guidance.
— ISO 9241-14 — Menu dialogues.
— ISO 9241-110 — Interaction principles.
— ISO 9241-112 — Principles for the presentation of information.
— ISO 9241-125 — Guidance on visual presentation of information.
— ISO/TS 9241-126 — Guidance on auditory presentation of information.
— ISO 9241-129 — Guidance on software individualization.
— ISO 9241-143 — Forms.
— ISO 9241-154 — Interactive voice response (IVR) applications.
— ISO 9241-161 — Visual user interface elements.
— ISO 9241-171 — Guidance on software accessibility.
— ISO 9241-303 — Requirements for electronic visual displays.
— ISO 9241-400 — Principles and requirements for physical input devices.
— ISO 9241-410 — Design criteria for physical input devices.
— ISO 9241-500 — Ergonomic principles for the design and evaluation of environments of interactive
systems.
— ISO/TR 9241-810 — Robotic, intelligent and autonomous systems.
— ISO 9241-820 — Ergonomic guidance on interactions in immersive environments including augmented
reality and virtual reality.
— ISO 9241-910 — Framework for tactile and haptic interaction.
— ISO 9241-920 — Guidance on tactile and haptic interactions.
There are other international standards containing guidance on user-system-interaction and user interfaces
that can be used as conformance criteria and can be found at www .iso .org and other organizations.
NOTE 2 Established conventions can also be used as conformance criteria. These established conventions typically
include rules published by suppliers of operating systems or development environments within their design guidelines.
EXAMPLE An example for an established convention is “a dialogue box always has an 'OK' and 'cancel' button at
the bottom right corner of the dialogue box”.
5 Conformance
An evaluation report conforms to this document if it contains all the required content elements in Clause 7
that are applicable to the type of evaluation.
6 Overview of content elements within an evaluation report
The content elements in Clause 6 of this document for documenting evaluations can be integrated in any
type of process model.
NOTE Process models and associated information items for human-centred design of interactive systems are
contained in ISO 9241-220 and ISO/TS 18152 which specify human-centred design processes and ISO 9241-221 which
specifies the assessment of capability in human-centred design.
Annex A provides a checklist that can be used to ensure inclusion of required and recommended information.
Annex B provides an evaluation report template that can be used to develop an evaluation report based on
the requirements and recommendations in this document.
A usability evaluation report shall include the following sections:
— Executive summary.
— Description of the object of evaluation.
— Purpose of evaluation.
— Evaluation methodology.
— Data analysis and results.
A usability evaluation report should include the following sections:
— Recommendations.
— Conclusions.
— Appendix.
The content elements for each section of an evaluation report are determined by the type(s) of evaluation
to be conducted. Furthermore, there are elements that are always required and conditional elements that
can be selected for the evaluation, depending on whether they are used (e.g. statistical analysis or provided
recommendations) or applicable (e.g. parts of the object evaluated or measures used in the evaluation).
Evaluations can contain more than one type of evaluation (e.g. usability test and subsequent user survey).
As a result, the evaluation report will include the content elements for both types of evaluation.
The order in which the sections and the elements within the evaluation report are introduced does not
prescribe a required order for a usability evaluation report. Furthermore, the grouping of the content
elements themselves can be defined by the author of the report (e.g. combining information such as methods
and procedures into one section of the evaluation report).
Annex A contains a table that gives an overview of all required and recommended content elements for each
type of evaluation.
Subclauses 7.1 to 7.8 describe the content elements that are included in a usability evaluation report. The
report sections refer to all three types of evaluation (inspection, usability test and user survey).
7 Description of content elements within an evaluation report
7.1 Executive summary
This section of the usability evaluation report provides a concise overview of the evaluation. This section
is intended to provide information for those who will potentially not read the technical body of the report.
The executive summary shall include:
a) name and description of the object of evaluation;
b) summary of method(s) and the procedure;
c) summary of results including key findings, and any recommendations and related conclusions.
NOTE 1 Sometimes, only an executive summary is produced based on a performed evaluation. However, that is not
a substitute for an evaluation report as defined in this document.
NOTE 2 Executive summaries can provide the basis for presentations about the usability evaluation.
7.2 Description of the object of evaluation
This section of the usability evaluation report identifies the entity which was actually evaluated.
NOTE Examples of objects of evaluation include concepts, user interface prototypes, functioning software and
hardware products, or components of an interactive system.
Information about the object of evaluation shall include:
a) formal name and release or version;
b) brief description of the object of evaluation, the purpose of the object, and the intended use for which
the object was evaluated;
c) parts of the object that were evaluated (if only parts were evaluated);
d) prior usability evaluation report summaries (if applicable).
7.3 Purpose of the usability evaluation
This section of the usability evaluation report identifies the reasons for which the evaluation was conducted,
and why only specific parts of the object were evaluated (if applicable).
NOTE 1 Purposes for an evaluation can include:
— improving design by providing feedback into the design process;
— identifying usability defects and usability problems;
— confirming/identifying user requirements;
— confirming assumptions;
— testing concepts;
— measuring the level of usability (i.e. effectiveness and/or efficiency and/or user satisfaction);
— establishing benchmarks;
— assessing whether a product, system or service meets specific conformance criteria/acceptance criteria;
— identifying strengths and weaknesses of a product, system or service;
— identifying the consequences that could arise from poor usability;
— resolving disputes between users and/or stakeholders;
— identifying whether a product, system or service is accessible;
— acquiring a certification:
— to pass an internal quality decision point;
— to pass a certification of an external certification body.
7.4 Evaluation methodology
7.4.1 General
This section of the evaluation report describes the required content elements for documenting the
methodology with which the evaluation was designed and conducted.
NOTE Further content elements related to the evaluation methodology can be placed in the appendix (see 7.8).
7.4.2 Type(s) of evaluation used
The methodology section of the evaluation report shall state which type(s) of evaluation have been used.
NOTE Types
...








Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...