ISO/IEC TR 14143-3:2003

TECHNICAL ISO/IEC
REPORT TR
14143-3
First edition
2003-04-15
Information technology — Software
measurement — Functional size
measurement —
Part 3:
Verification of functional size
measurement methods
Technologies de l'information — Mesurage du logiciel — Mesurage de
la taille fonctionnelle —
Partie 3: Vérification des méthodes de mesure de la taille fonctionnelle

Reference number
©
ISO/IEC 2003
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2003
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2003 — All rights reserved

Contents Page
Foreword. iv
Introduction . vi
1 Scope. 1
2 Normative references. 1
3 Terms and definitions. 1
4 Verification. 4
4.1 General. 4
4.2 Constitute the verification team . 7
4.2.1 Determine the verification team's competencies . 7
4.2.2 Verification team responsibilities. 7
4.3 Assemble the verification inputs. 7
4.3.1 Prepare the verification input . 7
4.3.2 Gather the FSM Method documentation. 8
4.3.3 Compile the list of statements to be verified . 8
4.3.4 Create the list of test requests . 8
4.3.5 Plan the verification (Verification plan) . 9
4.4 Conduct the verification. 9
4.5 Compile the verification output . 9
Annex A (normative) Presentation of test requests . 11
A.1 General requirements of test requests . 11
A.2 Description of test requests. 11
A.2.1 Tests relating to repeatability and reproducibility of an FSM Method . 11
A.2.2 Tests relating to accuracy of an FSM Method. 11
A.2.3 Tests relating to convertibility of an FSM Method. 13
A.2.4 Tests relating to discrimination threshold of an FSM Method. 14
A.2.5 Tests relating to applicability (of an FSM Method) to a functional domain . 14
Annex B (normative) Verification methods. 15
B.1 Verification methods for assessment of repeatability and reproducibility of an FSM Method. 15
B.1.1 Repeatability. 15
B.1.2 Reproducibility. 15
B.2 Verification methods for assessment of accuracy of an FSM Method. 16
B.2.1 Verification for the accuracy of the application of an FSM Method. 16
B.2.2 Verification with respect to a theory as a reference. 16
B.3 Verification methods for convertibility of an FSM Method. 16
B.4 Verification methods for discrimination threshold of an FSM Method . 17
B.5 Verification methods for applicability (of an FSM Method) to Functional Domains . 18
Annex C (informative) Example of a verification report . 19
C.1 Introduction. 19
C.2 Executive summary. 19
C.3 Purpose or scope and context of the verification sponsor. 21
C.4 List of statements . 21
C.5 List of test requests . 22
C.6 Verification plan. 22
C.7 Results. 22
C.8 Analysis of the test results . 22
C.9 Evaluation of the correctness of the statements.22
Bibliography . 23

© ISO/IEC 2003 — All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
In exceptional circumstances, the joint technical committee may propose the publication of a Technical Report
of one of the following types:
— type 1, when the required support cannot be obtained for the publication of an International Standard,
despite repeated efforts;
— type 2, when the subject is still under technical development or where for any other reason there is the
future but not immediate possibility of an agreement on an International Standard;
— type 3, when the joint technical committee has collected data of a different kind from that which is
normally published as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether
they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to
be reviewed until the data they provide are considered to be no longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC TR 14143-3, which is a Technical Report of type 2, was prepared by Joint Technical Committee
ISO/IEC JTC 1, Information technology, Subcommittee SC 7, Software and system engineering.
This document is being issued in the Technical Report (type 2) series of publications (according to the
Procedures for the technical work of ISO/IEC JTC 1) as a “prospective standard for provisional application” in
the field of software measurement because there is an urgent need for guidance on how standards in this field
should be used to meet an identified need.
This document is not to be regarded as an “International Standard”. It is proposed for provisional application
so that information and experience of its use in practice may be gathered. Comments on the content of this
document should be sent to the ISO Central Secretariat.
A review of this Technical Report (type 2) will be carried out not later than three years after its publication with
the options of: extension for another three years; conversion into an International Standard; or withdrawal.
ISO/IEC 14143 consists of the following parts, under the general title Information technology — Software
measurement — Functional size measurement:
 Part 1: Definition of concepts
iv © ISO/IEC 2003 — All rights reserved

 Part 2: Conformity evaluation of software size measurement methods to ISO/IEC 14143-1:1998
 Part 3: Verification of functional size measurement methods
 Part 4: Reference model
 Part 5: Determination of functional domains for use with functional size measurement
© ISO/IEC 2003 — All rights reserved v

Introduction
Functional Size Measurement (FSM) is a technique used to measure the size of software by quantifying the
1)
Functional User Requirements of the software . The first published method to embrace this concept was
Function Point Analysis, developed by Allan Albrecht in the late 1970s. Since then, numerous extensions and
variations of the original method have been developed. The end user may have many variants from which to
choose - each with its own advantages in specific situations. This part of ISO/IEC 14143 was developed to
provide a process to assist the user in choosing a method appropriate to their needs, by providing a process
for verifying the extent to which statements made for certain performance properties of an FSM Method are
true.
Tests are carried out according to the provisions of this Technical Report, concerning statements made for
performance properties of a particular FSM Method. The results of these tests will be helpful to prospective
users of the FSM Method in judging whether it is appropriate to their needs.
ISO/IEC 14143-1:1998 was developed to define the concepts of FSM and provides a basis against which all
variants can be compared.
This part of ISO/IEC 14143
a) establishes a framework for verifying certain performance properties of an FSM Method,
b) defines several performance properties against which an FSM Method can be verified,
c) describes the types of tests which can be performed,
d) defines the process for verification of an FSM Method, and
e) provides an example template for the verification report.
Verification is conducted by a verification team that has the competencies described in this part of
ISO/IEC 14143. This part of ISO/IEC 14143 assumes familiarity with the concepts and definitions described in
ISO/IEC 14143-1:1998.
The verification process is designed to meet the requirements of the verification sponsor and involves
a) identifying the performance properties that need to be verified,
b) identifying the tests that need to be conducted,
c) conducting the tests, and
d) reporting the verification test results.

1) Refer ISO/IEC 14143-1:1998, Information technology — Software measurement — Functional size measurement —
Part 1: Definition of concepts.
vi © ISO/IEC 2003 — All rights reserved

The output from the verification is the verification report, which provides objective evidence of the extent to
which an FSM Method exhibits certain performance properties. The verification report consists of the results
for each test carried out, and can be used to
a) determine the correctness of the statements made by a particular FSM Method,
b) determine the extent that a particular FSM Method exhibits the particular performance properties tested,
and
c) assist prospective users of the FSM Method to make informed decisions about which method best meets
their needs.
© ISO/IEC 2003 — All rights reserved vii

TECHNICAL REPORT ISO/IEC TR 14143-3:2003(E)

Information technology — Software measurement — Functional
size measurement —
Part 3:
Verification of functional size measurement methods
1 Scope
This part of ISO/IEC 14143 establishes a framework for verifying the statements of an FSM Method and/or for
conducting tests requested by the verification sponsor, relative to the following performance properties:
 repeatability and reproducibility;
 accuracy;
 convertibility;
 discrimination threshold;
 applicability to Functional Domains.
NOTE Statements and test requests relative to other performance properties are outside the scope of this document.
It aims to ensure that the output from the verification is objective, impartial, consistent and repeatable.
The verification report, produced as a result of applying this part of ISO/IEC 14143, will enable the prospective
user to select the FSM Method which best meets their needs.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 14143-1:1998, Information technology — Software measurement — Functional size measurement —
Part 1: Definition of concepts
ISO/IEC 14143-2:2002, Information technology — Software measurement — Functional size measurement —
Part 2: Conformity evaluation of software size measurement methods to ISO/IEC 14143-1:1998
ISO/IEC 14143-4:2002, Information technology — Software measurement — Functional size measurement —
Part 4: Reference model
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 14143-1:1998 and the following
apply.
© ISO/IEC 2003 — All rights reserved 1

3.1
accuracy of measurement
the closeness of the agreement between the result of a measurement and the true value of the measurand
NOTE 1 “Accuracy” is a qualitative concept.
NOTE 2 The term precision should not be used for “accuracy”.
[International vocabulary of basic and general terms in metrology, 1993, definition 3.5]
NOTE 3 A true value, as defined in the ISO Vocabulary on basic and general terms in metrology (1993), is a value
consistent with the definition of a given particular quantity and this is a value that would be obtained by a perfect
measurement. In contexts where perfect measurement is not practically feasible, a conventional true value is a value
attributed to a particular quantity and accepted, sometimes by convention, as having an uncertainty appropriate for a given
purpose. ‘Conventional true value’, in the same reference, is sometimes called assigned value, best estimate of the value,
conventional value or reference value. The accuracy should be expressed in terms of the Mean magnitude of relative error.
3.2
applicability to a Functional Domain
the ability of an FSM Method to take into account the characteristics of Functional User Requirements (FUR)
which are pertinent to FSM in a Functional Domain
3.3
convertibility
the ability to convert the results from applying two or more FSM Methods in the measurement of a Functional
Size of the same set of Functional User Requirements
NOTE Convertibility of an FSM Method is a recommendation of ISO/IEC 14143-1:1998.
3.4
discrimination (threshold)
largest change in a stimulus that produces no detectable change in the response of a measuring instrument,
the change in the stimulus taking place slowly and monotonically
NOTE The discrimination threshold may depend on, for example, noise (internal or external) or friction. It may also
depend on the value of the stimulus.
[International vocabulary of basic and general terms in metrology, 1993, definition 5.11]
3.5
measurand
particular quantity subject to measurement
EXAMPLE Vapour pressure of a given sample of water at 20 °C.
NOTE 1 The specification of a measurand may require statements about quantities such as time, temperature and
pressure.
[International vocabulary of basic and general terms in metrology, 1993, definition 2.6]
NOTE 2 In this part of ISO/IEC 14143, the measurand refers to the FUR.
3.6
measuring instrument
device intended to be used to make measurements, alone or in conjunction with supplementary device(s)
[International vocabulary of basic and general terms in metrology, 1993, definition 4.1]
NOTE In this part of ISO/IEC 14143, the core element of the measurement instrument is the FSM Method.
2 © ISO/IEC 2003 — All rights reserved

3.7
owner of the FSM Method
the person or organization that owns the intellectual property rights for the FSM Method
3.8
repeatability (of results of measurements)
closeness of the agreement between the results of successive measurements of the same measurand carried
out under the same conditions of measurement
NOTE 1 These conditions are called repeatability conditions.
NOTE 2 Repeatability conditions include:
 the same measurement procedure
 the same observer
 the same measuring instrument, used under the same conditions
 the same location
 repetition over a short period of time.
NOTE 3 Repeatability may be expressed quantitatively in terms of the dispersion characteristics of the results.
[International vocabulary of basic and general terms in metrology, 1993, definition 3.6]
3.9
reproducibility (of results of measurements)
closeness of the agreement between the results of measurements of the same measurand carried out under
changed conditions of measurement
NOTE 1 A valid statement of reproducibility requires specification of the conditions changed.
NOTE 2 The changed conditions may include:
 principle of measurement
 method of measurement
 observer
 measuring instrument
 reference standard
 location
 conditions of use
 time.
NOTE 3 Reproducibility may be expressed quantitatively in terms of the dispersion characteristics of the results.
NOTE 4 Results are here usually understood to be corrected results.
[International vocabulary of basic and general terms in metrology, 1993, definition 3.7]
© ISO/IEC 2003 — All rights reserved 3

3.10
verification method
a method that tests an FSM Method, and provides objective evidence of the extent to which a particular
performance property is exhibited
NOTE The purpose of applying this part of ISO/IEC 14143 is to enable the user to select the FSM Method which
“best” meets their needs. Therefore, verification of an FSM Method should produce a result that indicates:
 the extent to which a performance property is exhibited, or
 whether a performance property is exhibited to a stated extent.
For this reason, there is no concept of “pass” or “fail”. An FSM Method can be considered to be either “verified” or “not
verified”, for a particular performance property, based on whether or not the appropriate verification has been conducted.
3.11
verification sponsor
the person or organization that requires the verification to be performed and provides financial or other
resources to carry it out
4 Verification
4.1 General
4.1.1 The purpose of verification is to provide objective evidence of the extent to which an FSM Method
exhibits certain performance properties. The level of acceptability may be dependent on the context and
purpose of the verification sponsor, e.g. one level might be acceptable for one purpose, but unacceptable for
another purpose.
4.1.2 Verification of an FSM Method shall be performed by a verification team with the objective of
a) determining the correctness of the statements of the FSM Method, and/or
b) applying tests requested by the verification sponsor.
4.1.3 Verification shall consist of the following activities:
a) constitute the verification team (a verification team is constituted based on identified competencies – refer
to 4.2);
b) assemble the verification inputs (the verification inputs are identified or produced – refer to 4.3);
c) conduct the verification (verification is conducted – refer to 4.4);
d) compile the verification output (a verification report is produced – refer to 4.5).
NOTE Figure 1 provides a diagrammatic representation (numbering within the diagram refers to clauses in this
document).
4.1.4 A verification report shall only be valid for the particular version of an FSM Method that was the
subject of the verification. Each version of a method, including a local customization, is considered to be
another FSM Method and shall be verified separately. If the verification team can identify the similarities
and/or differences between an FSM Method and a previously verified version of the same method, they may
use the results of that previous verification as the basis for the new verification. If any verification tests have
been reported for a previously verified version of the same FSM Method, then the verification team shall
consider such verification tests during the current verification.
NOTE If the verification team bases a verification of an FSM Method on a previously verified FSM Method, then they
need to be aware of the risks involved as the two versions may have differences that have not been noted. The verification
team must take care that the net effect of all the changes is taken into account during the verification.
4 © ISO/IEC 2003 — All rights reserved

4.1.5 The verification team shall verify that the FSM Method Documentation is complete, as defined in 4.3.2,
and correct for the version of the FSM Method being evaluated.
4.1.6 The verification team should liaise with the verification sponsor during the verification.
4.1.7 If the owner of the FSM Method can be contacted, then the verification team shall
a) liaise with the owner during the verification, and
b) document the subject of the liaison with the owner within the verification report and, where appropriate,
cross-reference the provision or verification activity to which it relates.
4.1.8 The verification team shall determine whether information received, from the owner of the FSM
Method during the liaison, would result in a different version of the method than that submitted for this
verification. In this case 4.1.4 shall apply.
4.1.9 If the owner of the FSM Method can be contacted, then the owner shall be provided with the
opportunity to respond to the findings of the verification and to add comments to the verification report before
its publication.
4.10 If the owner of the FSM Method does not respond to the findings of the verification report within a
reasonable time period, then the verification team may proceed with publication of the report. This time period
should be agreed upon by the owner of the FSM Method and the verification team at the outset of the
verification.
© ISO/IEC 2003 — All rights reserved 5

tasks
activities participants
Constitute the Determine the
verification team verification team’s
Verification sponsor
(4.2)
competencies
(4.2.1)
Gather the FSM Method
documentation
Verification sponsor
(4.3.2)
Verification team,
Assemble the
Compile the list of Create the list
verification sponsor,
verification inputs
statements to be verified of test requests owner of the FSM Method
(4.3)
(4.3.4)
(4.3.3)
Verification team,
verification sponsor,
owner of the FSM Method
Plan the verification
(verification plan)
Verification team and
(4.3.5)
verification sponsor
Conduct the
required tests
Verification team
(4.4-a)
Record the results
Conduct the
Verification team
(4.4-b)
verification
(4.4)
Analyse the test
results
Verification team
(4.4-c)
Review the results and
evaluate the correctness
Verification team
of statements, if any
(4.4-d)
Compile the
verification output Produce the
(4.5) verification report
Verification team
(4.5.1)
Figure 1 — Verification
6 © ISO/IEC 2003 — All rights reserved

4.2 Constitute the verification team
4.2.1 Determine the verification team's competencies
To verify an FSM Method in accordance with this part of ISO/IEC 14143, the verification team should have
a) knowledge of the concepts of all parts of ISO/IEC 14143,
b) a thorough understanding of the details of parts 3 and 4 of ISO/IEC 14143,
c) a thorough understanding of measurement concepts,
d) experience with the FSM Method to be verified, and
e) experience in the conduct of verification tests in accordance with National or International Standards in a
software environment (not necessarily under this part of ISO/IEC 14143).
NOTE Measurement concepts include
 statistical methods,
 measurement theory, and
 the concepts of software size measurement.
4.2.2 Verification team responsibilities
The verification team shall be responsible for ensuring that all activities in the verification are completed.
These activities shall include the following:
a) develop the verification plan;
b) define the verification procedure;
c) conduct the verification (apply the verification methods, record and analyse the test results, and evaluate
the correctness of any statements);
d) produce the verification report.
4.3 Assemble the verification inputs
4.3.1 Prepare the verification input
4.3.1.1 As a minimum, the input to the verificat
...