Metrics and measurement of trustworthy systems

Accurate measurement of the quality of systems is crucial to building trustworthy systems. Such a measurement indicates whether a system is working properly and meeting its requirements. Although security and dependability metrics are regarded as key metrics for measuring the quality of systems, they are not sufficient for measuring the quality of systems that are placed in a multi-domain environment including hardware, software, network, human factors, and physical environments. In order to embrace multidimensional aspects of the quality of a system, we introduce a trustworthiness metric framework that supports three key submetrics of trust, resilience, and agility, and propose an ontology-based framework with three corresponding sub-ontologies. We also discuss how the key metrics are related to the severity of threats and the quality of assessment tools. This work is part of the cyber defense effort conducted by the Trustworthy Systems Working Group (TSWG) under the Cyber Strategic Challenge Group (CSCG) of The Technical Cooperation Program (TTCP), which is an international cooperation organization for enhancing defense science and technology.