From Draft to Standard: An Expert-Validated Catalog of API Gateway Software Metrics

Context: In the fast-growing API Economy scenario, API Gateways became an essential tool for microservice reliability and scalability, yet lack standardized quality models and measures to assess these tools in an objective fashion. Objective: This study quantitatively and qualitatively assessed the accuracy and relevance of a~set of 59 metrics, gathered from prior analysis of 68 mainstream API Gateway software offerings. Method: An expert judgment evaluation study was conducted with seven domain specialists (N=7) from both academia and industry, who assessed each metric’s accuracy and relevance using 5-point Likert scales. A~multi-statistical method was performed to assess ratings, inter-observer consistency and reliability. Results: Although ratings were generally neutral to positive, statistical analysis revealed low consensus across experts. A~subset of 24 core metrics with strong agreement and high scores was identified, alongside moderate- and low-agreement sets. These items cover key API management concerns, including traffic monitoring, error handling, latency, and resource usage. Conclusions: By combining practical definitions, expert judgment, and a~tiered classification mechanism, this work established a~broad and comprehensive set of metrics that will help to bridge the gap between metric adoption in real-world platforms and their theoretical grounding in software quality models.