This is a replication package for a paper Assessing REST API Test Generation Strategies with Log Coverage, which is accepted for publication in Proceedings of the 30th International Conference on Evaluation and Assessment in Software Engineering (EASE 2026). The package consist of tests, logs, and log templates for Locust, EvoMaster, and LLM (GPT-5. 2-Codex and Claude Opus 4. 6) test strategies. In addition, the scripts for generating the tests with EvoMaster and LLMs are provided and they can be tested in sandbox-like environment. Instruction can be found from README. Each test generation strategy (Locust, EvoMaster, GPT, Claude Opus) has its own directory, which contains subdirectories for generated tests (except for Locust), per service raw logs, and per service parsed unique log templates for 10 separate runs executed during the study. For every artifact (tests, logs, and templates) the run number consistently refers to the same execution: for example, running run₁ tests produces run₁ logs, from which run₁ templates are derived. Locust directory does not include tests, as they were borrowed unchanged from a previous study. Similarly, the Locust logs originate from that study, and their file names have been preserved to make it easier to match selected runs with the original ones. Therefore, also template directories follow the same naming convention. Generating new tests in sandbox-like environemnt requires installing LO2 system locally to be able to execute tests against the system and collect the logs. The pipeline for test generation, running of tests, and log collection is essentially the same for each automated test generation strategy. The resulting test artifacts are stored in a sandbox directory for each test generation strategy, ensuring they remain separate from the tests generated during the study. For test generation, the entire content of this package needs to be placed in the root of LO2 system. Notes: The link goes to the fork of the original Light-OAuth2 system by networknt. This fork is used in the previous LO2 study from where the Locust tests comes. Installing the LO2 system requires i. a. MySQL, Locust, and Docker. See full list from the LO2 repository.
Reinikainen et al. (Fri,) studied this question.