Abstract A frequently asked question about government-funded research and development (R&D) projects is: What is the return on investment (ROI)? We initially addressed this question by conducting a benefit–cost–risk analysis (BCRA) and adapting it to 25 R&D projects funded by the Science and Technology Directorate (S&T) of the United States Department of Homeland Security (DHS). The net benefits and the associated benefit-to-cost ratios were mostly high for transitioned and used projects. Still, substantial uncertainty remained about the benefits of projects in transition that had not yet been implemented or put to use. Conducting BCRAs on cybersecurity R&D poses additional problems. First, many cybersecurity R&D projects are at a low technology readiness level (TRL). Second, the benefits of cybersecurity are the avoided risks and damages, which are very uncertain. Our objective is to provide a proof-of-concept demonstration of a novel BCRA methodology that accounts for uncertainties in the cybersecurity R&D domain. The innovative methodology described in this paper consists of conducting a BCRA, assuming that the cybersecurity R&D projects will be successfully transitioned and implemented, then discounting the net benefits by the probability of success. Both BCRAs employed a methodology that includes a decomposition of the projects' costs and benefits, characterization of uncertainty through subject-matter expert (SME) assessments, and Monte Carlo simulation via an Excel Add-In to account for uncertainty. We applied this methodology to two cybersecurity projects funded by the DHS. The primary benefit of the first project was automating malware detection, thereby saving time and labor costs. The benefits of the second project were to enhance early detection and removal of malware, thereby reducing the risks and costs of cyberattacks. The BCRA methodology provided estimates of the mean net benefit over 5 years and the benefit–cost ratio for each cybersecurity project. More importantly, the innovative BCRA methodology using Monte Carlo simulation yielded a distribution of net benefits for each project over its expected lifespan, as well as break-even analyses for both projects to achieve positive net benefits.
John et al. (Thu,) studied this question.