ConfBench: A Tool for Easy Evaluation of Confidential Virtual Machines

Ensuring the security and confidentiality of cloud computing workloads is essential. To this end, major cloud providers offer computing instances based on trusted execution environments (TEEs) to support confidential computing in virtual machines. TEEs are hardware-based shielded environments building on technologies available today, such as Intel TDX or AMD SEV-SNP or that will soon be, as with ARM CCA.To lower the barriers to experimenting with these technologies for researchers and practitioners, we developed ConfBench, a tool for easy evaluation of confidential virtual machines. ConfBench supports both cloud-native workloads (Function-as-a-Service) and classic applications. ConfBench facilitates the management of the full lifecycle of such workloads, from their deployment to the gathering of performance metrics, taking into account the specifics of TEE-enabled confidential virtual machines. We use ConfBench to collect execution overhead measurements for different VM-enabled TEEs (Intel TDX and AMD SEV-SNP) through extensive experiments. We also showcase how ConfBench’s architecture allows for validating also simulation-based TEEs, reporting preliminary results with ARM CCA. We highlight the intrinsic overheads of such confidential VMs by conducting stress tests against machine learning inference tasks, DBMS and native-OS operations benchmarking, as well as by evaluating the costs of attestation operations required in the context of confidential computing. The results indicate generally tenable overheads with modern TEEs, with exceptions mainly from I/O-intensive tasks, especially with TDX. ConfBench’s multi-language support for FaaS workloads also lets us gain insights into differences stemming from varying complexities behind language runtimes. We release ConfBench to the research community and provide instructions to reproduce our experiments.