PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
February 5, 20260 citations

Using Containers to Speed Up Development, to Run Integration Tests and to Teach About Distributed Systems

View Full Paper
DDDavid DykstraBCBruno Moreira CoimbraNUNamratha Urs

Key Points

  • The research aims to showcase how specialized containers can facilitate development, testing, and education in distributed systems.
  • Developed system-like containers to support development and testing
  • Built a base system with three main containers for compute and scheduling
  • Evaluated different container runtimes for performance and ease of use
  • Integrated the system with IDEs like VS Code for debugging and development
  • Used the system for training new team members and in educational workshops
  • Demonstrated that workspaces enable quick reconfiguration for testing changes
  • Showed improved onboarding experience for new developers
  • Provided students with hands-on experience in distributed system mechanisms
  • Assessed ease of use and performance of various container runtimes

Abstract

GlideinWMS is a workload manager provisioning resources for many experiments, including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development, we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we will talk about what differentiates workspaces from other containers. We will describe our base system, composed of three containers: a one-node cluster including a compute element and a batch system, a GlideinWMS Factory controlling pilot jobs, and a scheduler and Frontend to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop, and we will share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we will talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system even when offline. They simplified the training and onboarding of new team members and summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Dykstra et al. (2025) studied this question.

synapsesocial.com/papers/698434dff1d9ada3c1fb3879https://doi.org/10.1051/epjconf/202533701317/pdf
Ask AI
Helpful
Bookmark
Share
View Full Paper