GPU programming has rapidly emerged as a crucial paradigm for accelerating computational workloads, particularly in the context of High-Performance Computing (HPC). This paper presents an integrated study on CUDA (Compute Unified Device Architecture), a parallel computing platform and programming model developed by NVIDIA, highlighting its relevance in achieving superior performance across various domains. The research explores CUDA architecture, programming models, memory hierarchy, and optimization strategies that transform general-purpose GPUs into powerful accelerators. Furthermore, we compare CPU and GPU performance through experimental vector addition programs and examine profiling tools and domainspecific applications. As a result of the comparison, it was noted that when we used smaller data the CPU performed about 0.14 times faster than the GPU whereas when we used a larger data for the same the GPU is almost 2.89 times faster than the CPU, which concludes that using the GPU for a large data is highly essential. The paper concludes with a discussion on scalability, challenges, and future advancements in CUDA-based HPC systems.
Jaltare et al. (Mon,) studied this question.