Benchmarking Inference Efficiency of Sub-3B Open-Weight Language Models on Commodity GPUs

This preprint presents a reproducible methodology and an open benchmarking script for measuring the inference efficiency of three sub-3B open-weight instruction-tuned large language models on a free NVIDIA T4 GPU provided by Google Colab. The three models evaluated are meta-llama/Llama-3. 2-1B-Instruct (1. 24B parameters), Qwen/Qwen2. 5-1. 5B-Instruct (1. 54B parameters), and google/gemma-2-2b-it (2. 61B parameters), all loaded in BF16 precision with the default Hugging Face Transformers backend. Three metrics are reported for each model: (1) peak VRAM, measured via torch. cuda. maxₘemoryₐllocated () ; (2) time-to-first-token (TTFT), measured as the wall-clock latency from input submission to the first generated token; and (3) tokens per second (TPS), measured as the steady-state decode-phase throughput with the prefill phase excluded. To ensure reproducibility, the protocol uses three warm-up runs and three measured runs per model, reports the median, and uses greedy decoding for bit-identical token sequences across sessions. The numerical values reported in the preprint's results table are reference estimates derived from the published parameter counts of each model, the BF16 weight size (2 bytes per parameter), the T4 GPU's published memory bandwidth (320 GB/s), and the attention-head configurations described in each model's technical report. They are clearly marked with a dagger in the preprint and are intended as a transparent starting point. The companion Colab script (benchmarkcolab. py) produces measured values on a real T4 GPU, which should replace the reference estimates before the preprint is cited in downstream work. The companion artifact includes the complete Python benchmarking script ready to paste into a Google Colab notebook, a 6-page preprint compiled into a PDF with three figures and a clear notice explaining the source of the numbers, matplotlib scripts for regenerating the figures, and a portfolio of supporting text assets. The methodology is designed to be reproducible by any reader with a Google account and a free Hugging Face token in under fifteen minutes at zero cost. The preprint discusses the architectural reasons behind the observed trade-offs between parameter count, KV-cache memory, and decode throughput, including the effect of grouped-query attention in Llama-3. 2 and Qwen2. 5 and the sliding-window attention scheme in Gemma-2-2B-it. Limitations: only the default Transformers backend is benchmarked (no vLLM or TensorRT-LLM), only a single prompt length is tested, and absolute TPS values are not directly transferable to HBM-equipped data-center GPUs, although the relative ranking of the three models is expected to be preserved.