A Reproducible Comparative Study of PageRank Across the Hadoop Ecosystem

PageRank is a canonical workload for evaluating large-scale graph-processing systems, yet implementations written for different frameworks are rarely checked for numerical equivalence before their performance is compared. We present a reproducible, single-codebase study that pairs a dependency-free pure-Python reference engine with five Hadoop-ecosystem implementations of PageRank: Python mrjob (a local engine and a genuine single-pass MapReduce job), PySpark (RDD and DataFrame), Hadoop Streaming, Java MapReduce, and Apache Pig. The reference engine implements the standard, weighted, and personalized variants with correct dangling-node handling and matches networkx.pagerank to a maximum per-node error of 2.6e-12; every executed distributed implementation agrees with it to within 2.4e-8 (up to eight-decimal output precision), so any timing difference reflects engineering overhead rather than a different computation. A single configuration-driven harness measures wall-clock time, peak memory, and accuracy on synthetic Barabasi-Albert graphs, repeating each configuration three times. On a single multi-core node the two in-memory engines are fastest and scale near-linearly; the mrjob local engine adds only a 1.0-1.8x constant, the process-per-iteration Hadoop Streaming pipeline is 6-13x slower while using the least memory (~10 MB), the genuine single-pass mrjob MapReduce job is 36-82x slower, and PySpark's DataFrame engine is 190-1000x slower with a ~1.6 GB JVM footprint (its RDD counterpart slower still, ~59 s/iteration, dominated by per-iteration localCheckpoint). We analyze the architectural causes, document a self-normalization property, a single-pass dangling-mass limitation, and a Spark lineage-growth pitfall, and release the full toolchain so that cluster-scale evaluation of the remaining JVM frameworks is a single command. The study argues for a discipline of "validate first, then compare".