Establishing a web site environment-controlled three-layer monitoring system for observational quality assessment of web analytics measurement

Background Client-side web analytics is often interpreted as if recorded events directly reflect user behavior, although actual measurement depends on multiple observable stages. Because Google Analytics 4 (GA4) and Google Tag Manager (GTM) function as black boxes from the analyst side, practical improvement requires focusing on controllable site-side conditions. This study examined whether disagreement across observable layers becomes visible under controlled conditions and whether that profile varies by dwell time, click type, operating system, and browser. Methods A Three-Layer Monitoring System (3LMS) was developed to record dataLayer events, GA4-related payload requests, and event counts displayed in the GA4 interface. Using Python/Playwright, 900 automated trials were conducted on a minimal test site across three click types and three dwell-time conditions under Windows 10 + Chrome, Windows 11 + Chrome, and Windows 11 + Firefox. Pairwise layer comparisons were evaluated statistically where applicable. Supplementary analyses added pseudo-tag logging, URL-change monitoring, and ultra-short dwell-time conditions in Windows 11 + Chrome. Results Disagreement was most evident under short dwell times. In both Chrome environments, the 1 s condition produced the largest discrepancies, whereas agreement improved at 2 s and was largely complete at 3 s. Under Windows 10 + Chrome, retention at 1 s declined progressively from the dataLayer to the payload layer and then to the GA4 layer. Under Windows 11 + Chrome, the 1 s discrepancy was concentrated mainly at the final GA4 layer despite complete upstream retention. Under Windows 11 + Firefox, upstream monitoring appeared incomplete despite complete downstream GA4 retention, indicating that cross-browser comparability of observability cannot be assumed automatically. Conclusions This study developed a 3LMS as a quality assurance (QA) framework for examining disagreement across publicly observable stages of the GA4/GTM measurement pathway under controlled site-side conditions. The findings support its use as a bounded comparative framework within the tested environment. The 3LMS may serve as a bounded observational QA framework for evaluating site-side conditions associated with layer-wise disagreement in GA4/GTM measurement.