Have We Controlled Properly in Public Administration Research? Findings Based on a Systematic Literature Review

Quantitative research methodologies have gained significant traction in the field of public administration research, garnering attention for their systematic and objective approach to understanding complex phenomena. Despite its growing popularity, quantitative research has not been without its share of criticism, with a central point of contention being the challenges associated with ensuring its reliability and validity. Scholars argue that while quantitative methods offer a structured framework for analysis, they also present unique obstacles that may compromise the integrity of the research outcomes. A key strategy employed in quantitative research to strengthen the precision of estimating the relationships among core variables is the utilization of statistical techniques to account for extraneous variables. By controlling for these extraneous factors, researchers seek to isolate the effects of any variables of interest, thereby enhancing the accuracy of their findings. However, the effectiveness of this approach hinges on the proper selection and application of the control variables. In practice, the use of control variables is fraught with challenges stemming from ambiguous methodological guidelines and potential oversights by the researcher. These challenges often lead to errors in the implementation of the control variables, which may cast a shadow over the validity of the entire study. This study presents an extensive analysis of 760 articles published across eight well-regarded public administration journals over the decade from 2014 to 2023. The focus of the analysis is on identifying the limitations as well as other issues related to the selection, analysis, and reporting of the control variables in these studies. The findings reveal a concerning pattern of methodological shortcomings. First, the selection of the control variables frequently lacks a solid theoretical foundation, with variables often included as a matter of routine rather than based on a clear rationale linked to the research question. Second, there is a notable homogenization in the choice of the control variables, with researchers tending to rely on a standard set of variables without adequately considering the unique context of their studies or exploring alternative variables that may yield deeper insights. Moreover, this study highlights the frequent omission of multicollinearity tests, which are crucial for ensuring that the control variables do not introduce noise into the model by being correlated with one another. The absence of such tests may lead to distorted results and incorrect inferences. In addition, many studies fail to provide comparative analytical results, which are essential for contextualizing the findings and understanding the relative impact of the various variables. The reporting of the control variables is also often incomplete, leaving readers in the dark about the rationale behind their inclusion and the methods used to analyze them. To address these identified gaps, this study offers a set of comprehensive recommendations for the proper use of control variables. These recommendations are designed to help researchers navigate the complexities of quantitative research designs and to enhance the robustness of their studies. By adhering to these guidelines, researchers can significantly improve the reliability and validity of public administration research. This study underscores the importance of careful and thoughtful methodological practices in an era when quantitative research continues to play a pivotal role in advancing our understanding of public administration issues.