ABSTRACT It is well established that combining information via a simple algorithm (mechanical combination) results in better hiring decisions than a holistic combination. Nevertheless, mechanical combination is perceived negatively and rarely used in practice. Transparency is an often‐mentioned determinant of algorithmic trust in the AI‐literature, but has hardly received attention regarding mechanical judgment. Therefore, in four studies (total N = 1205), we experimentally investigated the effect of various operationalizations of transparency on the perceptions of simple, mechanical algorithmic hiring procedures among decision‐makers and applicants. Quantitative analyses were supplemented with thematic analyses of open‐ended responses. The original between‐subjects studies hardly showed differences between conditions, but open question responses indicated that this may have been due to an emphasis on “filler information.” Subsequent within‐subjects studies did show the expected positive effects of transparency, but mostly with small effect sizes. However, both applicants and decision‐makers indicated a strong preference for transparent algorithms over opaque algorithms and over holistic judgment when asked to choose directly. Moreover, unexpectedly, compared to holistic judgment, the applicant sample showed more positive perceptions, even for an opaque algorithm. Open question responses suggest that both groups generally perceived algorithms as fairer, but not more valid than holistic judgment, but the quantitative results did not fully support that pattern. In addition, the open responses suggest that while transparency is requested when not provided, it may also evoke more scrutiny when it is, which may not necessarily affect perceptions positively.
Niessen et al. (Mon,) studied this question.