Discussing Hidden Bias in Observational Studies

In observational studies or nonrandomized experiments, treated and control groups may differ in their outcomes even if the treatment has no effect; this may happen if the groups were not comparable before the start of treatment. The groups may fail to be comparable in either of two ways: They may differ with respect to characteristics that have been measured, in which case there is an overt bias, or they may differ in ways that have not been measured, in which case there is a hidden bias. Overt biases are controlled through adjustments, such as matching. Hidden bias is more difficult to address because the relevant measurements are not available. A sensitivity analysis asks how much hidden bias would need to be present if hidden bias were to explain the differing outcomes in the treated and control groups. A sensitivity analysis provides a tangible and specific framework for discussing hidden biases.