Surveying Difficult Populations: Lessons Learned from a National Survey of State Trial Court Judges*

Surveying trial court judges presents a variety of serious challenges, but recent Supreme Court decisions and the use of increasingly complex scientific evidence in trials make it necessary to assess trial judges’ knowledge and views of scientific evidence. In this article, we describe a successful nationwide survey of state trial court judges. Response rates were high, and the information obtained was valuable. Rigorous attention to detail and to proper methodology at every step is important; i.e., knowledge of the topic being investigated and the respondents’ circumstances; creation and assessment of questionnaires; construction and assessment of the codebook; training and assessment of interviewers and coders; creation of analysis plans; and flexibility. This kind of care of is costly and effortful, but it can make the difference between a successful survey and a waste of time and money. There are many compelling reasons to study the judiciary. However, judges are often perceived by researchers and those who fund research as difficult to study. Reasons for this perception of difficulty probably include the high status and professional remoteness of the judiciary in American society, judicial time constraints, assumed resentment or unwillingness to be tested, concerns by judges about confidentiality of responses, and perhaps a distrust, dislike, or perceived irrelevance of social and behavioral science and scientists. 1 The perception that judges are difficult to study can have negative consequences