Unmixing Mixed‐Methods Research

Over the last decade, mixed-methods research has become a growth industry. Legions of instructional texts on mixed-methods research (e.g., Bergman, 2008; Greene, 2007; Tashakkori Creswell, Klassen, Plano Clark, Heyvaert, Hannes, Maes, http://mmira.wildapricot.org/) was recently launched to promote the development of an international and interdisciplinary mixed methods research community, with an inaugural conference scheduled for 2014. Mixed-methods research has become an academic discipline and movement with its own experts, discourse, culture, and even “nationality,” to which ever-growing numbers of researchers identifying with both qualitative and quantitative research are claiming “dual citizenship” (Cooper, Glaesser, Gomm, Travers, 2009; Wiles, Crow, Mertens, 2007). Yet mono-method QL or QN studies may be just as and even more complex than mixed-methods studies in the sense of being multi-faceted and requiring sophisticated skills, and more (e.g., data collection measures, data analysis techniques) is not necessarily better in the matter of research design. As Greene and Caracelli (1997, p. 5) noted, “multiple and diverse methods is a good idea, but is not automatically good science.” A second persistent idea is that QL and QN research approaches offer different understandings of targeted problems and phenomena. Yet because philosophical/theoretical orientations to inquiry are not uniformly tied to any specific methodologies, techniques, or research designs, and because data may be conceived the same way regardless of how they were generated, mixed-methods research will not necessarily yield these different understandings. For example, as I proposed in more detail elsewhere (Sandelowski, 2011), a study might include both standardized closed-ended and highly structured questionnaires (typically conceived as QN data collection) and open-ended minimally structured interviews (typically conceived as QL data collection) addressing illness management. But if participants' responses to both of these data collection modalities are treated the same way—that is, as more or less accurate indices of participants' feelings, thoughts, and actions regarding illness management, as opposed to, for example, as scripted cultural performances—the result may be more information but not necessarily different perspectives on illness management. A third persistent assumption about mixed-methods research is that the respective strengths and weaknesses of QL or QN research approaches will offset each other. Yet strength and weakness are not attributes of research approaches but rather judgments researchers make about them (Sandelowski, 2012). The choice of combination of elements to use must be defended as meeting the specific objectives of a study, and it is the choice researchers make that will be judged as strong or weak. The term mixed in mixed-methods research is used variously to refer to combinations of research elements, often with no clear delineation of the type of mix and no acknowledgment that mono-method research is also characterized by mixes of elements or that research elements may be used in the same study without mixing them. The mixed in mixed-methods research signals one of three basic types of mixes, including the mere use together, linking, or actual integration of one or more QL and one or more QN elements at one or more time points in one study or in a program of research encompassing more than one study. The most common meeting points, or points of interface (Guest, 2013), between elements conceived as either QL or QN are at the methodological (e.g., randomized controlled trial and narrative study) and operational levels of inquiry, the latter including combinations of sampling (e.g., purposeful and probability) and/or data collection (e.g., standardized questionnaire and ethnographic interview), analysis (e.g., multiple regression and narrative analyses), interpretation (e.g., structural equation model and grounded theory), and representational (e.g., words, numbers, visual displays) techniques typically classified as either QL or QN. In the use-together type of mixed-methods research, QL and QN components remain separate, as when open-ended and minimally structured ethnographic interviews and closed-ended highly structured standardized questionnaires are used to generate/collect data. In the linking type of mixed-methods research, one or more QL and QN elements are placed in juxtaposition to each other but not merged, as when the results from the ethnographic interviews and standardized questionnaires are compared to ascertain their relationship to each other: that is, to determine whether they confirm, refute, or otherwise extend or modify each other. In the integration type of mixed methods research, QL and QN elements are assimilated into each other, as when data in one form are converted to another form to create a new data set. Qualitizing and quantitizing are the terms typically used to refer to the transformation or conversion of QN into QL data, and QL into QN data, respectively (Greene, 2007; Teddlie or verbal summaries, profiles, or categories may be created from the statistical results derived from standardized questionnaires. The integration type of mixed-methods research is evident also when findings derived from QL or QN analyses are used as placeholders for each other. In this form of integration, a grounded theory derived from ethnographic interviews and observations is used to hold in place, or organize, house, or configure findings from a statistical analysis of responses to standardized questionnaires, or the findings derived from the statistical analysis of these responses are used to delineate further the grounded theory derived from the ethnographic interviews and observations. Any one mixed-methods study or program of research may entail mixes of these types of mixes (use-together, linking, or integration) at one or more time points. These mixes may be planned ahead of time or decided on as a result of data analysis. This is why the many typologies advanced for mixed-methods research designs (Nastasi, Hitchcock, Onwuegbuzie Slone, 2009). Indeed, the default to the QL/QN binary has been singled out as one (if not the) major impediment to mindful, inventive, and well-crafted inquiry (e.g., Allwood, 2012; Ercikan Gorard, 2004; Sandelowski, Voils, Vogt, 2008). Both QL and QN research are still presented in instructional texts as either/or with sharp lines drawn between QL subjective/QN objective, QL inductive/QN deductive, QL word/QN number, and the like. These distinctions elide the commonalities between elements designated as QL or QN, and the enormous variations within the QL and QN research domains, respectively. The designations QL and QN are at best umbrella terms signifying a range of different and even conflicting elements, from research paradigms to research designs to techniques for sampling, data collection, and data analysis, and to modes of representation. Contributing to the within-QL and within-QN methodological diversity is the diversity in researchers from the practice, behavioral, social science, and arts and humanities disciplines, with highly varied and often conflicting ideas about the purpose and practice of QL and/or QN research. The QL/QN binary masks the reality that no ostensibly QN study escapes qualitizing, and few ostensibly QL studies escape quantitizing (Gorard, 2004; Martin, 2004; Maxwell, 2010; Sandelowski et al., 2009). Moreover, data are neither QL nor QN, but rather aspects of experiences or phenomena transformed into words, numbers, visual forms, and the like, each of which may in turn be transformed again into other forms (Valsiner, 2000). The idea, for example, that the closed-ended and highly structured questionnaire constitutes a QN element and the open-ended minimally structured interview a QL element effaces the countless variations in how questionnaires and interviews may be conceived, developed, conducted, or administered, in the purposes they are intended to fulfill, and in the way questionnaire and interview data may be analyzed, interpreted, and represented. Questionnaires may become more like interviews and the social interaction characterizing interviews when they are administered orally and in person, regardless of the training the person administering the questionnaire received to avoid such practices as helping respondents with or indicating (dis)approval of their answers. Interviews may become more like questionnaires and the reader/text interaction characterizing questionnaires when participants are asked to respond to questions without an interviewer present. Both questionnaires and interviews are typically conceived as ways to collect or generate data about a target phenomenon, but the questionnaire may also be conceived as a communication vehicle between an imagined researcher and imagined participant (Smith, 2008) and analyzed as a social interaction (Potter, 2003). The interview in turn may also be conceived as a cultural performance and vehicle for impression management (Rapley, 2001; Riessman, 2008). All behavioral and social science questionnaires may be viewed as QL because respondents interpret both verbal anchors and the numerical values attached to them (Sandelowski et al., 2009). Closed-ended highly structured questionnaires are typically used to draw formal generalizations from probability samples to populations, but they may also be used to draw particularizing generalizations from and about purposefully selected cases. Questionnaire items may be used as elicitation devices in narrative interviewing. Responses to both open- and closed-ended data collection devices may be analyzed via a range of diverse approaches, including varieties of content, narrative, conversation, and statistical analyses. In short, neither questionnaire nor interview can be easily categorized as QL or QN. Moreover, in variously encompassing QL and QN elements, each one may be seen as itself a mixed modality. The tendency to align the QL side of the QL/QN binary with such descriptors as subjective, inductive, verbal, and constructivist and the QN side with such descriptors as objective, deductive, numerical, and realist mistakes foreground/background features for categorical differences lying on one or the other side of the QL/QN divide. The subjective/objective binary, for example, signifies differences in emphasis in QL and QN discourses, not in the actual conduct of inquiry; that is, the disciplined subjectivity characterizing all empirical inquiry tends to be center stage in discourses about QL (both favorable and unfavorable to it), while it remains in the background in QN discourses (Sandelowski, 2011). To characterize QL research as inductive and QN research as deductive is to move to the background the iterative cycling between induction and deduction that characterizes all inquiry. The tendency to align QN but not QL with hypothesis testing, the interest in causation, the ability to generalize, and the emphasis on validity overlooks the varied conceptualizations and/or operationalizations of inquiry objectives common to all empirical inquiry. Hypothesis testing is integral to forms of grounded theory and analytic induction (Hammersley, 2012). QL approaches are often directed toward examining causal processes (Maxwell, 2004). Generalization is not defined exclusively by the ability to draw inferences from samples to populations (formal generalization), but also by the ability to draw inferences from and about individual cases (idiographic generalization) and from theoretical concepts and typologies (analytical generalization), and to apply concepts derived from the study of a phenomenon in one group to the understanding of a different phenomenon in a different group (transferability; Halkier, 2011; Larsson, 2009; Polit Maxwell, 2012) that are objectives in all manner of empirical inquiry are achieved by a variety of practices, the aspects of which often overlap. For example, there are aspects of cognitive interviewing that overlap with member checking and of reliability coding that overlap with negotiated consensus. At its best and most useful, mixed-methods research represents a novel discursive repackaging of the artful assemblages of research elements characterizing empirical inquiry in general. Indeed, the singular contribution of the mixed-methods research movement may be to remind researchers and teachers of research methods that mixing methods may just be the “natural way to conduct high quality relevant research” (Gorard, 2010, p. 64). The mixed-methods research movement may be said to have generated the “rebirth of research as a craft” (Symonds & Gorard, 2010): that is, the playful discipline and disciplined play whereby researchers creatively combine theories, methodologies, and/or materials to accommodate their specific research purposes. There are no “best practices” (Creswell et al., 2011) for designing, conducting, and evaluating mixed-methods research that do not apply also to mono-method research. Indeed, as mono-method research typically entails the mixing of elements, the distinction is itself called into question. In my effort to unmix mixed-methods research here, I am not positioning myself against mixed-methods research qua research, or proposing that theories and methodologies cannot or should not be distinguished from each other. Rather, I am encouraging the adoption of a mindset less focused on the “mixing” of methods conceived to lie on the QL/QN divide and more focused on which method and design assemblages make sense for answering significant research questions. I am urging an “organic” view of empirical research whereby the unity of a study or program of study is emphasized over its “methodological parts”(Song, Sandelowski, & Happ, 2010, p. 730). Regrettably, the emphasis on these methodological parts—especially evident in the “song and dance” (Symonds & Gorard, 2010, p. 131) of mixed-methods language and typologies researchers feel obliged to perform in their proposals and reports to establish their mixed-methods research bona fides—has increasingly taken priority over communicating clearly the significance of the research problem and what was or will be done to address it.