Using mixed methods
2042533313479197pmcid: pmc3697857using mixed methods in health researchshema es of authors’ mixed methods research – es of authors’ mixed methods research – y, quantitative research is associated with a positivist stance and a belief that reality that can be measured and observed objectively. However, for research that aims to directly impact on policy and practice, the findings of qualitative research can be limited by the small sample sizes that are necessary for in-depth exploratory work and the consequent lack of methods research therefore has the potential to harness the strengths and counterbalance the weaknesses of both approaches and can be especially powerful when addressing complex, multifaceted issues such as health services interventions9 and living with chronic are many reasons why researchers choose to combine quantitative and qualitative methods in a study. 12 we list some common reasons below, using a hypothetical research question about adolescents’ adherence to anticonvulsant medication to illustrate real world mentarity: using data obtained by one method to illustrate results from another. Semi-structured interviews with a sub-group of those surveyed may allow us to explore barriers to pment: using results from one method to develop or inform the use of the other method. We could then develop a mobile phone ‘app’ that reminds patients to take their medication and conduct an intervention study to assess its impact on adherence tion: using results from different methods specifically to look for areas of incongruence in order to generate new insights. A review of case notes may find adherence levels of over 90% in a clinic population; however, semi-structured interviews with peer researchers may reveal lower levels of adherence and barriers to open discussion with ion: setting out to examine different aspects of a research question, where each aspect warrants different methods. A large-scale survey of adolescents with epilepsy would provide information on adherence levels and associations whilst interviews and focus groups may allow us to engage with individual experiences of chronic illness and medication in ulation: using data obtained by both methods to corroborate findings. Qualitative methods such as video diaries may confirm adherence this list we would also add political commitment. That is to say, researchers may recognize, and wish to deploy, the strengths of quantitative research in producing generalizable results but may also be committed to representing the voice of participants in their er the reasons for mixing methods, it is important that authors present these explicitly as it allows us to assess if a mixed methods study design is appropriate for answering the research question. Embarking on a mixed methods research project it is important to consider:The methods that will be used;the priority of the methods;the sequence in which the methods are to be used. Both the research question and the data required will be the main determinants of the methods used. To a lesser extent, the choice of methods may be influenced by feasibility, the research team’s skills and experience and time ty of methods relates to the emphasis placed on each method in the study.
Again the choice of when to use each method is largely dependent on the research priority and sequence of mixing methods have been elaborated in a typology of mixed methods research models. See table 1 for typology and specific es of studies using mixed is data analysed in a mixed methods project? Most important, and perhaps most difficult, aspect of mixed methods research is integrating the qualitative and quantitative data. The quantitative and qualitative data are kept analytically distinct and are analysed using techniques usually associated with that type of data; for example, statistical techniques could be used to analyse survey data whilst thematic analysis may be used to analyse interview data. In this approach, the integrity of each data is preserved whilst also capitalizing on the potential for enhanced understanding from combining the two data and sets of r approach to mixed methods data analysis is the integrative strategy. An example of this is the development of narrative psychological ‘types’ from numerical data obtained by ial challenges in conducting mixed methods researchdespite its considerable strengths as an approach, mixed methods research can present researchers with challenges. Researchers subscribing to this view argue that it is neither possible nor desirable to combine quantitative and qualitative methods in a study as they represent essentially different and conflicting ways of viewing the world and how we collect information about it. Other researchers take a more pragmatic view, believing that concerns about the incommensurability of worldviews can be set aside if the combination of quantitative and qualitative methods addresses the research question effectively. This pragmatic view informs much applied mixed methods research in health services or ly, combining two methods in one study can be time consuming and requires experience and skills in both quantitative and qualitative methods. This can mean, in reality, that a mixed methods project requires a team rather than a lone researcher in order to conduct the study rigorously and within the specified time frame. We believe that a project leader with experience in both quantitative and qualitative methods can act as an important bridge in a mixed methods y, achieving true integration of the different types of data can be difficult. If this is not the case then integration may not have occurred y, many researchers cite the difficulty in presenting the results of mixed methods study as a barrier to conducting this type of research.
This strategy may involve a decision to publish additional work focusing on the interpretations and conclusions which come from comparing and contrasting findings from the different data types. Many journals in the medical sciences have a distinct methodological base and relatively restrictive word limits which may preclude the publication of complex, mixed methods studies. However, as the number of mixed methods studies increases in the health research literature we would expect researchers to feel more confident in the presentation of this type of sionmany of the areas we explore in health are complex and multifaceted. Mixed methods research (combining quantitative and qualitative methods in one study) is an innovative and increasingly popular way of addressing these complexities. Although mixed methods research presents some challenges, in much the same way as every methodology does, this approach provides the research team with a wider range of tools at their disposal in order to answer a question. St wrote the manuscript with revisions and editing done by jwacknowledgementswe thank professors jonathan elford and ruth gilbert for their comments on draft manuscriptsprovenancethis article was submitted by the authors and peer reviewed by geoffrey g g, gantley ative methods: beyond the cookbook. Cathain a, murphy e, nicholl , and how, mixed methods research is undertaken in health services research in england: a mixed methods study. 7: 85–85 [pmc free article] [pubmed]ll jw, plano clark ing and conducting mixed methods research, london: sage publications ltd, n j, allister j, rafi i, et al. 62: e478–86 [pmc free article] [pubmed]n j, freemantle n, allister j, de lusignan s, gilbert r, petersen ion in recorded child maltreatment concerns in uk primary care records: a cohort study using the health improvement network (thin) database. 24: 978–85 [pubmed] s, pillen a, tookey pa, brown ae, elford impact of african ethnicity and migration on pregnancy in women living with hiv in the uk: design and methods. 12: 596–596 [pmc free article] [pubmed] e, dingwall r, greatbatch d, parker s, watson ative research methods in health technology assessment: a review of the literature. Symptom management in hiv/aids symptom management in hiv/aids: a mixed methods approach to describe collaboration and concordance between persons living with hiv and their close support persons.
J mix methods res 2012;3:217– j, caracelli v, graham a conceptual framework for mixed-method evaluation designs. Educ eval policy ng the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. 13: 92–8 [pubmed]g a, ghaleb m, suokas a, et scale organisational intervention to improve patient safety in four uk hospitals: mixed method evaluation. 342: d195–d195 [pmc free article] [pubmed] sarah l, kowalski c, damschroder l, forman j, kaufman s, saint ting ventilator associated pneumonia in the united states: a multicenter mixed methods study. 29: 933–40 [pubmed]r ep, webster nj, blixen ce, et l consumption decisions among nonabusing drinkers diagnosed with hepatitis c. Educ eval policy kori a, teddlie methodology: combining qualitative and quantitative approaches, thousand oaks, ca: sage, ing qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. J mix methods n rb, onwuegbuzie methods research: a research paradigm whose time has come. 18: 1574–85 [pubmed]articles from jrsm short reports are provided here courtesy of royal society of medicine s:article | pubreader | epub (beta) | pdf (513k) | licated reviews of educational research methods research designs.. Version of this field of mixed methods has only been widely accepted for the last decade, though researchers have long been using multiple methods, just not calling them “mixed. Mixed methods research takes advantage of using multiple ways to explore a research can be based on either or both ch problems can become research questions and/or hypotheses based on prior literature, knowledge, experience, or the research sizes vary based on methods collection can involve any technique available to retation is continual and can influence stages in the research use mixed methods? Discrepancies between different types of designs generate unequal be difficult to decide when to proceed in sequential guidance on transformative ologist john creswell suggested a systematic framework for approaching mixed methods research. His framework involves four decisions to consider and six decisions for mixed method designs (creswell, 2003, p.
The results are integrated in the interpretation e: to employ the methods that best serve a theoretical perspective. Concurrent terized by: two or more methods used to confirm, cross-validate, or corroborate findings within a study. Data collection is e: generally, both methods are used to overcome a weakness in using one method with the strengths of terized by: a nested approach that gives priority to one of the methods and guides the project, while another is embedded or “nested. On february 11, 2015 thursday, february 5, sage author russell schutt presented in a webinar about why and when it is appropriate to used mixed methods to better understand different aspects of our social world. We screened all interested guests in the three mental health shelters for safety and then randomly assigned all those who were eligible for the project to either the group or independent housing conditions. In addition, in most analyses (including in the book and the many project articles cited in it), i used statistical controls to adjust for subject differences, most often multiple regression did you decide on the various methods used? We included qualitative questions whenever possible to allow us to reflect on what research participants meant by their answers or simply to provide a more engaging social the most effective presentation of the analysis, i organized the book to maximize the value of having multiple methods of investigation for most of the major research questions in which i was interested. Each of my chapters (except the first two and the last one) is on a major topic that must be considered in order to understand the experiences and outcomes of persons living in our project’s housing. At the start of most of these chapters, i review relevant literature, then present a quantitative analysis using the baseline data, then a qualitative analysis of related activities throughout the project, and then conclude with a quantitative analysis of change over time. So my basic approach in the analysis was to mix methods whenever i had data collected about a topic using both qualitative and quantitative techniques. The case studies emerged as i identified basic patterns in the ethnographic data and then searched for individuals who represented those are the challenges of using mixed methods? Are many that researchers should consider, even while we recognize the benefits of mixed methods.
I have included a lengthy section on strengths and limitations of mixed methods in the mixed methods chapter of the 8th edition of investigating the social world and i discuss in that section each of the major challenges. Here are the highlights: (1) one of the appeals of mixed methods is that it can help us “triangulate” our measurement strategy–in effect, use different measures of the same concept to provide a more robust overall measure. In some cases, qualitative methods can provide information that helps us to answer this question. See my example of the non-disclosure of substance abuse in the “strengths and limitations” section or in the methodological appendix to homelessness, housing, and mental illness, or the journal of nervous and mental disease article cited below). My discussion of cross-site differences in the multi-site nimh-funded study of housing and homelessness is an example of how a careful review can generate reasonable explanations, but confirmation must await additional research. Of course, a major incentive for using mixed methods is to uncover unexpected patterns and generate new research questions, so in that sense the real challenge is to keep garnering resources to allow us to refine our knowledge of social r major challenge for users of mixed methods is the different types of expertise required. Most social researchers can manage adding some qualitative questions to a primarily quantitative survey, or collecting some quantitative indicators in a qualitative project and then analyzing the resulting mixed methods data to good effect. So developing a strong mixed methods project will often require collaboration of different researchers with different types of expertise. For example, at one point in the work that led up to homelessness, housing, and mental illness, i had hoped to construct social network measures based on ethnographic data collected by three different ethnographers in each of the six group homes we studied. Are many different approaches to this combination and their value depends on the specific type of mixed methods design used and the research question asked (see exhibit 15. I emphasized in my webinar the importance of the research question, so i will first review the research question in the nimh study i focused on in the webinar and demonstrate how it shaped the way in which i combined conclusions from the mixed methods in that study. There were no problems getting them the overall research question, about achieving community integration, led to more research questions that required different types of methods.
You can see an example of conclusions that are developed from a mixed methods but that make no mention of this in a report that i wrote for a national cancer institute-funded study about community health workers and cancer clinical will see in the report extensive use of quotes from semi-structured interviews, but also tables and graphs that summarize quantitative data collected in these same interviews. Yet in the conclusions you’ll see that i simply discuss what we learned about the research question and do not distinguish the different methods we common, however, is the use of qualitative data to clarify or suggest a process or complexity that the quantitative findings identified. There is an example from the research by brown and his colleagues (2013:341-343) in schutt 2015: top tier journals becoming more open to publishing mixed methods research? We use a qualitative research and a quantitative research, can we say it is mixed methods? Although they are not categorized in any type of mixed methods that text are no sharp boundaries between the different types of mixed methods or between single method and those mixed method designs. So one reason to study mixed methods as a separate topic is to better understand why it can be so important to add some elements of one approach to research using another approach, such as cognitive (qualitative) interviewing when developing survey questions or counts of specific behaviors when observing groups. Of investigating (8th edition) are true mixed methods designs, but those involving an “embedded” approach may be better understood as single method designs if the “embedded” component is minor and intended only to enhance the value of the primary method. For example, including a few open-ended questions or a follow-up probe in a structured survey does not make it a mixed methods design. Counting the number of individuals on street corners at different times during an ethnographic neighborhood study does not make it a mixed methods the other hand, in the type of mixed methods design i label as a “research program” (schutt 2015:exhibit 15. Different researchers may have conducted different investigations of the same research question that build on each other but that use different, single methods. It is helpful to think of such a research program in terms of the issues that arise in a mixed methods design, but of course the specific research studies are only using single method we use a survey with rating scales (quan) and open-ended questions (qual), is it sufficient to call it a mixed methods (only with open-ended questions)? I mentioned in response to the preceding question, a survey that has primarily fixed choice questions (whether with or without rating scales) and just a few open-ended questions is not reasonably termed one that uses a mixed methods design.
However, two caveats are important: (1) it is valuable to consider issues and possibilities that arise in mixed methods designs when planning, collecting, and analyzing data from such a survey; (2) the more open-ended questions are included in such a semi-structured survey, the more that it begins to approximate a mixed methods study. For example, the survey instrument that appears at the end of this report would qualify as a mixed methods survey due to the many open-ended questions (apologies for the poor formatting in the online document): http:///russell_schutt/building%20the% . As i mentioned previously, there are no sharp boundaries between research designs that can reasonably be termed mixed methods and those that are best thought of as single method you repeat the name of the book where you mention the software you used to analyze qualitative data? I see that it has not recently been updated, but you can order it at: http:///books/ you elaborate more into the integration model of mixed method vs. Types of mixed methods) distinguishes the “integrated” type of mixed method design as involving equal priority to and concurrent use of qualitative and quantitative methods, while i distinguish the “staged” type of mixed methods design as involving sequential use of qual to quan and quan to qual where one of those methods is given priority, and the “research program” type of mixed methods design as involving sequential use of quan and qual methods but giving them equal priority. Using a concurrent qual+quan integrated design requires planning in advance how to mesh the two approaches within one project and then balancing both types of methods (and researchers) at the same time. You may learn after the fact about how data collected in one mode contributes or alters insights developed using the other mode. Are all very expert mixed methodologists and we can all learn much from them when making design decisions. I don’t feel that i’ve been able to keep up with all the different excellent choices now available in qualitative data analysis software, so i don’t feel i can provide more specific is the best strategy to approach requesting funding for a mixed methods project? It is also important to ground the mixed methods logic in a very firm understanding of the research literature about the planned research question, so that you can make a clear argument about how the mixed methods elements are necessary to overcome prior limitations, puzzling findings, or contradictory the use of mm mean that one or the other of the 2 approaches is less ‘rigorous? It is certainly possible that a mm study may require resources be spread across two methods in a way that lessens the rigor that could be achieved when using just one method. Historical and comparative methods also provide many good examples of designs made more rigorous with the use of both qualitative and quantitative methods (see chapter 13 in isw8).
Nonetheless, it is very difficult to combine intensive ethnographic methods and quantitative elements, and it may not necessarily even be desirable to do so. Research projects that are so ambitious as our nimh-funded study of housing alternatives as to include a major ethnographic component with a quantitative experimental design require substantial funding, considerable time, and a carefully planned group of collaborators representing the different necessary forms of looking at survey data and qualitative research, how do we rectify contradictions in participants’ answers? That’s what i concluded in the housing and homelessness study in relation to an index of social support that had been developed with college students (schutt 2011:305-306). 2) is the difference between the two methods revealing that the concept you intended both to measure is multidimensional or otherwise more complex than you had anticipated? So the fact that qualitative and quantitative “measures” could result in apparently contradictory results is not a reason to avoid mixed methods, but rather an indication of how valuable mixing methods can be to uncovering a more complex social reality than we had is the format for writing mixed methods results? A key issue in deciding which approach to take is whether the different methods are being used to provide more insight into the same issue, or to investigate different aspects of an overarching research do you determine which method is most appropriate for the given research? I emphasized in my webinar that a mixed methods design should be considered the more the research question is original, complex, has ambiguous or conflicting implications, and suggests a challenge for an authentic identification of a causal mechanism or causal context. We can enrich the answer to any research question by mixing methods, but it is not necessarily a good idea to do so if resources are limited and the question is relatively simple, etc. It’s also important to have sufficient expertise with each planned method and/or a plan with appropriate collaborators before deciding to adopt a mixed methods design. But no researcher should feel that they are somehow missing the boat if they don’t use mixed methods in a particular investigation, as the insights from so much single-method research shows us. Rather, it’s a matter to be considered in identifying limitations of a single study and avenues for further considering what methods to use, is there any difference when considering evaluation questions vs research questions? Admittedly, some institutional review boards (irbs) interpret federal guidelines as allowing evaluation research on behalf of a client to be exempt from human subjects review, so that can be one practically important difference, but that is not a key issue from the standpoint of the appropriateness of different methods.
However, evaluations designed as needs assessments, evaluability assessments, or as process evaluations usually include a major qualitative component and hence are likely to involve mixed methods. See chapter 12 of investigating, 8th edition, for a more detailed discussion of these types of evaluations and the value of qualitative methods in evaluation research (pp. Again, it always makes sense to consider mixed methods, even if this leads to a decision to concentrate efforts with just one method. Discuss the mixed methods designs options in chapter 15 of investigating the social world, 8th edition, and distinguish them in terms of the dimensions of priority and sequence in exhibit 15. Using that scheme, i categorized our nimh-funded study as having used an “integrated method” design, symbolized as qual+quan. We had both practical and scholarly reasons for choosing this type of mixed methods design. Million from nimh for the research and related services and $10 million from hud for developing the housing alternatives) and we had connections with excellent researchers having different types of expertise and ourselves represented (that is the co-investigator team) represented different disciplines. In this way, a massively mixed methods design was made possible, with resources to collect and analyze data ranging from thousands pages of ethnographic notes to neuropsychological test results. When you read my book (schutt 2011), you can decide for yourself how well our mixed methods design allowed us to understand these complex it make sense then to separate the mentally ill based on what knowledge we have gained into both group living and individual living or apartments based on the mixed methods approach, (quantitative and ethnographic studies)? I will quote from a few of those recommendations that bear on this question of housing most important policy recommendation from the analysis … is to give community processes their due in housing programs. Selected what i believe are very useful articles using mixed methods to illustrate different techniques in investigating the social world, 8th edition. In order to gain a basic, solid understanding of qualitative methods per se, i suggest purchasing the book and reading both chapters 10 and 11.
Advancing the study of violence against women using mixed methods: integrating qualitative methods into a quantitative research program. There any kinds of research questions that you would say would not profit from mixed methods? Some subsequent research has extended this study without any need for qualitative determines whether you begin your mixed methods study with the qualitative (discovery) vs quantitative (relationships) analysis? In investigating the social world, 8th edition, some types of mixed methods designs do not use a sequenced approach. However, among those that do, certainly the use of qualitative methods for an initial more exploratory investigation of a research question, followed by a structured quantitative investigation (often a structured survey) is the most common sequence. This was the approach in the research by zhong and arnett (2014) that i discuss in my mixed methods chapter (pp. Researchers are also starting to use more complex mixed methods designs that alternate between qualitative and quantitative methods.