comparative analysis of quantitative and qualitative research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Qualitative vs. Quantitative Research | Differences, Examples & Methods

Qualitative vs. Quantitative Research | Differences, Examples & Methods

Published on April 12, 2019 by Raimo Streefkerk . Revised on June 22, 2023.

When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Quantitative research is at risk for research biases including information bias , omitted variable bias , sampling bias , or selection bias . Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs. quantitative research, how to analyze qualitative and quantitative data, other interesting articles, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyze data, and they allow you to answer different kinds of research questions.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observational studies or case studies , your data can be represented as numbers (e.g., using rating scales or counting frequencies) or as words (e.g., with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which different types of variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations : Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups : Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organization for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis )
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs. deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”

You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: “How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analyzing quantitative data

Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores ( means )
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analyzing qualitative data

Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analyzing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, June 22). Qualitative vs. Quantitative Research | Differences, Examples & Methods. Scribbr. Retrieved March 18, 2024, from https://www.scribbr.com/methodology/qualitative-quantitative-research/

Is this article helpful?

Raimo Streefkerk

Other students also liked, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, mixed methods research | definition, guide & examples, what is your plagiarism score.

Join thousands of product people at Insight Out Conf on April 11. Register free.

Insights hub solutions

Analyze data

Uncover deep customer insights with fast, powerful features, store insights, curate and manage insights in one searchable platform, scale research, unlock the potential of customer insights at enterprise scale.

Featured reads

Tips and tricks

Four ways Dovetail helps Product Managers master continuous product discovery

Product updates

Dovetail retro: our biggest releases from the past year

How to affinity map using the canvas

Events and videos

© Dovetail Research Pty. Ltd.

What is comparative analysis? A complete guide

Last updated

18 April 2023

Reviewed by

Jean Kaluza

Comparative analysis is a valuable tool for acquiring deep insights into your organization’s processes, products, and services so you can continuously improve them. 

Similarly, if you want to streamline, price appropriately, and ultimately be a market leader, you’ll likely need to draw on comparative analyses quite often.

When faced with multiple options or solutions to a given problem, a thorough comparative analysis can help you compare and contrast your options and make a clear, informed decision.

If you want to get up to speed on conducting a comparative analysis or need a refresher, here’s your guide.

Make comparative analysis less tedious

Dovetail streamlines comparative analysis to help you uncover and share actionable insights

• What exactly is comparative analysis?

A comparative analysis is a side-by-side comparison that systematically compares two or more things to pinpoint their similarities and differences. The focus of the investigation might be conceptual—a particular problem, idea, or theory—or perhaps something more tangible, like two different data sets.

For instance, you could use comparative analysis to investigate how your product features measure up to the competition.

After a successful comparative analysis, you should be able to identify strengths and weaknesses and clearly understand which product is more effective.

You could also use comparative analysis to examine different methods of producing that product and determine which way is most efficient and profitable.

The potential applications for using comparative analysis in everyday business are almost unlimited. That said, a comparative analysis is most commonly used to examine

Emerging trends and opportunities (new technologies, marketing)

Competitor strategies

Financial health

Effects of trends on a target audience

• Why is comparative analysis so important? 

Comparative analysis can help narrow your focus so your business pursues the most meaningful opportunities rather than attempting dozens of improvements simultaneously.

A comparative approach also helps frame up data to illuminate interrelationships. For example, comparative research might reveal nuanced relationships or critical contexts behind specific processes or dependencies that wouldn’t be well-understood without the research.

For instance, if your business compares the cost of producing several existing products relative to which ones have historically sold well, that should provide helpful information once you’re ready to look at developing new products or features.

• Comparative vs. competitive analysis—what’s the difference?

Comparative analysis is generally divided into three subtypes, using quantitative or qualitative data and then extending the findings to a larger group. These include

Pattern analysis —identifying patterns or recurrences of trends and behavior across large data sets.

Data filtering —analyzing large data sets to extract an underlying subset of information. It may involve rearranging, excluding, and apportioning comparative data to fit different criteria. 

Decision tree —flowcharting to visually map and assess potential outcomes, costs, and consequences.

In contrast, competitive analysis is a type of comparative analysis in which you deeply research one or more of your industry competitors. In this case, you’re using qualitative research to explore what the competition is up to across one or more dimensions.

For example

Service delivery —metrics like the Net Promoter Scores indicate customer satisfaction levels.

Market position — the share of the market that the competition has captured.

Brand reputation —how well-known or recognized your competitors are within their target market.

• Tips for optimizing your comparative analysis

Conduct original research

Thorough, independent research is a significant asset when doing comparative analysis. It provides evidence to support your findings and may present a perspective or angle not considered previously. 

Make analysis routine

To get the maximum benefit from comparative research, make it a regular practice, and establish a cadence you can realistically stick to. Some business areas you could plan to analyze regularly include:

Profitability

Competition

Experiment with controlled and uncontrolled variables

In addition to simply comparing and contrasting, explore how different variables might affect your outcomes.

For example, a controllable variable would be offering a seasonal feature like a shopping bot to assist in holiday shopping or raising or lowering the selling price of a product.

Uncontrollable variables include weather, changing regulations, the current political climate, or global pandemics.

Put equal effort into each point of comparison

Most people enter into comparative research with a particular idea or hypothesis already in mind to validate. For instance, you might try to prove the worthwhileness of launching a new service. So, you may be disappointed if your analysis results don’t support your plan.

However, in any comparative analysis, try to maintain an unbiased approach by spending equal time debating the merits and drawbacks of any decision. Ultimately, this will be a practical, more long-term sustainable approach for your business than focusing only on the evidence that favors pursuing your argument or strategy.

Writing a comparative analysis in five steps

To put together a coherent, insightful analysis that goes beyond a list of pros and cons or similarities and differences, try organizing the information into these five components:

1. Frame of reference

Here is where you provide context. First, what driving idea or problem is your research anchored in? Then, for added substance, cite existing research or insights from a subject matter expert, such as a thought leader in marketing, startup growth, or investment

2. Grounds for comparison Why have you chosen to examine the two things you’re analyzing instead of focusing on two entirely different things? What are you hoping to accomplish?

3. Thesis What argument or choice are you advocating for? What will be the before and after effects of going with either decision? What do you anticipate happening with and without this approach?

For example, “If we release an AI feature for our shopping cart, we will have an edge over the rest of the market before the holiday season.” The finished comparative analysis will weigh all the pros and cons of choosing to build the new expensive AI feature including variables like how “intelligent” it will be, what it “pushes” customers to use, how much it takes off the plates of customer service etc.

Ultimately, you will gauge whether building an AI feature is the right plan for your e-commerce shop.

4. Organize the scheme Typically, there are two ways to organize a comparative analysis report. First, you can discuss everything about comparison point “A” and then go into everything about aspect “B.” Or, you alternate back and forth between points “A” and “B,” sometimes referred to as point-by-point analysis.

Using the AI feature as an example again, you could cover all the pros and cons of building the AI feature, then discuss the benefits and drawbacks of building and maintaining the feature. Or you could compare and contrast each aspect of the AI feature, one at a time. For example, a side-by-side comparison of the AI feature to shopping without it, then proceeding to another point of differentiation.

5. Connect the dots Tie it all together in a way that either confirms or disproves your hypothesis.

For instance, “Building the AI bot would allow our customer service team to save 12% on returns in Q3 while offering optimizations and savings in future strategies. However, it would also increase the product development budget by 43% in both Q1 and Q2. Our budget for product development won’t increase again until series 3 of funding is reached, so despite its potential, we will hold off building the bot until funding is secured and more opportunities and benefits can be proved effective.”

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 September 2023

Last updated: 14 February 2024

Last updated: 17 February 2024

Last updated: 19 November 2023

Last updated: 5 March 2024

Last updated: 5 February 2024

Last updated: 30 January 2024

Last updated: 12 October 2023

Last updated: 6 March 2024

Last updated: 31 January 2024

Last updated: 10 April 2023

Latest articles

Related topics, log in or sign up.

Get started for free

Encyclopedia of Quality of Life and Well-Being Research pp 1125–1127 Cite as

Comparative Analysis

• Sonja Drobnič 3  
• Reference work entry

981 Accesses

Context of comparisons ; Radical positivism

The goal of comparative analysis is to search for similarity and variance among units of analysis. Comparative research commonly involves the description and explanation of similarities and differences of conditions or outcomes among large-scale social units, usually regions, nations, societies, and cultures.

Description

In the broadest sense, it is difficult to think of any analysis in the social sciences that is not comparative. In a laboratory experiment, we compare the outcomes for the experimental and control group to ascertain the effects of some experimental stimulus. When we analyze quality of life of men and women, old and young, or rich and poor, we actually perform a comparison of individuals along certain dimensions, such as gender, age, and wealth/income. However, this meaning of comparative analysis is too general to be really useful in research. “Comparative analysis has come to mean the description and...

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Böhnke, P. (2008). Does society matter? Life satisfaction in the enlarged Europe. Social Indicators Research, 87 , 189–210.

Google Scholar  

Esping-Andersen, G. (1999). Social foundations of postindustrial economies . Oxford: Oxford University Press.

Hagerty, M. R., Cummins, R. A., Ferriss, A. L., Land, K., Michalos, A. C., Peterson, M., Sharpe, A., Sirgy, J., & Vogel, J. (2001). Quality of life indexes for national policy: Review and agenda for research. Social Indicators Research, 55 , 1–96.

Kohn, M. L. (1987). Cross-national research as an analytic strategy. American Sociological Review, 52 , 713–731.

Mills, M., van de Bunt, G. G., & de Bruijn, J. (2006). Comparative research. Persistent problems and promising solutions. International Sociology, 21 (5), 619–631.

Ragin, C. C. (1987). The comparative method: Moving beyond qualitative and quantitative strategies . Berkley/Los Angeles: University of California Press.

Smelser, N. J. (2003). On comparative analysis, interdisciplinarity and internationalization in sociology. International Sociology, 18 (4), 643–657.

Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis. An introduction to basic and advanced multilevel modeling . Thousand Oaks, CA: Sage.

Download references

Author information

Authors and affiliations.

Bremen International Graduate School of Social Sciences (BIGSSS), University of Bremen, Wiener Str., FVG, 28359, Bremen, Germany

Sonja Drobnič

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sonja Drobnič .

Editor information

Editors and affiliations.

University of Northern British Columbia, Prince George, BC, Canada

Alex C. Michalos

(residence), Brandon, MB, Canada

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this entry

Cite this entry.

Drobnič, S. (2014). Comparative Analysis. In: Michalos, A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0753-5_492

Download citation

DOI : https://doi.org/10.1007/978-94-007-0753-5_492

Publisher Name : Springer, Dordrecht

Print ISBN : 978-94-007-0752-8

Online ISBN : 978-94-007-0753-5

eBook Packages : Humanities, Social Sciences and Law

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Open access
• Published: 07 May 2021

The use of Qualitative Comparative Analysis (QCA) to address causality in complex systems: a systematic review of research on public health interventions

• Benjamin Hanckel 1 ,
• Mark Petticrew 2 ,
• James Thomas 3 &
• Judith Green 4  

BMC Public Health volume  21 , Article number:  877 ( 2021 ) Cite this article

20k Accesses

42 Citations

35 Altmetric

Metrics details

Qualitative Comparative Analysis (QCA) is a method for identifying the configurations of conditions that lead to specific outcomes. Given its potential for providing evidence of causality in complex systems, QCA is increasingly used in evaluative research to examine the uptake or impacts of public health interventions. We map this emerging field, assessing the strengths and weaknesses of QCA approaches identified in published studies, and identify implications for future research and reporting.

PubMed, Scopus and Web of Science were systematically searched for peer-reviewed studies published in English up to December 2019 that had used QCA methods to identify the conditions associated with the uptake and/or effectiveness of interventions for public health. Data relating to the interventions studied (settings/level of intervention/populations), methods (type of QCA, case level, source of data, other methods used) and reported strengths and weaknesses of QCA were extracted and synthesised narratively.

The search identified 1384 papers, of which 27 (describing 26 studies) met the inclusion criteria. Interventions evaluated ranged across: nutrition/obesity ( n  = 8); physical activity ( n  = 4); health inequalities ( n  = 3); mental health ( n  = 2); community engagement ( n  = 3); chronic condition management ( n  = 3); vaccine adoption or implementation ( n  = 2); programme implementation ( n  = 3); breastfeeding ( n  = 2), and general population health ( n  = 1). The majority of studies ( n  = 24) were of interventions solely or predominantly in high income countries. Key strengths reported were that QCA provides a method for addressing causal complexity; and that it provides a systematic approach for understanding the mechanisms at work in implementation across contexts. Weaknesses reported related to data availability limitations, especially on ineffective interventions. The majority of papers demonstrated good knowledge of cases, and justification of case selection, but other criteria of methodological quality were less comprehensively met.

QCA is a promising approach for addressing the role of context in complex interventions, and for identifying causal configurations of conditions that predict implementation and/or outcomes when there is sufficiently detailed understanding of a series of comparable cases. As the use of QCA in evaluative health research increases, there may be a need to develop advice for public health researchers and journals on minimum criteria for quality and reporting.

Peer Review reports

Interest in the use of Qualitative Comparative Analysis (QCA) arises in part from growing recognition of the need to broaden methodological capacity to address causality in complex systems [ 1 , 2 , 3 ]. Guidance for researchers for evaluating complex interventions suggests process evaluations [ 4 , 5 ] can provide evidence on the mechanisms of change, and the ways in which context affects outcomes. However, this does not address the more fundamental problems with trial and quasi-experimental designs arising from system complexity [ 6 ]. As Byrne notes, the key characteristic of complex systems is ‘emergence’ [ 7 ]: that is, effects may accrue from combinations of components, in contingent ways, which cannot be reduced to any one level. Asking about ‘what works’ in complex systems is not to ask a simple question about whether an intervention has particular effects, but rather to ask: “how the intervention works in relation to all existing components of the system and to other systems and their sub-systems that intersect with the system of interest” [ 7 ]. Public health interventions are typically attempts to effect change in systems that are themselves dynamic; approaches to evaluation are needed that can deal with emergence [ 8 ]. In short, understanding the uptake and impact of interventions requires methods that can account for the complex interplay of intervention conditions and system contexts.

To build a useful evidence base for public health, evaluations thus need to assess not just whether a particular intervention (or component) causes specific change in one variable, in controlled circumstances, but whether those interventions shift systems, and how specific conditions of interventions and setting contexts interact to lead to anticipated outcomes. There have been a number of calls for the development of methods in intervention research to address these issues of complex causation [ 9 , 10 , 11 ], including calls for the greater use of case studies to provide evidence on the important elements of context [ 12 , 13 ]. One approach for addressing causality in complex systems is Qualitative Comparative Analysis (QCA): a systematic way of comparing the outcomes of different combinations of system components and elements of context (‘conditions’) across a series of cases.

The potential of qualitative comparative analysis

QCA is an approach developed by Charles Ragin [ 14 , 15 ], originating in comparative politics and macrosociology to address questions of comparative historical development. Using set theory, QCA methods explore the relationships between ‘conditions’ and ‘outcomes’ by identifying configurations of necessary and sufficient conditions for an outcome. The underlying logic is different from probabilistic reasoning, as the causal relationships identified are not inferred from the (statistical) likelihood of them being found by chance, but rather from comparing sets of conditions and their relationship to outcomes. It is thus more akin to the generative conceptualisations of causality in realist evaluation approaches [ 16 ]. QCA is a non-additive and non-linear method that emphasises diversity, acknowledging that different paths can lead to the same outcome. For evaluative research in complex systems [ 17 ], QCA therefore offers a number of benefits, including: that QCA can identify more than one causal pathway to an outcome (equifinality); that it accounts for conjectural causation (where the presence or absence of conditions in relation to other conditions might be key); and that it is asymmetric with respect to the success or failure of outcomes. That is, that specific factors explain success does not imply that their absence leads to failure (causal asymmetry).

QCA was designed, and is typically used, to compare data from a medium N (10–50) series of cases that include those with and those without the (dichotomised) outcome. Conditions can be dichotomised in ‘crisp sets’ (csQCA) or represented in ‘fuzzy sets’ (fsQCA), where set membership is calibrated (either continuously or with cut offs) between two extremes representing fully in (1) or fully out (0) of the set. A third version, multi-value QCA (mvQCA), infrequently used, represents conditions as ‘multi-value sets’, with multinomial membership [ 18 ]. In calibrating set membership, the researcher specifies the critical qualitative anchors that capture differences in kind (full membership and full non-membership), as well as differences in degree in fuzzy sets (partial membership) [ 15 , 19 ]. Data on outcomes and conditions can come from primary or secondary qualitative and/or quantitative sources. Once data are assembled and coded, truth tables are constructed which “list the logically possible combinations of causal conditions” [ 15 ], collating the number of cases where those configurations occur to see if they share the same outcome. Analysis of these truth tables assesses first whether any conditions are individually necessary or sufficient to predict the outcome, and then whether any configurations of conditions are necessary or sufficient. Necessary conditions are assessed by examining causal conditions shared by cases with the same outcome, whilst identifying sufficient conditions (or combinations of conditions) requires examining cases with the same causal conditions to identify if they have the same outcome [ 15 ]. However, as Legewie argues, the presence of a condition, or a combination of conditions in actual datasets, are likely to be “‘quasi-necessary’ or ‘quasi-sufficient’ in that the causal relation holds in a great majority of cases, but some cases deviate from this pattern” [ 20 ]. Following reduction of the complexity of the model, the final model is tested for coverage (the degree to which a configuration accounts for instances of an outcome in the empirical cases; the proportion of cases belonging to a particular configuration) and consistency (the degree to which the cases sharing a combination of conditions align with a proposed subset relation). The result is an analysis of complex causation, “defined as a situation in which an outcome may follow from several different combinations of causal conditions” [ 15 ] illuminating the ‘causal recipes’, the causally relevant conditions or configuration of conditions that produce the outcome of interest.

QCA, then, has promise for addressing questions of complex causation, and recent calls for the greater use of QCA methods have come from a range of fields related to public health, including health research [ 17 ], studies of social interventions [ 7 ], and policy evaluation [ 21 , 22 ]. In making arguments for the use of QCA across these fields, researchers have also indicated some of the considerations that must be taken into account to ensure robust and credible analyses. There is a need, for instance, to ensure that ‘contradictions’, where cases with the same configurations show different outcomes, are resolved and reported [ 15 , 23 , 24 ]. Additionally, researchers must consider the ratio of cases to conditions, and limit the number of conditions to cases to ensure the validity of models [ 25 ]. Marx and Dusa, examining crisp set QCA, have provided some guidance to the ‘ceiling’ number of conditions which can be included relative to the number of cases to increase the probability of models being valid (that is, with a low probability of being generated through random data) [ 26 ].

There is now a growing body of published research in public health and related fields drawing on QCA methods. This is therefore a timely point to map the field and assess the potential of QCA as a method for contributing to the evidence base for what works in improving public health. To inform future methodological development of robust methods for addressing complexity in the evaluation of public health interventions, we undertook a systematic review to map existing evidence, identify gaps in, and strengths and weakness of, the QCA literature to date, and identify the implications of these for conducting and reporting future QCA studies for public health evaluation. We aimed to address the following specific questions [ 27 ]:

1. How is QCA used for public health evaluation? What populations, settings, methods used in source case studies, unit/s and level of analysis (‘cases’), and ‘conditions’ have been included in QCA studies?

2. What strengths and weaknesses have been identified by researchers who have used QCA to understand complex causation in public health evaluation research?

3. What are the existing gaps in, and strengths and weakness of, the QCA literature in public health evaluation, and what implications do these have for future research and reporting of QCA studies for public health?

This systematic review was registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 29 April 2019 ( CRD42019131910 ). A protocol was prepared in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) 2015 statement [ 28 ], and published in 2019 [ 27 ], where the methods are explained in detail. EPPI-Reviewer 4 was used to manage the process and undertake screening of abstracts [ 29 ].

Search strategy

We searched for peer-reviewed published papers in English, which used QCA methods to examine causal complexity in evaluating the implementation, uptake and/or effects of a public health intervention, in any region of the world, for any population. ‘Public health interventions’ were defined as those which aim to promote or protect health, or prevent ill health, in the population. No date exclusions were made, and papers published up to December 2019 were included.

Search strategies used the following phrases “Qualitative Comparative Analysis” and “QCA”, which were combined with the keywords “health”, “public health”, “intervention”, and “wellbeing”. See Additional file  1 for an example. Searches were undertaken on the following databases: PubMed, Web of Science, and Scopus. Additional searches were undertaken on Microsoft Academic and Google Scholar in December 2019, where the first pages of results were checked for studies that may have been missed in the initial search. No additional studies were identified. The list of included studies was sent to experts in QCA methods in health and related fields, including authors of included studies and/or those who had published on QCA methodology. This generated no additional studies within scope, but a suggestion to check the COMPASSS (Comparative Methods for Systematic Cross-Case Analysis) database; this was searched, identifying one further study that met the inclusion criteria [ 30 ]. COMPASSS ( https://compasss.org/ ) collates publications of studies using comparative case analysis.

We excluded studies where no intervention was evaluated, which included studies that used QCA to examine public health infrastructure (i.e. staff training) without a specific health outcome, and papers that report on prevalence of health issues (i.e. prevalence of child mortality). We also excluded studies of health systems or services interventions where there was no public health outcome.

After retrieval, and removal of duplicates, titles and abstracts were screened by one of two authors (BH or JG). Double screening of all records was assisted by EPPI Reviewer 4’s machine learning function. Of the 1384 papers identified after duplicates were removed, we excluded 820 after review of titles and abstracts (Fig.  1 ). The excluded studies included: a large number of papers relating to ‘quantitative coronary angioplasty’ and some which referred to the Queensland Criminal Code (both of which are also abbreviated to ‘QCA’); papers that reported methodological issues but not empirical studies; protocols; and papers that used the phrase ‘qualitative comparative analysis’ to refer to qualitative studies that compared different sub-populations or cases within the study, but did not include formal QCA methods.

Flow Diagram

Full texts of the 51 remaining studies were screened by BH and JG for inclusion, with 10 papers double coded by both authors, with complete agreement. Uncertain inclusions were checked by the third author (MP). Of the full texts, 24 were excluded because: they did not report a public health intervention ( n  = 18); had used a methodology inspired by QCA, but had not undertaken a QCA ( n  = 2); were protocols or methodological papers only ( n  = 2); or were not published in peer-reviewed journals ( n  = 2) (see Fig.  1 ).

Data were extracted manually from the 27 remaining full texts by BH and JG. Two papers relating to the same research question and dataset were combined, such that analysis was by study ( n  = 26) not by paper. We retrieved data relating to: publication (journal, first author country affiliation, funding reported); the study setting (country/region setting, population targeted by the intervention(s)); intervention(s) studied; methods (aims, rationale for using QCA, crisp or fuzzy set QCA, other analysis methods used); data sources drawn on for cases (source [primary data, secondary data, published analyses], qualitative/quantitative data, level of analysis, number of cases, final causal conditions included in the analysis); outcome explained; and claims made about strengths and weaknesses of using QCA (see Table  1 ). Data were synthesised narratively, using thematic synthesis methods [ 31 , 32 ], with interventions categorised by public health domain and level of intervention.

Quality assessment

There are no reporting guidelines for QCA studies in public health, but there are a number of discussions of best practice in the methodological literature [ 25 , 26 , 33 , 34 ]. These discussions suggest several criteria for strengthening QCA methods that we used as indicators of methodological and/or reporting quality: evidence of familiarity of cases; justification for selection of cases; discussion and justification of set membership score calibration; reporting of truth tables; reporting and justification of solution formula; and reporting of consistency and coverage measures. For studies using csQCA, and claiming an explanatory analysis, we additionally identified whether the number of cases was sufficient for the number of conditions included in the model, using a pragmatic cut-off in line with Marx & Dusa’s guideline thresholds, which indicate how many cases are sufficient for given numbers of conditions to reject a 10% probability that models could be generated with random data [ 26 ].

Overview of scope of QCA research in public health

Twenty-seven papers reporting 26 studies were included in the review (Table  1 ). The earliest was published in 2005, and 17 were published after 2015. The majority ( n  = 19) were published in public health/health promotion journals, with the remainder published in other health science ( n  = 3) or in social science/management journals ( n  = 4). The public health domain(s) addressed by each study were broadly coded by the main area of focus. They included nutrition/obesity ( n  = 8); physical activity (PA) (n = 4); health inequalities ( n  = 3); mental health ( n  = 2); community engagement ( n  = 3); chronic condition management ( n  = 3); vaccine adoption or implementation (n = 2); programme implementation ( n  = 3); breastfeeding ( n  = 2); or general population health ( n  = 1). The majority ( n  = 24) of studies were conducted solely or predominantly in high-income countries (systematic reviews in general searched global sources, but commented that the overwhelming majority of studies were from high-income countries). Country settings included: any ( n  = 6); OECD countries ( n  = 3); USA ( n  = 6); UK ( n  = 6) and one each from Nepal, Austria, Belgium, Netherlands and Africa. These largely reflected the first author’s country affiliations in the UK ( n  = 13); USA ( n  = 9); and one each from South Africa, Austria, Belgium, and the Netherlands. All three studies primarily addressing health inequalities [ 35 , 36 , 37 ] were from the UK.

Eight of the interventions evaluated were individual-level behaviour change interventions (e.g. weight management interventions, case management, self-management for chronic conditions); eight evaluated policy/funding interventions; five explored settings-based health promotion/behaviour change interventions (e.g. schools-based physical activity intervention, store-based food choice interventions); three evaluated community empowerment/engagement interventions, and two studies evaluated networks and their impact on health outcomes.

Methods and data sets used

Fifteen studies used crisp sets (csQCA), 11 used fuzzy sets (fsQCA). No study used mvQCA. Eleven studies included additional analyses of the datasets drawn on for the QCA, including six that used qualitative approaches (narrative synthesis, case comparisons), typically to identify cases or conditions for populating the QCA; and four reporting additional statistical analyses (meta-regression, linear regression) to either identify differences overall between cases prior to conducting a QCA (e.g. [ 38 ]) or to explore correlations in more detail (e.g. [ 39 ]). One study used an additional Boolean configurational technique to reduce the number of conditions in the QCA analysis [ 40 ]. No studies reported aiming to compare the findings from the QCA with those from other techniques for evaluating the uptake or effectiveness of interventions, although some [ 41 , 42 ] were explicitly using the study to showcase the possibilities of QCA compared with other approaches in general. Twelve studies drew on primary data collected specifically for the study, with five of those additionally drawing on secondary data sets; five drew only on secondary data sets, and nine used data from systematic reviews of published research. Seven studies drew primarily on qualitative data, generally derived from interviews or observations.

Many studies were undertaken in the context of one or more trials, which provided evidence of effect. Within single trials, this was generally for a process evaluation, with cases being trial sites. Fernald et al’s study, for instance, was in the context of a trial of a programme to support primary care teams in identifying and implementing self-management support tools for their patients, which measured patient and health care provider level outcomes [ 43 ]. The QCA reported here used qualitative data from the trial to identify a set of necessary conditions for health care provider practices to implement the tools successfully. In studies drawing on data from systematic reviews, cases were always at the level of intervention or intervention component, with data included from multiple trials. Harris et al., for instance, undertook a mixed-methods systematic review of school-based self-management interventions for asthma, using meta-analysis methods to identify effective interventions and QCA methods to identify which intervention features were aligned with success [ 44 ].

The largest number of studies ( n  = 10), including all the systematic reviews, analysed cases at the level of the intervention, or a component of the intervention; seven analysed organisational level cases (e.g. school class, network, primary care practice); five analysed sub-national region level cases (e.g. state, local authority area), and two each analysed country or individual level cases. Sample sizes ranged from 10 to 131, with no study having small N (< 10) sample sizes, four having large N (> 50) sample sizes, and the majority (22) being medium N studies (in the range 10–50).

Rationale for using QCA

Most papers reported a rationale for using QCA that mentioned ‘complexity’ or ‘context’, including: noting that QCA is appropriate for addressing causal complexity or multiple pathways to outcome [ 37 , 43 , 45 , 46 , 47 , 48 , 49 , 50 , 51 ]; noting the appropriateness of the method for providing evidence on how context impacts on interventions [ 41 , 50 ]; or the need for a method that addressed causal asymmetry [ 52 ]. Three stated that the QCA was an ‘exploratory’ analysis [ 53 , 54 , 55 ]. In addition to the empirical aims, several papers (e.g. [ 42 , 48 ]) sought to demonstrate the utility of QCA, or to develop QCA methods for health research (e.g. [ 47 ]).

Reported strengths and weaknesses of approach

There was a general agreement about the strengths of QCA. Specifically, that it was a useful tool to address complex causality, providing a systematic approach to understand the mechanisms at work in implementation across contexts [ 38 , 39 , 43 , 45 , 46 , 47 , 55 , 56 , 57 ], particularly as they relate to (in) effective intervention implementation [ 44 , 51 ] and the evaluation of interventions [ 58 ], or “where it is not possible to identify linearity between variables of interest and outcomes” [ 49 ]. Authors highlighted the strengths of QCA as providing possibilities for examining complex policy problems [ 37 , 59 ]; for testing existing as well as new theory [ 52 ]; and for identifying aspects of interventions which had not been previously perceived as critical [ 41 ] or which may have been missed when drawing on statistical methods that use, for instance, linear additive models [ 42 ]. The strengths of QCA in terms of providing useful evidence for policy were flagged in a number of studies, particularly where the causal recipes suggested that conventional assumptions about effectiveness were not confirmed. Blackman et al., for instance, in a series of studies exploring why unequal health outcomes had narrowed in some areas of the UK and not others, identified poorer outcomes in settings with ‘better’ contracting [ 35 , 36 , 37 ]; Harting found, contrary to theoretical assumptions about the necessary conditions for successful implementation of public health interventions, that a multisectoral network was not a necessary condition [ 30 ].

Weaknesses reported included the limitations of QCA in general for addressing complexity, as well as specific limitations with either the csQCA or the fsQCA methods employed. One general concern discussed across a number of studies was the problem of limited empirical diversity, which resulted in: limitations in the possible number of conditions included in each study, particularly with small N studies [ 58 ]; missing data on important conditions [ 43 ]; or limited reported diversity (where, for instance, data were drawn from systematic reviews, reflecting publication biases which limit reporting of ineffective interventions) [ 41 ]. Reported methodological limitations in small and intermediate N studies included concerns about the potential that case selection could bias findings [ 37 ].

In terms of potential for addressing causal complexity, the limitations of QCA for identifying unintended consequences, tipping points, and/or feedback loops in complex adaptive systems were noted [ 60 ], as were the potential limitations (especially in csQCA studies) of reducing complex conditions, drawn from detailed qualitative understanding, to binary conditions [ 35 ]. The impossibility of doing this was a rationale for using fsQCA in one study [ 57 ], where detailed knowledge of conditions is needed to make theoretically justified calibration decisions. However, others [ 47 ] make the case that csQCA provides more appropriate findings for policy: dichotomisation forces a focus on meaningful distinctions, including those related to decisions that practitioners/policy makers can action. There is, then, a potential trade-off in providing ‘interpretable results’, but ones which preclude potential for utilising more detailed information [ 45 ]. That QCA does not deal with probabilistic causation was noted [ 47 ].

Quality of published studies

Assessment of ‘familiarity with cases’ was made subjectively on the basis of study authors’ reports of their knowledge of the settings (empirical or theoretical) and the descriptions they provided in the published paper: overall, 14 were judged as sufficient, and 12 less than sufficient. Studies which included primary data were more likely to be judged as demonstrating familiarity ( n  = 10) than those drawing on secondary sources or systematic reviews, of which only two were judged as demonstrating familiarity. All studies justified how the selection of cases had been made; for those not using the full available population of cases, this was in general (appropriately) done theoretically: following previous research [ 52 ]; purposively to include a range of positive and negative outcomes [ 41 ]; or to include a diversity of cases [ 58 ]. In identifying conditions leading to effective/not effective interventions, one purposive strategy was to include a specified percentage or number of the most effective and least effective interventions (e.g. [ 36 , 40 , 51 , 52 ]). Discussion of calibration of set membership scores was judged adequate in 15 cases, and inadequate in 11; 10 reported raw data matrices in the paper or supplementary material; 21 reported truth tables in the paper or supplementary material. The majority ( n  = 21) reported at least some detail on the coverage (the number of cases with a particular configuration) and consistency (the percentage of similar causal configurations which result in the same outcome). The majority ( n  = 21) included truth tables (or explicitly provided details of how to obtain them); fewer ( n  = 10) included raw data. Only five studies met all six of these quality criteria (evidence of familiarity with cases, justification of case selection, discussion of calibration, reporting truth tables, reporting raw data matrices, reporting coverage and consistency); a further six met at least five of them.

Of the csQCA studies which were not reporting an exploratory analysis, four appeared to have insufficient cases for the large number of conditions entered into at least one of the models reported, with a consequent risk to the validity of the QCA models [ 26 ].

QCA has been widely used in public health research over the last decade to advance understanding of causal inference in complex systems. In this review of published evidence to date, we have identified studies using QCA to examine the configurations of conditions that lead to particular outcomes across contexts. As noted by most study authors, QCA methods have promised advantages over probabilistic statistical techniques for examining causation where systems and/or interventions are complex, providing public health researchers with a method to test the multiple pathways (configurations of conditions), and necessary and sufficient conditions that lead to desired health outcomes.

The origins of QCA approaches are in comparative policy studies. Rihoux et al’s review of peer-reviewed journal articles using QCA methods published up to 2011 found the majority of published examples were from political science and sociology, with fewer than 5% of the 313 studies they identified coming from health sciences [ 61 ]. They also reported few examples of the method being used in policy evaluation and implementation studies [ 62 ]. In the decade since their review of the field [ 61 ], there has been an emerging body of evaluative work in health: we identified 26 studies in the field of public health alone, with the majority published in public health journals. Across these studies, QCA has been used for evaluative questions in a range of settings and public health domains to identify the conditions under which interventions are implemented and/or have evidence of effect for improving population health. All studies included a series of cases that included some with and some without the outcome of interest (such as behaviour change, successful programme implementation, or good vaccination uptake). The dominance of high-income countries in both intervention settings and author affiliations is disappointing, but reflects the disproportionate location of public health research in the global north more generally [ 63 ].

The largest single group of studies included were systematic reviews, using QCA to compare interventions (or intervention components) to identify successful (and non-successful) configurations of conditions across contexts. Here, the value of QCA lies in its potential for synthesis with quantitative meta-synthesis methods to identify the particular conditions or contexts in which interventions or components are effective. As Parrott et al. note, for instance, their meta-analysis could identify probabilistic effects of weight management programmes, and the QCA analysis enabled them to address the “role that the context of the [paediatric weight management] intervention has in influencing how, when, and for whom an intervention mix will be successful” [ 50 ]. However, using QCA to identify configurations of conditions that lead to effective or non- effective interventions across particular areas of population health is an application that does move away in some significant respects from the origins of the method. First, researchers drawing on evidence from systematic reviews for their data are reliant largely on published evidence for information on conditions (such as the organisational contexts in which interventions were implemented, or the types of behaviour change theory utilised). Although guidance for describing interventions [ 64 ] advises key aspects of context are included in reports, this may not include data on the full range of conditions that might be causally important, and review research teams may have limited knowledge of these ‘cases’ themselves. Second, less successful interventions are less likely to be published, potentially limiting the diversity of cases, particularly of cases with unsuccessful outcomes. A strength of QCA is the separate analysis of conditions leading to positive and negative outcomes: this is precluded where there is insufficient evidence on negative outcomes [ 50 ]. Third, when including a range of types of intervention, it can be unclear whether the cases included are truly comparable. A QCA study requires a high degree of theoretical and pragmatic case knowledge on the part of the researcher to calibrate conditions to qualitative anchors: it is reliant on deep understanding of complex contexts, and a familiarity with how conditions interact within and across contexts. Perhaps surprising is that only seven of the studies included here clearly drew on qualitative data, given that QCA is primarily seen as a method that requires thick, detailed knowledge of cases, particularly when the aim is to understand complex causation [ 8 ]. Whilst research teams conducting QCA in the context of systematic reviews may have detailed understanding in general of interventions within their spheres of expertise, they are unlikely to have this for the whole range of cases, particularly where a diverse set of contexts (countries, organisational settings) are included. Making a theoretical case for the valid comparability of such a case series is crucial. There may, then, be limitations in the portability of QCA methods for conducting studies entirely reliant on data from published evidence.

QCA was developed for small and medium N series of cases, and (as in the field more broadly, [ 61 ]), the samples in our studies predominantly had between 10 and 50 cases. However, there is increasing interest in the method as an alternative or complementary technique to regression-oriented statistical methods for larger samples [ 65 ], such as from surveys, where detailed knowledge of cases is likely to be replaced by theoretical knowledge of relationships between conditions (see [ 23 ]). The two larger N (> 100 cases) studies in our sample were an individual level analysis of survey data [ 46 , 47 ] and an analysis of intervention arms from a systematic review [ 50 ]. Larger sample sizes allow more conditions to be included in the analysis [ 23 , 26 ], although for evaluative research, where the aim is developing a causal explanation, rather than simply exploring patterns, there remains a limit to the number of conditions that can be included. As the number of conditions included increases, so too does the number of possible configurations, increasing the chance of unique combinations and of generating spurious solutions with a high level of consistency. As a rule of thumb, once the number of conditions exceeds 6–8 (with up to 50 cases) or 10 (for larger samples), the credibility of solutions may be severely compromised [ 23 ].

Strengths and weaknesses of the study

A systematic review has the potential advantages of transparency and rigour and, if not exhaustive, our search is likely to be representative of the body of research using QCA for evaluative public health research up to 2020. However, a limitation is the inevitable difficulty in operationalising a ‘public health’ intervention. Exclusions on scope are not straightforward, given that most social, environmental and political conditions impact on public health, and arguably a greater range of policy and social interventions (such as fiscal or trade policies) that have been the subject of QCA analyses could have been included, or a greater range of more clinical interventions. However, to enable a manageable number of papers to review, and restrict our focus to those papers that were most directly applicable to (and likely to be read by) those in public health policy and practice, we operationalised ‘public health interventions’ as those which were likely to be directly impacting on population health outcomes, or on behaviours (such as increased physical activity) where there was good evidence for causal relationships with public health outcomes, and where the primary research question of the study examined the conditions leading to those outcomes. This review has, of necessity, therefore excluded a considerable body of evidence likely to be useful for public health practice in terms of planning interventions, such as studies on how to better target smoking cessation [ 66 ] or foster social networks [ 67 ] where the primary research question was on conditions leading to these outcomes, rather than on conditions for outcomes of specific interventions. Similarly, there are growing number of descriptive epidemiological studies using QCA to explore factors predicting outcomes across such diverse areas as lupus and quality of life [ 68 ]; length of hospital stay [ 69 ]; constellations of factors predicting injury [ 70 ]; or the role of austerity, crisis and recession in predicting public health outcomes [ 71 ]. Whilst there is undoubtedly useful information to be derived from studying the conditions that lead to particular public health problems, these studies were not directly evaluating interventions, so they were also excluded.

Restricting our search to publications in English and to peer reviewed publications may have missed bodies of work from many regions, and has excluded research from non-governmental organisations using QCA methods in evaluation. As this is a rapidly evolving field, with relatively recent uptake in public health (all our included studies were after 2005), our studies may not reflect the most recent advances in the area.

Implications for conducting and reporting QCA studies

This systematic review has reviewed studies that deployed an emergent methodology, which has no reporting guidelines and has had, to date, a relatively low level of awareness among many potential evidence users in public health. For this reason, many of the studies reviewed were relatively detailed on the methods used, and the rationale for utilising QCA.

We did not assess quality directly, but used indicators of good practice discussed in QCA methodological literature, largely written for policy studies scholars, and often post-dating the publication dates of studies included in this review. It is also worth noting that, given the relatively recent development of QCA methods, methodological debate is still thriving on issues such as the reliability of causal inferences [ 72 ], alongside more general critiques of the usefulness of the method for policy decisions (see, for instance, [ 73 ]). The authors of studies included in this review also commented directly on methodological development: for instance, Thomas et al. suggests that QCA may benefit from methods development for sensitivity analyses around calibration decisions [ 42 ].

However, we selected quality criteria that, we argue, are relevant for public health research> Justifying the selection of cases, discussing and justifying the calibration of set membership, making data sets available, and reporting truth tables, consistency and coverage are all good practice in line with the usual requirements of transparency and credibility in methods. When QCA studies aim to provide explanation of outcomes (rather than exploring configurations), it is also vital that they are reported in ways that enhance the credibility of claims made, including justifying the number of conditions included relative to cases. Few of the studies published to date met all these criteria, at least in the papers included here (although additional material may have been provided in other publications). To improve the future discoverability and uptake up of QCA methods in public health, and to strengthen the credibility of findings from these methods, we therefore suggest the following criteria should be considered by authors and reviewers for reporting QCA studies which aim to provide causal evidence about the configurations of conditions that lead to implementation or outcomes:

The paper title and abstract state the QCA design;

The sampling unit for the ‘case’ is clearly defined (e.g.: patient, specified geographical population, ward, hospital, network, policy, country);

The population from which the cases have been selected is defined (e.g.: all patients in a country with X condition, districts in X country, tertiary hospitals, all hospitals in X country, all health promotion networks in X province, European policies on smoking in outdoor places, OECD countries);

The rationale for selection of cases from the population is justified (e.g.: whole population, random selection, purposive sample);

There are sufficient cases to provide credible coverage across the number of conditions included in the model, and the rationale for the number of conditions included is stated;

Cases are comparable;

There is a clear justification for how choices of relevant conditions (or ‘aspects of context’) have been made;

There is sufficient transparency for replicability: in line with open science expectations, datasets should be available where possible; truth tables should be reported in publications, and reports of coverage and consistency provided.

Implications for future research

In reviewing methods for evaluating natural experiments, Craig et al. focus on statistical techniques for enhancing causal inference, noting only that what they call ‘qualitative’ techniques (the cited references for these are all QCA studies) require “further studies … to establish their validity and usefulness” [ 2 ]. The studies included in this review have demonstrated that QCA is a feasible method when there are sufficient (comparable) cases for identifying configurations of conditions under which interventions are effective (or not), or are implemented (or not). Given ongoing concerns in public health about how best to evaluate interventions across complex contexts and systems, this is promising. This review has also demonstrated the value of adding QCA methods to the tool box of techniques for evaluating interventions such as public policies, health promotion programmes, and organisational changes - whether they are implemented in a randomised way or not. Many of the studies in this review have clearly generated useful evidence: whether this evidence has had more or less impact, in terms of influencing practice and policy, or is more valid, than evidence generated by other methods is not known. Validating the findings of a QCA study is perhaps as challenging as validating the findings from any other design, given the absence of any gold standard comparators. Comparisons of the findings of QCA with those from other methods are also typically constrained by the rather different research questions asked, and the different purposes of the analysis. In our review, QCA were typically used alongside other methods to address different questions, rather than to compare methods. However, as the field develops, follow up studies, which evaluate outcomes of interventions designed in line with conditions identified as causal in prior QCAs, might be useful for contributing to validation.

This review was limited to public health evaluation research: other domains that would be useful to map include health systems/services interventions and studies used to design or target interventions. There is also an opportunity to broaden the scope of the field, particularly for addressing some of the more intractable challenges for public health research. Given the limitations in the evidence base on what works to address inequalities in health, for instance [ 74 ], QCA has potential here, to help identify the conditions under which interventions do or do not exacerbate unequal outcomes, or the conditions that lead to differential uptake or impacts across sub-population groups. It is perhaps surprising that relatively few of the studies in this review included cases at the level of country or region, the traditional level for QCA studies. There may be scope for developing international comparisons for public health policy, and using QCA methods at the case level (nation, sub-national region) of classic policy studies in the field. In the light of debate around COVID-19 pandemic response effectiveness, comparative studies across jurisdictions might shed light on issues such as differential population responses to vaccine uptake or mask use, for example, and these might in turn be considered as conditions in causal configurations leading to differential morbidity or mortality outcomes.

When should be QCA be considered?

Public health evaluations typically assess the efficacy, effectiveness or cost-effectiveness of interventions and the processes and mechanisms through which they effect change. There is no perfect evaluation design for achieving these aims. As in other fields, the choice of design will in part depend on the availability of counterfactuals, the extent to which the investigator can control the intervention, and the range of potential cases and contexts [ 75 ], as well as political considerations, such as the credibility of the approach with key stakeholders [ 76 ]. There are inevitably ‘horses for courses’ [ 77 ]. The evidence from this review suggests that QCA evaluation approaches are feasible when there is a sufficient number of comparable cases with and without the outcome of interest, and when the investigators have, or can generate, sufficiently in-depth understanding of those cases to make sense of connections between conditions, and to make credible decisions about the calibration of set membership. QCA may be particularly relevant for understanding multiple causation (that is, where different configurations might lead to the same outcome), and for understanding the conditions associated with both lack of effect and effect. As a stand-alone approach, QCA might be particularly valuable for national and regional comparative studies of the impact of policies on public health outcomes. Alongside cluster randomised trials of interventions, or alongside systematic reviews, QCA approaches are especially useful for identifying core combinations of causal conditions for success and lack of success in implementation and outcome.

Conclusions

QCA is a relatively new approach for public health research, with promise for contributing to much-needed methodological development for addressing causation in complex systems. This review has demonstrated the large range of evaluation questions that have been addressed to date using QCA, including contributions to process evaluations of trials and for exploring the conditions leading to effectiveness (or not) in systematic reviews of interventions. There is potential for QCA to be more widely used in evaluative research, to identify the conditions under which interventions across contexts are implemented or not, and the configurations of conditions associated with effect or lack of evidence of effect. However, QCA will not be appropriate for all evaluations, and cannot be the only answer to addressing complex causality. For explanatory questions, the approach is most appropriate when there is a series of enough comparable cases with and without the outcome of interest, and where the researchers have detailed understanding of those cases, and conditions. To improve the credibility of findings from QCA for public health evidence users, we recommend that studies are reported with the usual attention to methodological transparency and data availability, with key details that allow readers to judge the credibility of causal configurations reported. If the use of QCA continues to expand, it may be useful to develop more comprehensive consensus guidelines for conduct and reporting.

Availability of data and materials

Full search strategies and extraction forms are available by request from the first author.

Abbreviations

Comparative Methods for Systematic Cross-Case Analysis

crisp set QCA

fuzzy set QCA

multi-value QCA

Medical Research Council

• Qualitative Comparative Analysis

randomised control trial

Physical Activity

Green J, Roberts H, Petticrew M, Steinbach R, Goodman A, Jones A, et al. Integrating quasi-experimental and inductive designs in evaluation: a case study of the impact of free bus travel on public health. Evaluation. 2015;21(4):391–406. https://doi.org/10.1177/1356389015605205 .

Article   Google Scholar  

Craig P, Katikireddi SV, Leyland A, Popham F. Natural experiments: an overview of methods, approaches, and contributions to public health intervention research. Annu Rev Public Health. 2017;38(1):39–56. https://doi.org/10.1146/annurev-publhealth-031816-044327 .

Article   PubMed   PubMed Central   Google Scholar  

Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ. 2008;336(7656):1281–3. https://doi.org/10.1136/bmj.39569.510521.AD .

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350(mar19 6):h1258. https://doi.org/10.1136/bmj.h1258 .

Pattyn V, Álamos-Concha P, Cambré B, Rihoux B, Schalembier B. Policy effectiveness through Configurational and mechanistic lenses: lessons for concept development. J Comp Policy Anal Res Pract. 2020;0:1–18.

Google Scholar  

Byrne D. Evaluating complex social interventions in a complex world. Evaluation. 2013;19(3):217–28. https://doi.org/10.1177/1356389013495617 .

Gerrits L, Pagliarin S. Social and causal complexity in qualitative comparative analysis (QCA): strategies to account for emergence. Int J Soc Res Methodol 2020;0:1–14, doi: https://doi.org/10.1080/13645579.2020.1799636 .

Grant RL, Hood R. Complex systems, explanation and policy: implications of the crisis of replication for public health research. Crit Public Health. 2017;27(5):525–32. https://doi.org/10.1080/09581596.2017.1282603 .

Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, et al. The need for a complex systems model of evidence for public health. Lancet. 2017;390(10112):2602–4. https://doi.org/10.1016/S0140-6736(17)31267-9 .

Article   PubMed   Google Scholar  

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95. https://doi.org/10.1186/s12916-018-1089-4 .

Craig P, Di Ruggiero E, Frohlich KL, Mykhalovskiy E and White M, on behalf of the Canadian Institutes of Health Research (CIHR)–National Institute for Health Research (NIHR) Context Guidance Authors Group. Taking account of context in population health intervention research: guidance for producers, users and funders of research. Southampton: NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.

Paparini S, Green J, Papoutsi C, Murdoch J, Petticrew M, Greenhalgh T, et al. Case study research for better evaluations of complex interventions: rationale and challenges. BMC Med. 2020;18(1):301. https://doi.org/10.1186/s12916-020-01777-6 .

Ragin. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: University of California Press; 1987.

Ragin C. Redesigning social inquiry: fuzzy sets and beyond - Charles C: Ragin - Google Books. The University of Chicago Press; 2008. https://doi.org/10.7208/chicago/9780226702797.001.0001 .

Book   Google Scholar  

Befani B, Ledermann S, Sager F. Realistic evaluation and QCA: conceptual parallels and an empirical application. Evaluation. 2007;13(2):171–92. https://doi.org/10.1177/1356389007075222 .

Kane H, Lewis MA, Williams PA, Kahwati LC. Using qualitative comparative analysis to understand and quantify translation and implementation. Transl Behav Med. 2014;4(2):201–8. https://doi.org/10.1007/s13142-014-0251-6 .

Cronqvist L, Berg-Schlosser D. Chapter 4: Multi-Value QCA (mvQCA). In: Rihoux B, Ragin C, editors. Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques. 2455 Teller Road, Thousand Oaks California 91320 United States: SAGE Publications, Inc.; 2009. p. 69–86. doi: https://doi.org/10.4135/9781452226569 .

Ragin CC. Using qualitative comparative analysis to study causal complexity. Health Serv Res. 1999;34(5 Pt 2):1225–39.

CAS   PubMed   PubMed Central   Google Scholar  

Legewie N. An introduction to applied data analysis with qualitative comparative analysis (QCA). Forum Qual Soc Res. 2013;14.  https://doi.org/10.17169/fqs-14.3.1961 .

Varone F, Rihoux B, Marx A. A new method for policy evaluation? In: Rihoux B, Grimm H, editors. Innovative comparative methods for policy analysis: beyond the quantitative-qualitative divide. Boston: Springer US; 2006. p. 213–36. https://doi.org/10.1007/0-387-28829-5_10 .

Chapter   Google Scholar  

Gerrits L, Verweij S. The evaluation of complex infrastructure projects: a guide to qualitative comparative analysis. Cheltenham: Edward Elgar Pub; 2018. https://doi.org/10.4337/9781783478422 .

Greckhamer T, Misangyi VF, Fiss PC. The two QCAs: from a small-N to a large-N set theoretic approach. In: Configurational Theory and Methods in Organizational Research. Emerald Group Publishing Ltd.; 2013. p. 49–75. https://pennstate.pure.elsevier.com/en/publications/the-two-qcas-from-a-small-n-to-a-large-n-set-theoretic-approach . Accessed 16 Apr 2021.

Rihoux B, Ragin CC. Configurational comparative methods: qualitative comparative analysis (QCA) and related techniques. SAGE; 2009, doi: https://doi.org/10.4135/9781452226569 .

Marx A. Crisp-set qualitative comparative analysis (csQCA) and model specification: benchmarks for future csQCA applications. Int J Mult Res Approaches. 2010;4(2):138–58. https://doi.org/10.5172/mra.2010.4.2.138 .

Marx A, Dusa A. Crisp-set qualitative comparative analysis (csQCA), contradictions and consistency benchmarks for model specification. Methodol Innov Online. 2011;6(2):103–48. https://doi.org/10.4256/mio.2010.0037 .

Hanckel B, Petticrew M, Thomas J, Green J. Protocol for a systematic review of the use of qualitative comparative analysis for evaluative questions in public health research. Syst Rev. 2019;8(1):252. https://doi.org/10.1186/s13643-019-1159-5 .

Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349(1):g7647. https://doi.org/10.1136/bmj.g7647 .

EPPI-Reviewer 4.0: Software for research synthesis. UK: University College London; 2010.

Harting J, Peters D, Grêaux K, van Assema P, Verweij S, Stronks K, et al. Implementing multiple intervention strategies in Dutch public health-related policy networks. Health Promot Int. 2019;34(2):193–203. https://doi.org/10.1093/heapro/dax067 .

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):45. https://doi.org/10.1186/1471-2288-8-45 .

Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews: a product from the ESRC methods Programme. 2006.

Wagemann C, Schneider CQ. Qualitative comparative analysis (QCA) and fuzzy-sets: agenda for a research approach and a data analysis technique. Comp Sociol. 2010;9:376–96.

Schneider CQ, Wagemann C. Set-theoretic methods for the social sciences: a guide to qualitative comparative analysis: Cambridge University Press; 2012. https://doi.org/10.1017/CBO9781139004244 .

Blackman T, Dunstan K. Qualitative comparative analysis and health inequalities: investigating reasons for differential Progress with narrowing local gaps in mortality. J Soc Policy. 2010;39(3):359–73. https://doi.org/10.1017/S0047279409990675 .

Blackman T, Wistow J, Byrne D. A Qualitative Comparative Analysis of factors associated with trends in narrowing health inequalities in England. Soc Sci Med 1982. 2011;72:1965–74.

Blackman T, Wistow J, Byrne D. Using qualitative comparative analysis to understand complex policy problems. Evaluation. 2013;19(2):126–40. https://doi.org/10.1177/1356389013484203 .

Glatman-Freedman A, Cohen M-L, Nichols KA, Porges RF, Saludes IR, Steffens K, et al. Factors affecting the introduction of new vaccines to poor nations: a comparative study of the haemophilus influenzae type B and hepatitis B vaccines. PLoS One. 2010;5(11):e13802. https://doi.org/10.1371/journal.pone.0013802 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ford EW, Duncan WJ, Ginter PM. Health departments’ implementation of public health’s core functions: an assessment of health impacts. Public Health. 2005;119(1):11–21. https://doi.org/10.1016/j.puhe.2004.03.002 .

Article   CAS   PubMed   Google Scholar  

Lucidarme S, Cardon G, Willem A. A comparative study of health promotion networks: configurations of determinants for network effectiveness. Public Manag Rev. 2016;18(8):1163–217. https://doi.org/10.1080/14719037.2015.1088567 .

Melendez-Torres GJ, Sutcliffe K, Burchett HED, Rees R, Richardson M, Thomas J. Weight management programmes: re-analysis of a systematic review to identify pathways to effectiveness. Health Expect Int J Public Particip Health Care Health Policy. 2018;21:574–84.

CAS   Google Scholar  

Thomas J, O’Mara-Eves A, Brunton G. Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev. 2014;3(1):67. https://doi.org/10.1186/2046-4053-3-67 .

Fernald DH, Simpson MJ, Nease DE, Hahn DL, Hoffmann AE, Michaels LC, et al. Implementing community-created self-management support tools in primary care practices: multimethod analysis from the INSTTEPP study. J Patient-Centered Res Rev. 2018;5(4):267–75. https://doi.org/10.17294/2330-0698.1634 .

Harris K, Kneale D, Lasserson TJ, McDonald VM, Grigg J, Thomas J. School-based self-management interventions for asthma in children and adolescents: a mixed methods systematic review. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD011651.pub2 .

Kahwati LC, Lewis MA, Kane H, Williams PA, Nerz P, Jones KR, et al. Best practices in the veterans health Administration’s MOVE! Weight management program. Am J Prev Med. 2011;41(5):457–64. https://doi.org/10.1016/j.amepre.2011.06.047 .

Warren J, Wistow J, Bambra C. Applying qualitative comparative analysis (QCA) to evaluate a public health policy initiative in the north east of England. Polic Soc. 2013;32(4):289–301. https://doi.org/10.1016/j.polsoc.2013.10.002 .

Warren J, Wistow J, Bambra C. Applying qualitative comparative analysis (QCA) in public health: a case study of a health improvement service for long-term incapacity benefit recipients. J Public Health. 2014;36(1):126–33. https://doi.org/10.1093/pubmed/fdt047 .

Article   CAS   Google Scholar  

Brunton G, O’Mara-Eves A, Thomas J. The “active ingredients” for successful community engagement with disadvantaged expectant and new mothers: a qualitative comparative analysis. J Adv Nurs. 2014;70(12):2847–60. https://doi.org/10.1111/jan.12441 .

McGowan VJ, Wistow J, Lewis SJ, Popay J, Bambra C. Pathways to mental health improvement in a community-led area-based empowerment initiative: evidence from the big local ‘communities in control’ study. England J Public Health. 2019;41(4):850–7. https://doi.org/10.1093/pubmed/fdy192 .

Parrott JS, Henry B, Thompson KL, Ziegler J, Handu D. Managing Complexity in Evidence Analysis: A Worked Example in Pediatric Weight Management. J Acad Nutr Diet. 2018;118:1526–1542.e3.

Kien C, Grillich L, Nussbaumer-Streit B, Schoberberger R. Pathways leading to success and non-success: a process evaluation of a cluster randomized physical activity health promotion program applying fuzzy-set qualitative comparative analysis. BMC Public Health. 2018;18(1):1386. https://doi.org/10.1186/s12889-018-6284-x .

Lubold AM. The effect of family policies and public health initiatives on breastfeeding initiation among 18 high-income countries: a qualitative comparative analysis research design. Int Breastfeed J. 2017;12(1):34. https://doi.org/10.1186/s13006-017-0122-0 .

Bianchi F, Garnett E, Dorsel C, Aveyard P, Jebb SA. Restructuring physical micro-environments to reduce the demand for meat: a systematic review and qualitative comparative analysis. Lancet Planet Health. 2018;2(9):e384–97. https://doi.org/10.1016/S2542-5196(18)30188-8 .

Bianchi F, Dorsel C, Garnett E, Aveyard P, Jebb SA. Interventions targeting conscious determinants of human behaviour to reduce the demand for meat: a systematic review with qualitative comparative analysis. Int J Behav Nutr Phys Act. 2018;15(1):102. https://doi.org/10.1186/s12966-018-0729-6 .

Hartmann-Boyce J, Bianchi F, Piernas C, Payne Riches S, Frie K, Nourse R, et al. Grocery store interventions to change food purchasing behaviors: a systematic review of randomized controlled trials. Am J Clin Nutr. 2018;107(6):1004–16. https://doi.org/10.1093/ajcn/nqy045 .

Burchett HED, Sutcliffe K, Melendez-Torres GJ, Rees R, Thomas J. Lifestyle weight management programmes for children: a systematic review using qualitative comparative analysis to identify critical pathways to effectiveness. Prev Med. 2018;106:1–12. https://doi.org/10.1016/j.ypmed.2017.08.025 .

Chiappone A. Technical assistance and changes in nutrition and physical activity practices in the National Early Care and education learning Collaboratives project, 2015–2016. Prev Chronic Dis. 2018;15. https://doi.org/10.5888/pcd15.170239 .

Kane H, Hinnant L, Day K, Council M, Tzeng J, Soler R, et al. Pathways to program success: a qualitative comparative analysis (QCA) of communities putting prevention to work case study programs. J Public Health Manag Pract JPHMP. 2017;23(2):104–11. https://doi.org/10.1097/PHH.0000000000000449 .

Roberts MC, Murphy T, Moss JL, Wheldon CW, Psek W. A qualitative comparative analysis of combined state health policies related to human papillomavirus vaccine uptake in the United States. Am J Public Health. 2018;108(4):493–9. https://doi.org/10.2105/AJPH.2017.304263 .

Breuer E, Subba P, Luitel N, Jordans M, Silva MD, Marchal B, et al. Using qualitative comparative analysis and theory of change to unravel the effects of a mental health intervention on service utilisation in Nepal. BMJ Glob Health. 2018;3(6):e001023. https://doi.org/10.1136/bmjgh-2018-001023 .

Rihoux B, Álamos-Concha P, Bol D, Marx A, Rezsöhazy I. From niche to mainstream method? A comprehensive mapping of QCA applications in journal articles from 1984 to 2011. Polit Res Q. 2013;66:175–84.

Rihoux B, Rezsöhazy I, Bol D. Qualitative comparative analysis (QCA) in public policy analysis: an extensive review. Ger Policy Stud. 2011;7:9–82.

Plancikova D, Duric P, O’May F. High-income countries remain overrepresented in highly ranked public health journals: a descriptive analysis of research settings and authorship affiliations. Crit Public Health 2020;0:1–7, DOI: https://doi.org/10.1080/09581596.2020.1722313 .

Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348(mar07 3):g1687. https://doi.org/10.1136/bmj.g1687 .

Fiss PC, Sharapov D, Cronqvist L. Opposites attract? Opportunities and challenges for integrating large-N QCA and econometric analysis. Polit Res Q. 2013;66:191–8.

Blackman T. Can smoking cessation services be better targeted to tackle health inequalities? Evidence from a cross-sectional study. Health Educ J. 2008;67(2):91–101. https://doi.org/10.1177/0017896908089388 .

Haynes P, Banks L, Hill M. Social networks amongst older people in OECD countries: a qualitative comparative analysis. J Int Comp Soc Policy. 2013;29(1):15–27. https://doi.org/10.1080/21699763.2013.802988 .

Rioja EC. Valero-Moreno S, Giménez-Espert M del C, Prado-Gascó V. the relations of quality of life in patients with lupus erythematosus: regression models versus qualitative comparative analysis. J Adv Nurs. 2019;75(7):1484–92. https://doi.org/10.1111/jan.13957 .

Dy SM. Garg Pushkal, Nyberg Dorothy, Dawson Patricia B., Pronovost Peter J., Morlock Laura, et al. critical pathway effectiveness: assessing the impact of patient, hospital care, and pathway characteristics using qualitative comparative analysis. Health Serv Res. 2005;40(2):499–516. https://doi.org/10.1111/j.1475-6773.2005.0r370.x .

MELINDER KA, ANDERSSON R. The impact of structural factors on the injury rate in different European countries. Eur J Pub Health. 2001;11(3):301–8. https://doi.org/10.1093/eurpub/11.3.301 .

Saltkjel T, Holm Ingelsrud M, Dahl E, Halvorsen K. A fuzzy set approach to economic crisis, austerity and public health. Part II: How are configurations of crisis and austerity related to changes in population health across Europe? Scand J Public Health. 2017;45(18_suppl):48–55.

Baumgartner M, Thiem A. Often trusted but never (properly) tested: evaluating qualitative comparative analysis. Sociol Methods Res. 2020;49(2):279–311. https://doi.org/10.1177/0049124117701487 .

Tanner S. QCA is of questionable value for policy research. Polic Soc. 2014;33(3):287–98. https://doi.org/10.1016/j.polsoc.2014.08.003 .

Mackenbach JP. Tackling inequalities in health: the need for building a systematic evidence base. J Epidemiol Community Health. 2003;57(3):162. https://doi.org/10.1136/jech.57.3.162 .

Stern E, Stame N, Mayne J, Forss K, Davies R, Befani B. Broadening the range of designs and methods for impact evaluations. Technical report. London: DfiD; 2012.

Pattyn V. Towards appropriate impact evaluation methods. Eur J Dev Res. 2019;31(2):174–9. https://doi.org/10.1057/s41287-019-00202-w .

Petticrew M, Roberts H. Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health. 2003;57(7):527–9. https://doi.org/10.1136/jech.57.7.527 .

Download references

Acknowledgements

The authors would like to thank and acknowledge the support of Sara Shaw, PI of MR/S014632/1 and the rest of the Triple C project team, the experts who were consulted on the final list of included studies, and the reviewers who provided helpful feedback on the original submission.

This study was funded by MRC: MR/S014632/1 ‘Case study, context and complex interventions (Triple C): development of guidance and publication standards to support case study research’. The funder played no part in the conduct or reporting of the study. JG is supported by a Wellcome Trust Centre grant 203109/Z/16/Z.

Author information

Authors and affiliations.

Institute for Culture and Society, Western Sydney University, Sydney, Australia

Benjamin Hanckel

Department of Public Health, Environments and Society, LSHTM, London, UK

Mark Petticrew

UCL Institute of Education, University College London, London, UK

James Thomas

Wellcome Centre for Cultures & Environments of Health, University of Exeter, Exeter, UK

Judith Green

You can also search for this author in PubMed   Google Scholar

Contributions

BH - research design, data acquisition, data extraction and coding, data interpretation, paper drafting; JT – research design, data interpretation, contributing to paper; MP – funding acquisition, research design, data interpretation, contributing to paper; JG – funding acquisition, research design, data extraction and coding, data interpretation, paper drafting. All authors approved the final version.

Corresponding author

Correspondence to Judith Green .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Competing interests

All authors declare they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Example search strategy.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hanckel, B., Petticrew, M., Thomas, J. et al. The use of Qualitative Comparative Analysis (QCA) to address causality in complex systems: a systematic review of research on public health interventions. BMC Public Health 21 , 877 (2021). https://doi.org/10.1186/s12889-021-10926-2

Download citation

Received : 03 February 2021

Accepted : 22 April 2021

Published : 07 May 2021

DOI : https://doi.org/10.1186/s12889-021-10926-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Public health
• Intervention
• Systematic review

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Technical Support
• Find My Rep

You are here

Qualitative Comparative Analysis in Mixed Methods Research and Evaluation

• Leila C. Kahwati - RTI International
• Heather L. Kane - RTI International
• Description

Qualitative Comparative Analysis in Mixed Methods Research and Evaluation provides a user-friendly introduction for using Qualitative Comparative Analysis (QCA) as part of a mixed methods approach to research and evaluation. Offering practical, in-depth, and applied guidance for this unique analytic technique that is not provided in any current mixed methods textbook, the chapters of this guide skillfully build upon one another to walk researchers through the steps of QCA in logical order. To enhance and further reinforce learning, authors Leila C. Kahwati and Heather L. Kane provide supportive learning objectives, summaries, and exercises, as well as author-created datasets for use in R via the companion site.   Qualitative Comparative Analysis in Mixed Methods Research and Evaluation is Volume 6 in SAGE’s Mixed Methods Research Series. To learn more about each text in the series, please visit sagepub.com/mmrs .

See what’s new to this edition by selecting the Features tab on this page. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email [email protected] . Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html .

For assistance with your order: Please email us at [email protected] or connect with your SAGE representative.

SAGE 2455 Teller Road Thousand Oaks, CA 91320 www.sagepub.com

Supplements

“This book is written in a way that is easy to follow and should expand the range of fields in which QCA is used. Also, there are quite a few principles and practice tips articulated, especially in later chapters, which are applicable more broadly across social sciences and evaluation work. Novice researchers will find those suggestions especially helpful, even if QCA does not become a major tool in their practice.”

“The practical, how-to, nature of the text is very appealing to me as an instructor. I like the examples and appreciate the numerous figures used to illustrate processes and arguments for visual learners.”

“The text introduces an important, specific approach to research.”

“I think the key strengths of this text are its organization and breadth. From an organization perspective, the wealth of resources and focus is essential for guiding the reader/learner toward practical keywords, i.e. language, and skills necessary to implement.”

This is a very good resource for students and teaching

• Use of a concrete example that is woven across multiple chapters provides a thread of continuity that allows readers to follow the step-by-step process for understanding the method. 
• A guiding heuristic helps orient the reader at the beginning of each chapter to understand where they are in the process of conducting an analysis.
• Analytic checklists easily summarize the analytic process described in the chapter and serve as a reference.
• Practice exercises provide essential practice and reinforce key concepts.
• Helpful summaries and key points succinctly summarize main points of each chapter.

Sample Materials & Chapters

Chapter 1: Qualitative Comparative Analysis as Part of a Mixed Methods Approach

Chapter 5: Analyzing the Data -- Initial Analyses

For instructors

Select a purchasing option, related products.

This title is also available on SAGE Research Methods , the ultimate digital methods library. If your library doesn’t have access, ask your librarian to start a trial .

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Comparative Analysis of Qualitative And Quantitative Research

2019, M.Lib.I.Sc. Project, Panjab University, under guidance of Dr. Shiv Kumar

There's no hard and fast rule for qualitative versus quantitative research, and it's often taken for granted. It is claimed here that the divide between qualitative and quantitative research is ambiguous, incoherent, and hence of little value, and that its widespread use could have negative implications. This conclusion is supported by a variety of arguments. Qualitative researchers, for example, have varying perspectives on fundamental problems (such as the use of quantification and causal analysis), which makes the difference as such shaky. In addition, many elements of qualitative and quantitative research overlap significantly, making it difficult to distinguish between the two. Practically in the case of field research, the Qualitative and quantitative approach can't be distinguished clearly as the study pointed. The distinction may limit innovation in the development of new research methodologies, as well as cause complication and wasteful activity. As a general rule, it may be desirable not to conceptualise research approaches at such abstract levels as are done in the context of qualitative or quantitative methodologies. Discussions of the benefits and drawbacks of various research methods, rather than general research questions, are recommended.

Related Papers

Forum Qualitative Sozialforschung Forum Qualitative Social Research

Margrit Schreier

Fernando Almeida

Scientific research adopts qualitative and quantitative methodologies in the modeling and analysis of numerous phenomena. The qualitative methodology intends to understand a complex reality and the meaning of actions in a given context. On the other hand, the quantitative methodology seeks to obtain accurate and reliable measurements that allow a statistical analysis. Both methodologies offer a set of methods, potentialities and limitations that must be explored and known by researchers. This paper concisely maps a total of seven qualitative methods and five quantitative methods. A comparative analysis of the most relevant and adopted methods is done to understand the main strengths and limitations of them. Additionally, the work developed intends to be a fundamental reference for the accomplishment of a research study, in which the researcher intends to adopt a qualitative or quantitative methodology. Through the analysis of the advantages and disadvantages of each method, it becomes possible to formulate a more accurate, informed and complete choice.

IRJET Journal

Research design methods, such as qualitative, quantitative as well as mixed methods were introduced and subsequently each method was discussed in detail with the help of literature review as well as some personal and live examples to substantiate the findings of various literature. From various literature as well as from the own experiences, it is concluded that both qualitative research design method and quantitative research design method are equally important. It is not fair to criticize one method as the researcher is inclined towards the other method. It is practically evidenced that usage of both methods in the research, the researcher can substantiate the case better. However, duration part while using mixed methods to be kept in mind as it will take more time compared to the qualitative and quantitative methods. Hurrying and aborting in the middle due to time constraint ultimately result in poor research. It would be better if the world view towards these methods changes from criticizing mode to effective utilization mode, which will help research community in focusing and bring up better research outcomes rather than wasting time in arguing which method is scientifically acceptable and which method is biased. While I agree that the ontological, epistemological, axiological, and methodological assumptions for qualitative research method and quantitative research method, researchers should know fully about these methods and keep them as effective tools to utilize them in mixed mode, wherever it is appropriate and required to arrive at adequate research findings.

iosrjournals.org

UNICAF University - Zambia

Ivan Steenkamp

Gregoire Nleme

In this paper, as a practitioner, I describe quantitative and qualitative inquiries. I explain some of their differences and similarities. I emphasize most important differences that researchers should consider when selecting one of the methods or when running a mixed study. I list differences in worldviews, research design, research processes, reliability assurance, and validity assurance. I propose a process flowchart for each type of inquiry. The purpose of the essay is to give researchers and primarily doctorate students a review of differences and similarities between the two methods of inquiry.

Ziabur Rahman

Munyaradzi Moyo

Marta Costa

International Journal of Social Research Methodology

Julia Brannen

RELATED PAPERS

Felix Medina

Revista Conhecimento Online

Physical Review B

Nils Schopohl

Engenharia Agrícola

Eduardo Furlani

Drugs in Context

Alejandro Cabeza

F1000Research

swarnali banik

Journal of emerging technologies and innovative research

Seema Nihalani

Biometrics & Biostatistics International Journal

Rama Shanker

Rodolfo E . Mardones

RSC Advances

oscar rojas

Sukarni Sukarni

Revista de Administração de Empresas

Luis Felipe Sánchez Arias

Textos e Debates

Alessandra Ferreira dos Santos

Daniel Quiroz

Orbis Tertius

adriana mancini

Tomaso Poggio

International journal for quality in health care : journal of the International Society for Quality in Health Care

Iram Fatima

Majalah Ilmiah Teknologi Elektro

Widya Cahyadi

Biochemical Toxicology - Heavy Metals and Nanomaterials

Percilia Cardoso Giaquinto

Richard Volckmann

Applied and Environmental Microbiology

Turkiye Klinikleri Journal of Gynecology and Obstetrics

Prof.Dr.Birol Vural

Raghu Pandurangi

Hud毕业证文凭哈德斯菲尔德大学本科毕业证书 英国学历学位认证如何办理

See More Documents Like This

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

You are using an outdated browser. Please upgrade your browser to improve your experience.

Comparative Analysis of Qualitative And Quantitative Research

• 1. Master Of Library And Information Science

Description

There’s no hard and fast rule for qualitative versus quantitative research, and it’s often taken for granted. It is claimed here that the divide between qualitative and quantitative research is ambiguous, incoherent, and hence of little value, and that its widespread use could have negative implications. This conclusion is supported by a variety of arguments. Qualitative researchers, for example, have varying perspectives on fundamental problems (such as the use of quantification and causal analysis), which makes the difference as such shaky. In addition, many elements of qualitative and quantitative research overlap significantly, making it difficult to distinguish between the two. Practically in the case of field research, the Qualitative and quantitative approach can't be distinguished clearly as the study pointed. The distinction may limit innovation in the development of new research methodologies, as well as cause complication and wasteful activity. As a general rule, it may be desirable not to conceptualise research approaches at such abstract levels as are done in the context of qualitative or quantitative methodologies. Discussions of the benefits and drawbacks of various research methods, rather than general research questions, are recommended.

Comparative Analysis of Qualitative and Quantitative Research (SSRN).pdf

Files (246.0 kb), additional details, identifiers.

• Allwood, C. M. (2011). The distinction between qualitative and quantitative research methods is problematic. Quality & Quantity, 46(5), 1417–1429. https://doi.org/10.1007/s11135-011-9455-8
• Bryman, A. (2016). Social Research Methods (5th ed.). Oxford, UK: Oxford University Press.
• Coghlan, D., & Brydon-Miller, M. (2014). The SAGE Encyclopedia of Action Research. https://doi.org/10.4135/9781446294406
• Creswell, J. W. (2008). Educational research : planning, conducting, and evaluating quantitative and qualitative research. Pearson Education International.
• Denzin, N. K., & Lincoln, Y. S. (2011). The SAGE Handbook of Qualitative Research (pp. 1–32). Thousand Oaks: Sage Publications.
• Humphrey, B. (n.d.). The difference between qualitative and quantitative research – Method in Madness by Dovetail. Retrieved August 4, 2018, from https://dovetailapp.com/blog/qualitative-quantitative-research/
• Madrigal, D., & McClain, B. (2012, September 3). Strengths and Weaknesses of Quantitative and Qualitative Research :: UXmatters. https://www.uxmatters.com/mt/archives/2012/09/strengths-and-weaknesses-of-quantitative-and-qualitative-research.php
• Miller, B. (2018a, January 27). 15 Advantages and Disadvantages of Quantitative Research. Retrieved from https://greengarageblog.org/15-advantages-and-disadvantages-of-quantitative-research
• Miller, B. (2018b, March 23). 19 Advantages and Disadvantages of Qualitative Research Methods. Retrieved from https://greengarageblog.org/19-advantages-and-disadvantages-of-qualitative-research-methods
• Paul Nchoji Nkwi, Nyamongo, I. K., & Gery Wayne Ryan. (2001). Field research into socio-cultural issues : methodological guidelines. Yaoundé, Cameroon: International Center For Applied Social Sciences, Research, And Training.
• Physiopedia. (n.d.). Qualitative Research (S. Buxton, Ed.). Retrieved August 7, 2018, from https://www.physio-pedia.com/Qualitative_Research#cite_note-Nkwi-5
• Punch, K. (1998). Introduction to social research : quantitative and qualitative approaches. London ; Thousand Oaks, Calif.: Sage Publications.
• Raghunath, M. (2016, March 3). 7 Differences Between Qualitative and Quantitative Research. Durofy - Business, Technology, Entertainment and Lifestyle Magazine. https://durofy.com/7-differences-between-qualitative-and-quantitative-research
• Streefkerk, R. (2019, April 12). Qualitative vs. quantitative research. Retrieved from https://www.scribbr.com/methodology/qualitative-quantitative-research/#:~:text=and%20qualitative%20methods%3F-
• Surbhi, S. (2018, November 19). Difference Between Qualitative and Quantitative Research (With Comparison Chart) - Key Differences. https://keydifferences.com/difference-between-qualitative-and-quantitative-research.html
• The Interaction Design Foundation. (n.d.). What is Quantitative Research? Retrieved August 8, 2018, from https://www.interaction-design.org/literature/topics/quantitative-research
• Voxco. (2018, September 9). Qualitative research methodology: Definition, Types and examples - Voxco. Retrieved from https://www.voxco.com/blog/qualitative-research-methodology-definition-types-and-examples/
• Wagle, K. (2018, May 25). Quantitative Vs Qualitative Research: 20+ Differences. http://www.publichealthnotes.com/differences-qualitative-quantitative-research/
• https://www.orau.gov/cdcynergy/soc2web/content/phase05/phase05_step03_deeper_qualitative_and_quantitative.htm

This paper is in the following e-collection/theme issue:

Published on 18.3.2024 in Vol 5 (2024)

The Role of Animal-Assisted Therapy in Enhancing Patients’ Well-Being: Systematic Study of the Qualitative and Quantitative Evidence

Authors of this article:

• Ramendra Pati Pandey 1, * , PhD ; 
• Himanshu ­ 2, 3, * , MSc ; 
• Gunjan ­ 2, 3 , MSc ; 
• Riya Mukherjee 2, 3 , MSc ; 
• Chung-Ming Chang 3, 4, * , PhD

1 School of Health Sciences & Technology, University of Petroleum and Energy Studies, , Dehradun, Uttarakhand, , India

2 Graduate Institute of Biomedical Sciences, Chang Gung University, , Taoyuan, , Taiwan

3 Master & PhD Program in Biotechnology Industry, Chang Gung University, , Taoyuan, , Taiwan

4 Department of Medical Biotechnology and Laboratory Science, Chang Gung University, , Taoyuan, , Taiwan

*these authors contributed equally

Corresponding Author:

Chung-Ming Chang, PhD

Background: Animal-assisted therapy, also known as pet therapy, is a therapeutic intervention that involves animals to enhance the well-being of individuals across various populations and settings.

Objective: This systematic study aims to assess the outcomes of animal-assisted therapy interventions and explore the associated policies.

Methods: A total of 16 papers published between 2015 and 2023 were selected for analysis. These papers were chosen based on their relevance to the research topic of animal-assisted therapy and their availability in scholarly databases. Thematic synthesis and meta-analysis were used to synthesize the qualitative and quantitative data extracted from the selected papers.

Results: The analysis included 16 studies that met the inclusion criteria and were deemed to be of moderate or higher quality. Among these studies, 4 demonstrated positive results for therapeutic mediation and one for supportive mediation in psychiatric disorders. Additionally, all studies showed positive outcomes for depression and neurological disorders. Regarding stress and anxiety, 3 studies indicated supportive mediation, while 2 studies showed activating mediation.

Conclusions: The overall assessment of animal-assisted therapy shows promise as an effective intervention in promoting well-being among diverse populations. Further research and the establishment of standardized outcome assessment measures and comprehensive policies are essential for advancing the field and maximizing the benefits of animal-assisted therapy.

Introduction

The inclusion of animals in psychological treatment is not a recent or uncommon practice. Throughout history, there has been an understanding of the positive impact animals can have on human well-being [ 1 ]. This connection between humans and animals is deeply ingrained in our collective subconscious, influencing our emotional experiences [ 2 ]. The earliest documented instance dates back to the late 18th century when animals were introduced into mental health institutions to enhance social interaction among patients [ 3 , 4 ]. Today, numerous programs worldwide incorporate animals to varying degrees in their services. These programs are particularly beneficial for individuals who have experienced trauma, including those diagnosed with posttraumatic stress disorder (PTSD), schizophrenia, Alzheimer disease, autism, etc [ 4 , 5 ].

In the past 50 years, the field of human-animal interaction and, specifically, animal-assisted therapy (AAT) has made significant advancements and progress. AAT is a therapeutic approach that uses animals to improve overall health and well-being. It encompasses emotional, psychological, and physical interactions between individuals, animals, and the environment [ 6 ]. AAT interventions involve qualified treatment providers facilitating interactions between patients and animals with specific therapeutic goals in mind. These interventions often involve collaborative activities between human-animal teams, aiming to promote therapeutic and supportive outcomes [ 7 ]. AAT interventions contribute to individuals’ well-being, supporting physical health and improving cognitive, emotional-affective, and social aspects, leading to enhanced emotional well-being, reduced anxiety, and decreased stress levels [ 8 - 10 ].

Research on therapies involving human-animal interaction has focused on specific animals such as dogs, cats, or horses and specific populations such as those with autism [ 11 ]. Dogs, in particular, are commonly preferred for therapy due to their exceptional bond with humans in modern times. Over thousands of years of shared evolutionary history [ 1 ], dogs have acquired adept socialization skills with humans through processes of domestication and natural selection. They have become our loyal companions, developing unique social skills for interacting with humans [ 12 ]. For instance, studies indicate that dogs possess a sensitivity to our emotional states [ 13 ] and can interpret our social cues [ 14 ], even engaging in sophisticated communication through behaviors like gaze alternation [ 15 ]. Furthermore, dogs are capable of forming intricate attachment relationships with humans, resembling the bonds found in relationships between infants and caregivers [ 16 ]. Research suggests that among the various animals involved in AAT, dogs tend to exhibit superior interactions with people compared to other species, benefiting both children and adults [ 6 ].

This systematic review and meta-analysis sheds light on the potential of animal-assisted interventions to enhance overall well-being and health. Our research aims to contribute to the growing body of evidence supporting the use of animals in therapeutic contexts and to explore the specific contexts in which these interventions are most effective. One of the unique aspects of our study is the incorporation of both quantitative and qualitative analyses to provide a comprehensive understanding of the effects of AAT. While previous research has predominantly relied on quantitative data, we believe that qualitative insights from participants who have experienced these interventions offer valuable perspectives. The special bonds formed between humans and animals are recognized as essential catalysts for transformation and are held in high regard, similar to the therapist-client relationship.

Search Strategy

The meta-analysis was carried out following the methodologies outlined in the esteemed Cochrane Handbook for Systematic Reviews of Interventions [ 17 ], and the findings were reported in compliance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [ 18 ]. To ensure comprehensive coverage, electronic databases were meticulously searched up until June 2023. A total of five English-language electronic databases, including PubMed, Web of Science, Clinical Trials, Science Direct, and Google Scholar, were meticulously explored. This thorough exploration entailed using a combination of pertinent controlled vocabulary terms (eg, Medical Subject Headings [MeSH]) and relevant free-text terms. The search strategy used can be summarized as follows: (animal assisted therapy OR animal assisted intervention OR animal assisted activity OR animal activity interaction OR animal assisted method OR animal facilitated therapy OR pet therapy OR canine assisted therapy OR dog assisted therapy) AND (quasi-experimental study OR randomized controlled trial) AND (pain OR anxiety OR depression OR blood pressure OR BP OR heart rate OR HR) AND (work-related stress OR workplace health OR employee well-being OR burnout) AND (tumor OR malignant OR carcinoma OR oncology OR hospitalization OR hospitalized patients OR inpatients). By using this extensive and refined approach, the meta-analysis aimed to capture a comprehensive body of evidence on the effects of AAI on various health outcomes.

Inclusion and Exclusion Criteria

The inclusion criteria were set based on the PICOS (Patient, Problem, or Population; Intervention; Comparison; Outcomes; and Study Design) framework: studies evaluating the effects of AAT, animal-assisted intervention, or animal-assisted activity; studies evaluating the effects of animal interactions on health and well-being (including depression, agitation, loneliness, stress, and quality of life), social interaction, engagement, physical function, behavioral symptoms, medication use, and adverse events; articles published in English; studies available in full-text format; and studies using quasi-experimental designs or randomized controlled trials. To maintain the rigor and relevance of the study, publications that lacked sufficient information regarding the therapy or did not involve an animal intervention were excluded from consideration.

Data Synthesis

In our study, we used thematic synthesis as a method to assess the eligibility and quality of the articles [ 19 ]. Each article was independently reviewed to determine its suitability for inclusion. We followed a traditional methodology for evaluating the papers that involved examining factors such as the presence of adequate control groups, control of confounders, randomization, well-described experimental design, and relevant outcome variables. Articles that met these criteria were selected and organized into a single sheet using Microsoft Office Excel (2019; Microsoft Corporation). For the included studies, we extracted and compiled various data points into a structured table. This information encompassed the author’s name, country of publication, year of publication, patient characteristics (including sample size, age, gender, and target group), type of study, study design, description of AAT, type of intervention, control group details, study duration, outcomes measured, and the authors’ conclusions. To effectively manage the papers, we used Mendeley software (version 1.19.8; Elsevier).

Classification

To determine the specific contexts in which AATs are effective, we classified the interventions into three categories. First, “supportive mediation” involves AATs providing emotional and psychological support to individuals. Second, “therapeutic mediation” entails AATs addressing specific therapeutic goals and needs in a structured manner. Finally, “activating mediation” comprises AATs designed to stimulate engagement and participation in various activities or tasks.

Risk of Bias

Assessing publication bias is a crucial component in safeguarding the strength and credibility of our meta-analysis, which examines the effects of AAT on improving the well-being of individuals in diverse populations and settings. To gauge the possible influence of publication bias on our results, we applied several established techniques recommended in the field. One of these methods involved visually examining a bias risk graph for signs of asymmetry, which can be an indication of publication bias. By using these comprehensive approaches, our objective was to address any potential bias and guarantee that our meta-analysis offers an impartial synthesis of the existing evidence regarding the beneficial effects of AAT.

Study Selection

The outcome of the search is depicted in Figure 1 . The search process resulted in 968 unique articles after initial searches from various electronic databases like PubMed, Web of Science, Science Direct, and Google Scholar, which yielded 942 articles. An additional 26 articles were extracted from other sources. After eliminating duplicate articles, the total number of articles was reduced to 507. Subsequently, the articles were assessed based on their title and abstract to determine eligibility. Among the initial pool, 389 articles were excluded as they did not meet the eligibility criteria, mainly due to the lack of relevance to AAT. After reading the full text of the remaining articles, 102 more articles were excluded. Of these 102 articles, 60 did not meet the inclusion criteria, and 42 were excluded due to being classified as nonoccupational mixed groups or having unrepresentative results. Finally, a total of 16 studies that met the inclusion criteria were included in the final analysis. The findings and details of these 16 studies are summarized in Tables 1 and 2 .

a AAT: animal-assisted therapy.

b PTSD: posttraumatic stress disorder.

c STAI: State-Trait Anxiety Inventory.

d MMSE: Mini-Mental State Examination.

e GDS: Geriatric Depression Scale.

c Not available.

Risk of Bias Determination

Figure 2 [ 5 , 20 - 34 ] provides a comprehensive evaluation of the overall risk and individual biases in each of the studies included. All the researchers carried out assessments to determine the likelihood of bias, and the results of these assessments showed a remarkable level of consistency across all the investigations. The data depicted in Figure 2 indicate that the articles authored by Hinic et al [ 28 ], Priyanka MB [ 30 ], and Menna et al [ 34 ] exhibited a high risk of bias. Additionally, the study conducted by Anderson and Brown [ 24 ] indicated a potential risk of bias. This consistent and rigorous approach enhances the confidence in the research paper’s results, underscoring the reliability of the reported biases and their impact on the study’s outcomes.

General Characteristics

To provide a concise summary of the selected studies, we have compiled an overview in Table 2 . It presents key information from each study, allowing for a quick and comprehensive understanding of the research landscape.

Quantitative Study

The studies analyzed in this research were conducted between 2015 and 2023, resulting in a total of 16 [ 5 , 20 - 34 ] included papers. These studies were carried out in various countries, including the United States (n=8), Taiwan (n=3), India (n=2), and Italy (n=3). Among the included studies, approximately 12 of 16 (75%) used a randomized controlled trial design. Two studies used conditional controlled designs, while 1 study followed a time series and daily announcement approach. The number of patients enrolled in the studies varied, ranging from 6 to 152 individuals. Specifically, 5 studies involved 0-40 patients, 2 studies included 41-80 patients, 8 studies comprised 81-120 patients, and 1 study encompassed 121-160 patients. The selected studies covered a diverse range of populations, with 7 studies focusing on children and adolescents from various disciplines including child psychology, psychiatry, and pediatrics. The same number of studies involved adult patients covering a range of fields such as general medicine, mental health, and geriatrics, and 2 studies specifically targeted older people, contributing to the fields of gerontology and geriatric medicine. In terms of gender distributions, women were more prominently represented, with ≥50% female participants in 13 of the 16 studies ( Table 2 ).

The interventions included in the studies were described using various terms such as pet encounter therapy, pet-facilitated therapy, pet-assisted living, animal-assisted intervention, AAT, animal-assisted activity, or simply dog visits/therapy. All of the studies incorporated dogs as the primary intervention. In terms of the duration of the interventions, the 7 studies had varying time periods per visit [ 5 , 24 , 25 , 27 , 28 , 31 , 32 ]. Five studies had interventions lasting for 12 weeks [ 20 , 21 , 23 , 30 , 33 ] and one for 6 weeks [ 29 ], while 2 studies had longer intervention periods of 6 months [ 26 , 34 ]. One study did not explicitly mention the duration of the intervention [ 22 ]. The majority of the studies used a one-to-one approach in delivering the intervention, emphasizing individual interactions between participants and the dogs. However, 1 study was conducted in a group setting [ 5 ]. Most studies actively encouraged touching and interaction with the animals, while in 2 studies, the interaction was described as unstructured [ 24 , 32 ].

Qualitative Study

The description of selected studies for qualitative analysis is presented in Tables 1 and 2 . Studies reported on patients’ experiences with animal-assisted interventions such as dog therapy or animal visits. Thematic analysis was used to identify recurring themes and extract meaningful insights from the type of disorder. Participants described the animals as a source of comfort, providing emotional support and reducing stress and anxiety. The interactions with the animals were reported to have a soothing effect and helped individuals cope with their challenges and emotional difficulties.

The qualitative analysis shed light on the subjective experiences and perceptions of individuals participating in the interventions. It provided valuable insights into the emotional, social, and therapeutic benefits associated with animal-assisted interventions, highlighting the potential of these interventions to enhance well-being and quality of life. The selected studies on psychiatric disorders predominantly focused on schizophrenia, with 5 studies specifically addressing this condition in adults. Additionally, 1 study explored acute mental disorders in children [ 33 ]. Six studies were dedicated to investigating stress and anxiety, targeting various populations such as children undergoing physical examinations, children with PTSD, patients with cancer, and nursing students. Three studies examined neurological disorders, including 1 study involving children with autism [ 30 ] and 2 studies involving older individuals with Alzheimer disease [ 26 , 34 ]. In 1 study, the intervention aimed to reduce depression among patients undergoing oncology surgery [ 25 ].

The number of studies with at least one statistically significant positive outcome measure, divided by patient condition and intervention category, is presented in Table 3 . The study aimed to comprehensively evaluate mental health, social functioning, and overall quality of life, taking into account various parameters specific to each measurement scale, for example, generic health-related quality of life measures like Posttraumatic Stress Disorder Reaction Index for DSM-5, State-Trait Anxiety Inventory (STAI) for Children, Patient Health Questionnaire–4 (PHQ-4), Mini-Mental State Examination (MMSE), and 15-item Geriatric Depression Scale (GDS), and general functional measures such as Mental Health–Social Functioning Scale, Social Adaptive Function Scale, chair stand test, Timed Up and Go, Assessment of Communication and Interaction Skills, etc.

Psychiatric Disorder

All 6 trials that focused on psychiatric disorders were categorized as AAT and involved interventions with dog therapy. Among these studies, 5 were conducted using a randomized controlled design, while 1 study used a time series design with randomized daily announcements within a pre-post experimental framework. One study specifically examined patients in child and adolescent psychiatry [ 33 ], while the remaining 5 studies focused on adult psychiatry patients [ 5 , 20 , 21 , 23 , 27 ]. The duration of the AAT programs varied, with some studies consisting of 12-week programs in different settings, while 2 studies provided weekly therapy sessions without specifying the intervention period [ 5 , 27 ].

Each of the 6 conducted studies involved a comparison between an intervention group receiving a specific therapy and a control group that did not participate in any related activities. Notably, the 5 studies specifically targeted middle-aged and older patients diagnosed with chronic schizophrenia. The results of these studies consistently demonstrated significant improvements in various areas, including reductions in psychiatric symptoms, enhanced social functioning, improved quality of life, enhanced cognitive function, increased agility and mobility, and decreased stress levels. These outcomes were measured using a variety of scales and assessment tools [ 5 , 20 , 21 , 23 , 33 ]. In a study conducted by Brown et al [ 27 ], the focus was on examining the impact of mood states and feelings among patients and staff in inpatient psychiatric units. The researchers observed significant changes in mood before and after sessions involving therapy dogs. Specifically, negative moods decreased, while positive moods, such as feelings of happiness, relaxation, and calmness, increased. These changes were measured using the visual analog mood scale [ 27 ]. Overall, these findings highlight the efficacy of AAT in positively impacting the well-being and overall functioning of individuals with psychiatric disorders.

Neurological Disorder

Among the studies that focused on neurological disorders, 3 used dog therapy as an intervention. One of these studies used a randomized controlled design [ 26 ], while the other 2 studies used purposive sampling based on the patients’ conditions. One study specifically targeted children and adolescents with autism, while the other 2 studies focused on older patients with Alzheimer disease. The duration of the AAT programs varied, ranging from 3 to 6 months.

In each of the 3 conducted studies, the intervention group was compared to a control group that did not participate in any activities to assess the outcomes of the therapy. Priyanka MB’s [ 30 ] study focused on children with autism and observed that engaging with a therapy dog, such as brushing the dog and attempting to draw and write for the dog, led to enhanced social and motor skills. Additionally, the children experienced a sense of relaxation and calmness in the presence of the dog. The studies conducted by Menna et al [ 34 ] and Santaniello et al [ 26 ], focusing on older patients with Alzheimer disease over 6 months, have shown promising results. Menna et al’s [ 34 ] study demonstrated the applicability and effectiveness of AAT interventions in stimulating cognition and improving mood. The interventions involved repeated multimodal stimulation, including verbal, visual, and tactile approaches. Similarly, Santaniello et al’s [ 26 ] study also revealed improvements in both cognitive function and mood in the AAT group, as measured by changes in the MMSE and GDS. Overall, these studies indicate that nonpharmacological therapies, particularly AAT, have the potential to reduce symptoms associated with neurological disorders.

Stress and Anxiety

The 6 trials that specifically addressed stress and anxiety used AAT interventions involving dog therapy. These studies exclusively targeted children and adolescents, using a randomized controlled design. In 5 of the studies, the therapy sessions lasted between 10-45 minutes, while 1 study did not specify the duration of the intervention period.

The study conducted by Allen et al [ 22 ] focused on youths who had experienced abuse and were diagnosed with PTSD. The results revealed that the group receiving the intervention showed greater improvements in caregiver-reported symptoms of PTSD, internalizing concerns, and externalizing problems compared to the control group [ 22 ]. In a study by Anderson and Brown [ 24 ] involving nursing students, the intervention group experienced interactions with dogs before testing. This interaction served as a stress reliever for the students, resulting in a decrease in anxiety as measured by the STAI. Thakkar et al [ 25 ] conducted a study on children who were undergoing dental assessments. The findings indicated that the intervention group showed a significantly greater anxiety reduction compared to the control group, as measured by the modified faces version of the Modified Child Dental Anxiety Scale. In the studies conducted by Hinic et al [ 28 ] and Branson et al [ 32 ], dog therapy was provided to children who were hospitalized, and their anxiety levels were assessed before and after the intervention. The results from the STAI for Children suggested that brief pet therapy visits served as a tool to decrease anxiety in children who were hospitalized and promote family satisfaction. McCullough et al [ 31 ] conducted a study where the intervention group participated in dog therapy, while the control group received standard care at the hospital. The findings demonstrated the applicability and effectiveness of AAT interventions in reducing stress and anxiety levels in patients with cancer.

Overall, when considering the results of all these studies, it becomes evident that each one exhibited at least one statistically significant positive effect. When these findings are examined collectively, they provide compelling evidence to suggest that particular modalities of AAT hold substantial promise in terms of reducing stress levels and fostering a positive impact on individuals’ overall mood and well-being.

Ginex et al [ 29 ] conducted a study to explore the impact of a dog-assisted intervention on an inpatient surgical oncology unit. The study used a randomized controlled design, with patients in the intervention group receiving therapy 4 days per week throughout the study period. In contrast, the control group underwent physical therapy without any modifications to their normal routine. Patients in the intervention group reported a significant decrease in depression and anxiety levels, as measured by the PHQ-4, compared to the control group. The findings of the study suggest that AAT fosters a healing environment for patients, incorporating a holistic and humanistic approach that elicits overwhelmingly positive responses.

The outcomes of this meta-analysis provide the long-standing belief that animals can play a beneficial role in the healing process. The study revealed positive and moderately strong results across various aspects, including medical well-being, behavioral outcomes, and the reduction of autism spectrum symptoms. Moreover, the effect on all four outcomes, which include psychiatric disorders, neurological disorders, stress and anxiety, and depression, were consistent and uniform. Additionally, support for AAT was evident from 4 studies comparing it with established interventions, demonstrating that AAT was equally or more effective. These compelling findings indicate that AAT is a robust intervention deserving of further exploration and use. This systematic review and meta-analysis specifically focused on dogs as the assisting animals in a health care setting. However, there were no limitations on the characteristics of the population included in the study. Although this research synthesis provides evidence in favor of the effectiveness of AAT, it is essential to acknowledge the complexities associated with interventions in general and the specific nuances related to the use of AAT.

The majority of articles included in this systematic review were based on randomized controlled trials conducted in various countries. Additionally, time series and daily announcements, divided according to different conditions, were also considered. The increased number of studies provided greater support in assessing the variance of heterogeneity and potential group differences. Although the results are speculative, the meta-analysis demonstrated homogeneity in the summary values, with only one exploratory group difference reaching statistical significance. Nonetheless, this analysis brought forth several intriguing questions and patterns, serving as a foundation for discussions or further research on the factors influencing the effectiveness of AAT. For instance, consistent benefits were observed in children, young age groups, and old age groups across all outcome variables, including symptoms associated with psychiatric disorders, stress, and anxiety [ 35 ]. In particular, among the adult group, a high prevalence of psychiatric disorders, followed by neurological disorders, stress, and anxiety, was found. In contrast, in children, a high number of cases related to stress and anxiety disorders were identified.

Several organizations in different countries are actively working to promote AAT. At the global level, the International Association of Human-Animal Interaction is the worldwide consortium of organizations involved in the practice, research, or education of AAT and the training of service animals [ 36 ]. In the United States, the Society for Healthcare Epidemiology of America has established comprehensive guidelines for animals in health care facilities, which emphasize the importance of written policies, designated AAI visit liaisons, and formal training programs for animals and handlers [ 37 ]. However, despite these guidelines, there is no legal requirement for health care facilities to adopt these measures. One notable organization in the United States, Pet Partners, stands out as the only national therapy animal organization that mandates volunteer training and biennial evaluations of animal-handler teams, and prohibits raw meat diets [ 38 ]. In Europe, the European Society for Animal Assisted Therapy (ESAAT) plays a substantial role as an influential organization operating across various disciplines and professions within the field. ESAAT’s primary mission is to accredit education and training programs in the domain of animal-assisted interventions [ 39 ]. While the Western world has made significant advancements in AAT, Eastern countries such as India, China, Taiwan, Japan, and Sri Lanka are still in the early stages of exploring and implementing such practices. These countries are currently in the infancy phase of using and developing their own AAT programs. As awareness and understanding of the benefits of AAT continue to grow worldwide, it is expected that these Eastern countries will gradually catch up and further enhance their ATT initiatives [ 40 ].

Our review was based on a limited number of studies, which can be attributed to our strict inclusion criteria and the presence of suboptimal study designs. Specifically, many of the randomized trials were characterized by small sample sizes, short durations, and a lack of follow-up assessments. Another limitation pertains to the suitability of the outcome measures used, which may not fully capture the important values and impacts as perceived by the participants. On the other hand, the qualitative research included in the review exhibited higher overall quality and contributed valuable insights to our findings.

In conclusion, the reviewed studies provide preliminary evidence of the potential benefits of AAT in certain conditions. It suggests that dog-assisted therapy can have minor to moderate effects in treating psychiatric disorders, cognitive disorders, neurological disorders, etc, and demonstrates potential in various medical interventions. However, it is important to note that some of the outcome measures analyzed did not show significant effects, and further research is needed to better understand the specific contexts and conditions. To foster the growth of such therapy, we need education campaigns, research programs, professional support, and media awareness to increase the effectiveness of AAT across different countries.

Acknowledgments

This research was funded by VtR Inc-CGU (SCRPD1L0221), DOXABIO-CGU (SCRPD1K0131), and a CGU grant (UZRPD1L0011, UZRPD1M0081).

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist

• Thalmann O, Shapiro B, Cui P, et al. Mitochondrial Genomes of ancient Canids suggest a European origin of domestic dogs. Science. Nov 15, 2013;342(6160):871-874. [ CrossRef ] [ Medline ]
• Gee NR, Rodriguez KE, Fine AH, Trammell JP. Dogs supporting human health and well-being: a biopsychosocial approach. Front Vet Sci. Mar 2021;8:630465. [ CrossRef ] [ Medline ]
• Sahebalzamani M, Rezaei O, Moghadam LF. Animal-assisted therapy on happiness and life quality of chronic psychiatric patients living in psychiatric residential care homes: a randomized controlled study. BMC Psychiatry. Dec 1, 2020;20(1):575. [ CrossRef ] [ Medline ]
• O’Haire ME, Guérin NA, Kirkham AC. Animal-assisted intervention for trauma: a systematic literature review. Front Psychol. Aug 7, 2015;6:1121. [ CrossRef ] [ Medline ]
• Nurenberg JR, Schleifer SJ, Shaffer TM, et al. Animal-assisted therapy with chronic psychiatric inpatients: equine-assisted psychotherapy and aggressive behavior. Psychiatr Serv. Jan 1, 2015;66(1):80-86. [ CrossRef ] [ Medline ]
• Şahin S, Kose B, Zarif M. Animal-assisted therapy in occupational therapy. Occup Ther Therapeutic Creative Use Activity. Nov 5, 2018. [ CrossRef ]
• The IAHAIO Defintions for animal assisted intervention and guidelines for wellness of animals involved in AAI. International Association of Human-Animal Interaction Organizations. 2014. URL: https://iahaio.org/wp/wp-content/uploads/2018/04/iahaio_wp_updated-2018-final.pdf [Accessed 2024-03-08]
• Menna LF, Santaniello A, Amato A, et al. Changes of oxytocin and serotonin values in dialysis patients after animal assisted activities (AAAs) with a dog—a preliminary study. Animals. Aug 3, 2019;9:526. [ CrossRef ]
• Menna LF, Santaniello A, Gerardi F, et al. Efficacy of animal-assisted therapy adapted to reality orientation therapy: measurement of salivary cortisol. Psychogeriatrics. Sep 2019;19(5):510-512. [ CrossRef ] [ Medline ]
• Menna LF, Santaniello A, Todisco M, et al. The human-animal relationship as the focus of animal-assisted interventions: a One Health approach. Int J Environ Res Public Health. Sep 29, 2019;16(19):3660. [ CrossRef ] [ Medline ]
• Hoagwood KE, Acri M, Morrissey M, Peth-Pierce R. Animal-assisted therapies for youth with or at risk for mental health problems: a systematic review. Appl Dev Sci. Jan 2017;21(1):1-13. [ CrossRef ] [ Medline ]
• Muldoon AL, Kuhns LM, Supple J, Jacobson KC, Garofalo R. A web-based study of dog ownership and depression among people living with HIV. JMIR Ment Health. Nov 8, 2017;4(4):e53. [ CrossRef ] [ Medline ]
• Albuquerque N, Guo K, Wilkinson A, Savalli C, Otta E, Mills D. Dogs recognize dog and human emotions. Biol Lett. Jan 2016;12(1):20150883. [ CrossRef ] [ Medline ]
• Hare B, Brown M, Williamson C, Tomasello M. The domestication of social cognition in dogs. Science. Nov 22, 2002;298(5598):1634-1636. [ CrossRef ] [ Medline ]
• Miklósi A, Kubinyi E, Topál J, Gácsi M, Virányi Z, Csányi V. A simple reason for a big difference: wolves do not look back at humans, but dogs do. Curr Biol. Apr 29, 2003;13(9):763-766. [ CrossRef ] [ Medline ]
• Payne E, Bennett PC, McGreevy PD. Current perspectives on attachment and bonding in the dog–human dyad. Psychol Res Behav Manag. Feb 24, 2015;8:71-79. [ CrossRef ] [ Medline ]
• Cochrane Handbook for Systematic Reviews of Interventions. Cochrane Training. URL: https://training.cochrane.org/handbook [Accessed 2023-06-24]
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [ CrossRef ] [ Medline ]
• Thompson Coon J, Gwernan-Jones R, Garside R, et al. Developing methods for the overarching synthesis of quantitative and qualitative evidence: the interweave synthesis approach. Res Synth Methods. Jul 2020;11(4):507-521. [ CrossRef ] [ Medline ]
• Shih CA, Yang MH. Effect of animal-assisted therapy (AAT) on social interaction and quality of life in patients with schizophrenia during the COVID-19 pandemic: an experimental study. Asian Nurs Res (Korean Soc Nurs Sci). Feb 2023;17(1):37-43. [ CrossRef ] [ Medline ]
• Chen CR, Hung CF, Lee YW, Tseng WT, Chen ML, Chen TT. Functional outcomes in a randomized controlled trial of animal-assisted therapy on middle-aged and older adults with schizophrenia. Int J Environ Res Public Health. May 21, 2022;19(10):6270. [ CrossRef ] [ Medline ]
• Allen B, Shenk CE, Dreschel NE, et al. Integrating animal-assisted therapy into TF-CBT for abused youth with PTSD: a randomized controlled feasibility trial. Child Maltreat. Aug 2022;27(3):466-477. [ CrossRef ] [ Medline ]
• Chen TT, Hsieh TL, Chen ML, Tseng WT, Hung CF, Chen CR. Animal-assisted therapy in middle-aged and older patients with schizophrenia: a randomized controlled trial. Front Psychiatry. Aug 3, 2021;12:713623. [ CrossRef ] [ Medline ]
• Anderson D, Brown S. The effect of animal-assisted therapy on nursing student anxiety: a randomized control study. Nurse Educ Pract. Mar 2021;52:103042. [ CrossRef ] [ Medline ]
• Thakkar TK, Naik SN, Dixit UB. Assessment of dental anxiety in children between 5 and 10 years of age in the presence of a therapy dog: a randomized controlled clinical study. Eur Arch Paediatr Dent. Jun 2021;22(3):459-467. [ CrossRef ] [ Medline ]
• Santaniello A, Garzillo S, Amato A, et al. Animal-assisted therapy as a non-pharmacological approach in Alzheimer’s disease: a retrospective study. Animals (Basel). Jul 6, 2020;10(7):1-10. [ CrossRef ] [ Medline ]
• Brown S, Snelders J, Godbold J, et al. Effects of animal-assisted activity on mood states and feelings in a psychiatric setting. J Am Psychiatr Nurses Assoc. Nov/Dec 2020;26(6):555-567. [ CrossRef ] [ Medline ]
• Hinic K, Kowalski MO, Holtzman K, Mobus K. The effect of a pet therapy and comparison intervention on anxiety in hospitalized children. J Pediatr Nurs. May-Jun 2019;46:55-61. [ CrossRef ] [ Medline ]
• Ginex P, Montefusco M, Zecco G, et al. Animal-facilitated therapy program: outcomes from caring canines, a program for patients and staff on an inpatient surgical oncology unit. Clin J Oncol Nurs. Apr 1, 2018;22(2):193-198. [ CrossRef ] [ Medline ]
• MB P. Effectiveness of animal assisted therapy on children and adolescents with autism. Int J Res Soc Sci. Feb 2018;8(2):2249-2496.
• McCullough A, Ruehrdanz A, Jenkins MA, et al. Measuring the effects of an animal-assisted intervention for pediatric oncology patients and their parents: a multisite randomized controlled trial. J Pediatr Oncol Nurs. May 2018;35(3):159-177. [ CrossRef ] [ Medline ]
• Branson SM, Boss L, Padhye NS, Trötscher T, Ward A. Effects of animal-assisted activities on biobehavioral stress responses in hospitalized children: a randomized controlled study. J Pediatr Nurs. Sep-Oct 2017;36:84-91. [ CrossRef ]
• Stefanini MC, Martino A, Allori P, Galeotti F, Tani F. The use of animal-assisted therapy in adolescents with acute mental disorders: a randomized controlled study. Complement Ther Clin Pract. Feb 2015;21(1):42-46. [ CrossRef ] [ Medline ]
• Menna LF, Santaniello A, Gerardi F, Di Maggio A, Milan G. Evaluation of the efficacy of animal-assisted therapy based on the reality orientation therapy protocol in Alzheimer’s disease patients: a pilot study. Psychogeriatrics. Jul 2016;16(4):240-246. [ CrossRef ] [ Medline ]
• Chiu CJ, Hsieh S, Li CW. Needs and preferences of middle-aged and older adults in Taiwan for companion robots and pets: survey study. J Med Internet Res. Jun 11, 2021;23(6):e23471. [ CrossRef ] [ Medline ]
• Who we are. International Association of Human-Animal Interaction Organizations. URL: https://iahaio.org/missions-goals/ [Accessed 2023-06-30]
• Murthy R, Bearman G, Brown S, et al. Animals in healthcare facilities: recommendations to minimize potential risks. Infect Control Hosp Epidemiol. May 2015;36(5):495-516. [ CrossRef ] [ Medline ]
• Pet Partners. URL: https://petpartners.org/ [Accessed 2023-06-28]
• ESAAT. About us. URL: https://www.esaat.org/en/about-us/ [Accessed 2023-06-29]
• Narvekar HN. A reflection on the current status of animal-assisted therapy in India. Hum Arenas. Sep 17, 2021;6(2023):760-775. [ CrossRef ]

Abbreviations

Edited by Edward Meinert, Gunther Eysenbach; submitted 11.08.23; peer-reviewed by Anonymous, Anonymous; final revised version received 16.12.23; accepted 27.12.23; published 18.03.24.

© Ramendra Pati Pandey, Himanshu, Gunjan, Riya Mukherjee, Chung-Ming Chang. Originally published in JMIRx Med (https://med.jmirx.org), 18.3.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIRx Med, is properly cited. The complete bibliographic information, a link to the original publication on https://med.jmirx.org/ , as well as this copyright and license information must be included.

COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK

Senior Associate Director of Survey Research and Data Analytics

• Columbia College
• Morningside
• Opening on: Mar 18 2024
• Job Type: Officer of Administration
• Regular/Temporary: Regular
• Hours Per Week: 35
• Salary Range: $100,000 - $110,000

Position Summary

The Senior Associate Director, Survey Research and Data Analytics designs, develops, executes, and maintains various data-driven processes and applications for reporting and analysis related to academic and curricular reporting for Columbia College.

The Senior Associate Director, working closely with the Senior Director of Data Analytics, leads the College’s internal survey and analysis service. The incumbent plays a critical role in helping the College understand its students and other key stakeholders. It is responsible for designing, executing, and analyzing surveys based on research objectives and audience characteristics, and generating executive-level qualitative and quantitative analyses to inform the College’s data-driven decisions in support of the needs of College’s strategic and business goals.

They further serve as a key resource and Subject Matter Expert to the College in the development, administration, and analysis of surveys, regularly consulting with College members who are engaged in survey research. In all work, they combine strong technical skills in survey research methods, data analysis, reporting, and visualization with knowledge of the university context to gather and relay critical information and insights from survey data clearly and accurately.

The Senior Associate Director collaborates with the Senior Director to manage the question selection, distribution, and analysis of the COFHE annual surveys. This position then packages the resulting analysis for presentation to and consideration by the College’s executive team for use in decision-making and possible follow-up internal surveys.

This position works closely with the Office of Planning and Institutional Research, the Enrollment Group, and other College departments to manage the College’s submissions to surveys such College Board, Peterson’s, and Princeton Review. The incumbent will coordinate the Data Group’s activities to gather the data, perform quality assurance, gain approval from external parties, submit and report on the status of these surveys submitted on behalf of the College.

The Senior Associate Director supports data integrity and data security, while working with large data sets and performing various processes utilizing ETL. The incumbent will work with various data sources to perform executive-level analysis, assessment, and reporting. As an integral member of the team, the Senior Associate Director will assist the Senior Director, Data and Analytics in overseeing the Data Group’s project portfolio and task management.

Responsibilities

• Serve as Subject Matter Expert for survey design, execution, and analysis for the College.
• Work with stakeholders to determine the best way to collect survey data; Develop sampling plans to ensure that surveys are representative of the population; Conduct focus groups or in-depth interviews to gather qualitative data as appropriate; choose the most appropriate method for administering surveys, based on the type of data being collected.
• Provide quantitative and qualitative analyses and feedback for research, surveys, and assessments; Identify trends, correlations, and other significant information. Create frameworks for forecasting and modeling projects. Present findings from survey analysis in a clear and concise manner, both verbally and in writing.
• Train and oversee other Data Group staff on how to administer and analyze surveys. Review surveys developed by others to ensure quality and accuracy; Identify issues or problems with survey methodology or implementation that may have affected results.
• Work with the Senior Director, Data and Analytics and the Executive Director of Information Technology and Strategic Analytics to deliver data analysis, data visualization, and business intelligence to the Office of the Dean, Senior Staff of the College, and other critical partners to provide insight and inform decision making. Produce dashboards and reports that visualize data, making them easily translatable and useful for business decisions. Facilitate processes that translate data into actionable insights in order to solve problems, provide insight, optimize operational processes, and improve efficiency. Identify, analyze, and interpret trends or patterns in complex data sets using statistical techniques. Participate in evaluating new technologies so as to ensure the advancement of best practices and business intelligence needs for the College.
• Work with the Office of Planning and Institutional Research, Enrollment Group, and other College departments to oversee data generation, review, and submission of surveys (College Board, Peterson’s, Princeton Review, etc.) and provide oversight for the Common Data Set generation.
• Create, maintain, and document innovative, user-friendly front-end reporting tools and platforms that provide point-and-click ability to produce reports for operational and analysis needs.  Build and maintain databases and automate data processes, including ETL, SQL, stored procedures, SSRS, SSIS, Tableau, etc., for in-house and third-party databases. Create and provide ad-hoc reports and datasets in response to requests from various departments.
• Assist the Senior Director, Data and Analytics with portfolio and task management, overseeing projects, requests, and workflows handled by the Data Group.
• Play an integral role in planning short and long-term projects with the Senior Director of Data and Analytics; the Executive Director of Information Technology and Strategic Analytics; and other team members, and participate in implementing strategic initiatives around research, forecasting, analysis, and reporting.
• With the Senior Director and Executive Director, lead the College’s Survey Governance Group to forecast the needs for information College-wide and coordinate survey execution across College departments.
• Research and evaluate available sources of data to extract, clean, and manipulate datasets for use. Facilitate processes that translate data into actionable insights in order to solve problems, provide insight, and optimize operational processes, and improve efficiency.
• Work on special projects as deemed relevant and be available as needed for technical and other support during critical periods.
• Other related duties as assigned.

Minimum Qualifications

• Bachelor’s degree or equivalent required.
• A minimum of four to six years of experience, or a combination of education and experience.
• The ability to work with a high degree of independence and accountability as a member of a cohesive team.
• Excellent verbal and written communication skills.
• Proficiency in SQL and query and reporting analysis tools including ETL, cubes, and reporting services.
• Familiarity with relational database design.
• Experience with both qualitative and quantitative statistical methods in survey design, research, or assessments.
• Experience with survey design and execution. Proficiency with the Microsoft Office Suite, especially Excel.
• Ability to prioritize and manage multiple tasks under short and changing deadlines, while ensuring timely and accurate execution.
• Proven ability to create project plans and successfully complete projects.
• Excellent analytical, organizational, and interpersonal skills required; ability to analyze and interpret complex data sets, make findings relevant, and explain data and technology to a general (non-technical) audience.
• Act as an analytical thinker with the ability to work collaboratively and independently.
• Must have the ability to exercise sound judgment, discretion, and tact in a highly confidential professional setting.

Preferred Qualifications

Experience with Qualtrics, SQL Server Reporting Services (SSRS), Tableau, or other enterprise-level reporting tools is preferred.

Knowledge of a programming language such as Python or R.

Experience using student information systems (SIS) and/or high-level enterprise resource planning (ERP) systems.

Equal Opportunity Employer / Disability / Veteran

Columbia University is committed to the hiring of qualified local residents.

Commitment to Diversity 

Columbia university is dedicated to increasing diversity in its workforce, its student body, and its educational programs. achieving continued academic excellence and creating a vibrant university community require nothing less. in fulfilling its mission to advance diversity at the university, columbia seeks to hire, retain, and promote exceptionally talented individuals from diverse backgrounds.  , share this job.

Thank you - we'll send an email shortly.

Other Recently Posted Jobs

Administrative Coordinator - Core Administration

Assistant director - audio visual, graduate practice administrator.

Refer someone to this job

• ©2022 Columbia University
• Accessibility
• Administrator Log in

Wait! Before you go, are you interested in a career at Columbia University? Sign up here! 

Thank you, for sharing your information. A member of our team will reach out to you soon!

This website uses cookies as well as similar tools and technologies to understand visitors' experiences. By continuing to use this website, you consent to Columbia University's usage of cookies and similar technologies, in accordance with the Columbia University Website Cookie Notice .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.38(37); 2023 Sep 18
• PMC10506897

Conducting and Writing Quantitative and Qualitative Research

Edward barroga.

1 Department of Medical Education, Showa University School of Medicine, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

Atsuko Furuta

Makiko arima, shizuma tsuchiya, chikako kawahara, yusuke takamiya.

Comprehensive knowledge of quantitative and qualitative research systematizes scholarly research and enhances the quality of research output. Scientific researchers must be familiar with them and skilled to conduct their investigation within the frames of their chosen research type. When conducting quantitative research, scientific researchers should describe an existing theory, generate a hypothesis from the theory, test their hypothesis in novel research, and re-evaluate the theory. Thereafter, they should take a deductive approach in writing the testing of the established theory based on experiments. When conducting qualitative research, scientific researchers raise a question, answer the question by performing a novel study, and propose a new theory to clarify and interpret the obtained results. After which, they should take an inductive approach to writing the formulation of concepts based on collected data. When scientific researchers combine the whole spectrum of inductive and deductive research approaches using both quantitative and qualitative research methodologies, they apply mixed-method research. Familiarity and proficiency with these research aspects facilitate the construction of novel hypotheses, development of theories, or refinement of concepts.

Graphical Abstract

INTRODUCTION

Novel research studies are conceptualized by scientific researchers first by asking excellent research questions and developing hypotheses, then answering these questions by testing their hypotheses in ethical research. 1 , 2 , 3 Before they conduct novel research studies, scientific researchers must possess considerable knowledge of both quantitative and qualitative research. 2

In quantitative research, researchers describe existing theories, generate and test a hypothesis in novel research, and re-evaluate existing theories deductively based on their experimental results. 1 , 4 , 5 In qualitative research, scientific researchers raise and answer research questions by performing a novel study, then propose new theories by clarifying their results inductively. 1 , 6

RATIONALE OF THIS ARTICLE

When researchers have a limited knowledge of both research types and how to conduct them, this can result in substandard investigation. Researchers must be familiar with both types of research and skilled to conduct their investigations within the frames of their chosen type of research. Thus, meticulous care is needed when planning quantitative and qualitative research studies to avoid unethical research and poor outcomes.

Understanding the methodological and writing assumptions 7 , 8 underpinning quantitative and qualitative research, especially by non-Anglophone researchers, is essential for their successful conduct. Scientific researchers, especially in the academe, face pressure to publish in international journals 9 where English is the language of scientific communication. 10 , 11 In particular, non-Anglophone researchers face challenges related to linguistic, stylistic, and discourse differences. 11 , 12 Knowing the assumptions of the different types of research will help clarify research questions and methodologies, easing the challenge and help.

SEARCH FOR RELEVANT ARTICLES

To identify articles relevant to this topic, we adhered to the search strategy recommended by Gasparyan et al. 7 We searched through PubMed, Scopus, Directory of Open Access Journals, and Google Scholar databases using the following keywords: quantitative research, qualitative research, mixed-method research, deductive reasoning, inductive reasoning, study design, descriptive research, correlational research, experimental research, causal-comparative research, quasi-experimental research, historical research, ethnographic research, meta-analysis, narrative research, grounded theory, phenomenology, case study, and field research.

AIMS OF THIS ARTICLE

This article aims to provide a comparative appraisal of qualitative and quantitative research for scientific researchers. At present, there is still a need to define the scope of qualitative research, especially its essential elements. 13 Consensus on the critical appraisal tools to assess the methodological quality of qualitative research remains lacking. 14 Framing and testing research questions can be challenging in qualitative research. 2 In the healthcare system, it is essential that research questions address increasingly complex situations. Therefore, research has to be driven by the kinds of questions asked and the corresponding methodologies to answer these questions. 15 The mixed-method approach also needs to be clarified as this would appear to arise from different philosophical underpinnings. 16

This article also aims to discuss how particular types of research should be conducted and how they should be written in adherence to international standards. In the US, Europe, and other countries, responsible research and innovation was conceptualized and promoted with six key action points: engagement, gender equality, science education, open access, ethics and governance. 17 , 18 International ethics standards in research 19 as well as academic integrity during doctoral trainings are now integral to the research process. 20

POTENTIAL BENEFITS FROM THIS ARTICLE

This article would be beneficial for researchers in further enhancing their understanding of the theoretical, methodological, and writing aspects of qualitative and quantitative research, and their combination.

Moreover, this article reviews the basic features of both research types and overviews the rationale for their conduct. It imparts information on the most common forms of quantitative and qualitative research, and how they are carried out. These aspects would be helpful for selecting the optimal methodology to use for research based on the researcher’s objectives and topic.

This article also provides information on the strengths and weaknesses of quantitative and qualitative research. Such information would help researchers appreciate the roles and applications of both research types and how to gain from each or their combination. As different research questions require different types of research and analyses, this article is anticipated to assist researchers better recognize the questions answered by quantitative and qualitative research.

Finally, this article would help researchers to have a balanced perspective of qualitative and quantitative research without considering one as superior to the other.

TYPES OF RESEARCH

Research can be classified into two general types, quantitative and qualitative. 21 Both types of research entail writing a research question and developing a hypothesis. 22 Quantitative research involves a deductive approach to prove or disprove the hypothesis that was developed, whereas qualitative research involves an inductive approach to create a hypothesis. 23 , 24 , 25 , 26

In quantitative research, the hypothesis is stated before testing. In qualitative research, the hypothesis is developed through inductive reasoning based on the data collected. 27 , 28 For types of data and their analysis, qualitative research usually includes data in the form of words instead of numbers more commonly used in quantitative research. 29

Quantitative research usually includes descriptive, correlational, causal-comparative / quasi-experimental, and experimental research. 21 On the other hand, qualitative research usually encompasses historical, ethnographic, meta-analysis, narrative, grounded theory, phenomenology, case study, and field research. 23 , 25 , 28 , 30 A summary of the features, writing approach, and examples of published articles for each type of qualitative and quantitative research is shown in Table 1 . 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43

QUANTITATIVE RESEARCH

Deductive approach.

The deductive approach is used to prove or disprove the hypothesis in quantitative research. 21 , 25 Using this approach, researchers 1) make observations about an unclear or new phenomenon, 2) investigate the current theory surrounding the phenomenon, and 3) hypothesize an explanation for the observations. Afterwards, researchers will 4) predict outcomes based on the hypotheses, 5) formulate a plan to test the prediction, and 6) collect and process the data (or revise the hypothesis if the original hypothesis was false). Finally, researchers will then 7) verify the results, 8) make the final conclusions, and 9) present and disseminate their findings ( Fig. 1A ).

Types of quantitative research

The common types of quantitative research include (a) descriptive, (b) correlational, c) experimental research, and (d) causal-comparative/quasi-experimental. 21

Descriptive research is conducted and written by describing the status of an identified variable to provide systematic information about a phenomenon. A hypothesis is developed and tested after data collection, analysis, and synthesis. This type of research attempts to factually present comparisons and interpretations of findings based on analyses of the characteristics, progression, or relationships of a certain phenomenon by manipulating the employed variables or controlling the involved conditions. 44 Here, the researcher examines, observes, and describes a situation, sample, or variable as it occurs without investigator interference. 31 , 45 To be meaningful, the systematic collection of information requires careful selection of study units by precise measurement of individual variables 21 often expressed as ranges, means, frequencies, and/or percentages. 31 , 45 Descriptive statistical analysis using ANOVA, Student’s t -test, or the Pearson coefficient method has been used to analyze descriptive research data. 46

Correlational research is performed by determining and interpreting the extent of a relationship between two or more variables using statistical data. This involves recognizing data trends and patterns without necessarily proving their causes. The researcher studies only the data, relationships, and distributions of variables in a natural setting, but does not manipulate them. 21 , 45 Afterwards, the researcher establishes reliability and validity, provides converging evidence, describes relationship, and makes predictions. 47

Experimental research is usually referred to as true experimentation. The researcher establishes the cause-effect relationship among a group of variables making up a study using the scientific method or process. This type of research attempts to identify the causal relationships between variables through experiments by arbitrarily controlling the conditions or manipulating the variables used. 44 The scientific manuscript would include an explanation of how the independent variable was manipulated to determine its effects on the dependent variables. The write-up would also describe the random assignments of subjects to experimental treatments. 21

Causal-comparative/quasi-experimental research closely resembles true experimentation but is conducted by establishing the cause-effect relationships among variables. It may also be conducted to establish the cause or consequences of differences that already exist between, or among groups of individuals. 48 This type of research compares outcomes between the intervention groups in which participants are not randomized to their respective interventions because of ethics- or feasibility-related reasons. 49 As in true experiments, the researcher identifies and measures the effects of the independent variable on the dependent variable. However, unlike true experiments, the researchers do not manipulate the independent variable.

In quasi-experimental research, naturally formed or pre-existing groups that are not randomly assigned are used, particularly when an ethical, randomized controlled trial is not feasible or logical. 50 The researcher identifies control groups as those which have been exposed to the treatment variable, and then compares these with the unexposed groups. The causes are determined and described after data analysis, after which conclusions are made. The known and unknown variables that could still affect the outcome are also included. 7

QUALITATIVE RESEARCH

Inductive approach.

Qualitative research involves an inductive approach to develop a hypothesis. 21 , 25 Using this approach, researchers answer research questions and develop new theories, but they do not test hypotheses or previous theories. The researcher seldom examines the effectiveness of an intervention, but rather explores the perceptions, actions, and feelings of participants using interviews, content analysis, observations, or focus groups. 25 , 45 , 51

Distinctive features of qualitative research

Qualitative research seeks to elucidate about the lives of people, including their lived experiences, behaviors, attitudes, beliefs, personality characteristics, emotions, and feelings. 27 , 30 It also explores societal, organizational, and cultural issues. 30 This type of research provides a good story mimicking an adventure which results in a “thick” description that puts readers in the research setting. 52

The qualitative research questions are open-ended, evolving, and non-directional. 26 The research design is usually flexible and iterative, commonly employing purposive sampling. The sample size depends on theoretical saturation, and data is collected using in-depth interviews, focus groups, and observations. 27

In various instances, excellent qualitative research may offer insights that quantitative research cannot. Moreover, qualitative research approaches can describe the ‘lived experience’ perspectives of patients, practitioners, and the public. 53 Interestingly, recent developments have looked into the use of technology in shaping qualitative research protocol development, data collection, and analysis phases. 54

Qualitative research employs various techniques, including conversational and discourse analysis, biographies, interviews, case-studies, oral history, surveys, documentary and archival research, audiovisual analysis, and participant observations. 26

Conducting qualitative research

To conduct qualitative research, investigators 1) identify a general research question, 2) choose the main methods, sites, and subjects, and 3) determine methods of data documentation access to subjects. Researchers also 4) decide on the various aspects for collecting data (e.g., questions, behaviors to observe, issues to look for in documents, how much (number of questions, interviews, or observations), 5) clarify researchers’ roles, and 6) evaluate the study’s ethical implications in terms of confidentiality and sensitivity. Afterwards, researchers 7) collect data until saturation, 8) interpret data by identifying concepts and theories, and 9) revise the research question if necessary and form hypotheses. In the final stages of the research, investigators 10) collect and verify data to address revisions, 11) complete the conceptual and theoretical framework to finalize their findings, and 12) present and disseminate findings ( Fig. 1B ).

Types of qualitative research

The different types of qualitative research include (a) historical research, (b) ethnographic research, (c) meta-analysis, (d) narrative research, (e) grounded theory, (f) phenomenology, (g) case study, and (h) field research. 23 , 25 , 28 , 30

Historical research is conducted by describing past events, problems, issues, and facts. The researcher gathers data from written or oral descriptions of past events and attempts to recreate the past without interpreting the events and their influence on the present. 6 Data is collected using documents, interviews, and surveys. 55 The researcher analyzes these data by describing the development of events and writes the research based on historical reports. 2

Ethnographic research is performed by observing everyday life details as they naturally unfold. 2 It can also be conducted by developing in-depth analytical descriptions of current systems, processes, and phenomena or by understanding the shared beliefs and practices of a particular group or culture. 21 The researcher collects extensive narrative non-numerical data based on many variables over an extended period, in a natural setting within a specific context. To do this, the researcher uses interviews, observations, and active participation. These data are analyzed by describing and interpreting them and developing themes. A detailed report of the interpreted data is then provided. 2 The researcher immerses himself/herself into the study population and describes the actions, behaviors, and events from the perspective of someone involved in the population. 23 As examples of its application, ethnographic research has helped to understand a cultural model of family and community nursing during the coronavirus disease 2019 outbreak. 56 It has also been used to observe the organization of people’s environment in relation to cardiovascular disease management in order to clarify people’s real expectations during follow-up consultations, possibly contributing to the development of innovative solutions in care practices. 57

Meta-analysis is carried out by accumulating experimental and correlational results across independent studies using a statistical method. 21 The report is written by specifying the topic and meta-analysis type. In the write-up, reporting guidelines are followed, which include description of inclusion criteria and key variables, explanation of the systematic search of databases, and details of data extraction. Meta-analysis offers in-depth data gathering and analysis to achieve deeper inner reflection and phenomenon examination. 58

Narrative research is performed by collecting stories for constructing a narrative about an individual’s experiences and the meanings attributed to them by the individual. 9 It aims to hear the voice of individuals through their account or experiences. 17 The researcher usually conducts interviews and analyzes data by storytelling, content review, and theme development. The report is written as an in-depth narration of events or situations focused on the participants. 2 , 59 Narrative research weaves together sequential events from one or two individuals to create a “thick” description of a cohesive story or narrative. 23 It facilitates understanding of individuals’ lives based on their own actions and interpretations. 60

Grounded theory is conducted by engaging in an inductive ground-up or bottom-up strategy of generating a theory from data. 24 The researcher incorporates deductive reasoning when using constant comparisons. Patterns are detected in observations and then a working hypothesis is created which directs the progression of inquiry. The researcher collects data using interviews and questionnaires. These data are analyzed by coding the data, categorizing themes, and describing implications. The research is written as a theory and theoretical models. 2 In the write-up, the researcher describes the data analysis procedure (i.e., theoretical coding used) for developing hypotheses based on what the participants say. 61 As an example, a qualitative approach has been used to understand the process of skill development of a nurse preceptor in clinical teaching. 62 A researcher can also develop a theory using the grounded theory approach to explain the phenomena of interest by observing a population. 23

Phenomenology is carried out by attempting to understand the subjects’ perspectives. This approach is pertinent in social work research where empathy and perspective are keys to success. 21 Phenomenology studies an individual’s lived experience in the world. 63 The researcher collects data by interviews, observations, and surveys. 16 These data are analyzed by describing experiences, examining meanings, and developing themes. The researcher writes the report by contextualizing and reporting the subjects’ experience. This research approach describes and explains an event or phenomenon from the perspective of those who have experienced it. 23 Phenomenology understands the participants’ experiences as conditioned by their worldviews. 52 It is suitable for a deeper understanding of non-measurable aspects related to the meanings and senses attributed by individuals’ lived experiences. 60

Case study is conducted by collecting data through interviews, observations, document content examination, and physical inspections. The researcher analyzes the data through a detailed identification of themes and the development of narratives. The report is written as an in-depth study of possible lessons learned from the case. 2

Field research is performed using a group of methodologies for undertaking qualitative inquiries. The researcher goes directly to the social phenomenon being studied and observes it extensively. In the write-up, the researcher describes the phenomenon under the natural environment over time with no implantation of controls or experimental conditions. 45

DIFFERENCES BETWEEN QUANTITATIVE AND QUALITATIVE RESEARCH

Scientific researchers must be aware of the differences between quantitative and qualitative research in terms of their working mechanisms to better understand their specific applications. This knowledge will be of significant benefit to researchers, especially during the planning process, to ensure that the appropriate type of research is undertaken to fulfill the research aims.

In terms of quantitative research data evaluation, four well-established criteria are used: internal validity, external validity, reliability, and objectivity. 23 The respective correlating concepts in qualitative research data evaluation are credibility, transferability, dependability, and confirmability. 30 Regarding write-up, quantitative research papers are usually shorter than their qualitative counterparts, which allows the latter to pursue a deeper understanding and thus producing the so-called “thick” description. 29

Interestingly, a major characteristic of qualitative research is that the research process is reversible and the research methods can be modified. This is in contrast to quantitative research in which hypothesis setting and testing take place unidirectionally. This means that in qualitative research, the research topic and question may change during literature analysis, and that the theoretical and analytical methods could be altered during data collection. 44

Quantitative research focuses on natural, quantitative, and objective phenomena, whereas qualitative research focuses on social, qualitative, and subjective phenomena. 26 Quantitative research answers the questions “what?” and “when?,” whereas qualitative research answers the questions “why?,” “how?,” and “how come?.” 64

Perhaps the most important distinction between quantitative and qualitative research lies in the nature of the data being investigated and analyzed. Quantitative research focuses on statistical, numerical, and quantitative aspects of phenomena, and employ the same data collection and analysis, whereas qualitative research focuses on the humanistic, descriptive, and qualitative aspects of phenomena. 26 , 28

Structured versus unstructured processes

The aims and types of inquiries determine the difference between quantitative and qualitative research. In quantitative research, statistical data and a structured process are usually employed by the researcher. Quantitative research usually suggests quantities (i.e., numbers). 65 On the other hand, researchers typically use opinions, reasons, verbal statements, and an unstructured process in qualitative research. 63 Qualitative research is more related to quality or kind. 65

In quantitative research, the researcher employs a structured process for collecting quantifiable data. Often, a close-ended questionnaire is used wherein the response categories for each question are designed in which values can be assigned and analyzed quantitatively using a common scale. 66 Quantitative research data is processed consecutively from data management, then data analysis, and finally to data interpretation. Data should be free from errors and missing values. In data management, variables are defined and coded. In data analysis, statistics (e.g., descriptive, inferential) as well as central tendency (i.e., mean, median, mode), spread (standard deviation), and parameter estimation (confidence intervals) measures are used. 67

In qualitative research, the researcher uses an unstructured process for collecting data. These non-statistical data may be in the form of statements, stories, or long explanations. Various responses according to respondents may not be easily quantified using a common scale. 66

Composing a qualitative research paper resembles writing a quantitative research paper. Both papers consist of a title, an abstract, an introduction, objectives, methods, findings, and discussion. However, a qualitative research paper is less regimented than a quantitative research paper. 27

Quantitative research as a deductive hypothesis-testing design

Quantitative research can be considered as a hypothesis-testing design as it involves quantification, statistics, and explanations. It flows from theory to data (i.e., deductive), focuses on objective data, and applies theories to address problems. 45 , 68 It collects numerical or statistical data; answers questions such as how many, how often, how much; uses questionnaires, structured interview schedules, or surveys 55 as data collection tools; analyzes quantitative data in terms of percentages, frequencies, statistical comparisons, graphs, and tables showing statistical values; and reports the final findings in the form of statistical information. 66 It uses variable-based models from individual cases and findings are stated in quantified sentences derived by deductive reasoning. 24

In quantitative research, a phenomenon is investigated in terms of the relationship between an independent variable and a dependent variable which are numerically measurable. The research objective is to statistically test whether the hypothesized relationship is true. 68 Here, the researcher studies what others have performed, examines current theories of the phenomenon being investigated, and then tests hypotheses that emerge from those theories. 4

Quantitative hypothesis-testing research has certain limitations. These limitations include (a) problems with selection of meaningful independent and dependent variables, (b) the inability to reflect subjective experiences as variables since variables are usually defined numerically, and (c) the need to state a hypothesis before the investigation starts. 61

Qualitative research as an inductive hypothesis-generating design

Qualitative research can be considered as a hypothesis-generating design since it involves understanding and descriptions in terms of context. It flows from data to theory (i.e., inductive), focuses on observation, and examines what happens in specific situations with the aim of developing new theories based on the situation. 45 , 68 This type of research (a) collects qualitative data (e.g., ideas, statements, reasons, characteristics, qualities), (b) answers questions such as what, why, and how, (c) uses interviews, observations, or focused-group discussions as data collection tools, (d) analyzes data by discovering patterns of changes, causal relationships, or themes in the data; and (e) reports the final findings as descriptive information. 61 Qualitative research favors case-based models from individual characteristics, and findings are stated using context-dependent existential sentences that are justifiable by inductive reasoning. 24

In qualitative research, texts and interviews are analyzed and interpreted to discover meaningful patterns characteristic of a particular phenomenon. 61 Here, the researcher starts with a set of observations and then moves from particular experiences to a more general set of propositions about those experiences. 4

Qualitative hypothesis-generating research involves collecting interview data from study participants regarding a phenomenon of interest, and then using what they say to develop hypotheses. It involves the process of questioning more than obtaining measurements; it generates hypotheses using theoretical coding. 61 When using large interview teams, the key to promoting high-level qualitative research and cohesion in large team methods and successful research outcomes is the balance between autonomy and collaboration. 69

Qualitative data may also include observed behavior, participant observation, media accounts, and cultural artifacts. 61 Focus group interviews are usually conducted, audiotaped or videotaped, and transcribed. Afterwards, the transcript is analyzed by several researchers.

Qualitative research also involves scientific narratives and the analysis and interpretation of textual or numerical data (or both), mostly from conversations and discussions. Such approach uncovers meaningful patterns that describe a particular phenomenon. 2 Thus, qualitative research requires skills in grasping and contextualizing data, as well as communicating data analysis and results in a scientific manner. The reflective process of the inquiry underscores the strengths of a qualitative research approach. 2

Combination of quantitative and qualitative research

When both quantitative and qualitative research methods are used in the same research, mixed-method research is applied. 25 This combination provides a complete view of the research problem and achieves triangulation to corroborate findings, complementarity to clarify results, expansion to extend the study’s breadth, and explanation to elucidate unexpected results. 29

Moreover, quantitative and qualitative findings are integrated to address the weakness of both research methods 29 , 66 and to have a more comprehensive understanding of the phenomenon spectrum. 66

For data analysis in mixed-method research, real non-quantitized qualitative data and quantitative data must both be analyzed. 70 The data obtained from quantitative analysis can be further expanded and deepened by qualitative analysis. 23

In terms of assessment criteria, Hammersley 71 opined that qualitative and quantitative findings should be judged using the same standards of validity and value-relevance. Both approaches can be mutually supportive. 52

Quantitative and qualitative research must be carefully studied and conducted by scientific researchers to avoid unethical research and inadequate outcomes. Quantitative research involves a deductive process wherein a research question is answered with a hypothesis that describes the relationship between independent and dependent variables, and the testing of the hypothesis. This investigation can be aptly termed as hypothesis-testing research involving the analysis of hypothesis-driven experimental studies resulting in a test of significance. Qualitative research involves an inductive process wherein a research question is explored to generate a hypothesis, which then leads to the development of a theory. This investigation can be aptly termed as hypothesis-generating research. When the whole spectrum of inductive and deductive research approaches is combined using both quantitative and qualitative research methodologies, mixed-method research is applied, and this can facilitate the construction of novel hypotheses, development of theories, or refinement of concepts.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Data curation: Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C, Takamiya Y, Izumi M.
• Formal analysis: Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C.
• Investigation: Barroga E, Matanguihan GJ, Takamiya Y, Izumi M.
• Methodology: Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C, Takamiya Y, Izumi M.
• Project administration: Barroga E, Matanguihan GJ.
• Resources: Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C, Takamiya Y, Izumi M.
• Supervision: Barroga E.
• Validation: Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C, Takamiya Y, Izumi M.
• Visualization: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C, Takamiya Y, Izumi M.