conclusions in a research

Get science-backed answers as you write with Paperpal's Research feature

How to Write a Conclusion for Research Papers (with Examples)

The conclusion of a research paper is a crucial section that plays a significant role in the overall impact and effectiveness of your research paper. However, this is also the section that typically receives less attention compared to the introduction and the body of the paper. The conclusion serves to provide a concise summary of the key findings, their significance, their implications, and a sense of closure to the study. Discussing how can the findings be applied in real-world scenarios or inform policy, practice, or decision-making is especially valuable to practitioners and policymakers. The research paper conclusion also provides researchers with clear insights and valuable information for their own work, which they can then build on and contribute to the advancement of knowledge in the field.

The research paper conclusion should explain the significance of your findings within the broader context of your field. It restates how your results contribute to the existing body of knowledge and whether they confirm or challenge existing theories or hypotheses. Also, by identifying unanswered questions or areas requiring further investigation, your awareness of the broader research landscape can be demonstrated.

Remember to tailor the research paper conclusion to the specific needs and interests of your intended audience, which may include researchers, practitioners, policymakers, or a combination of these.

Table of Contents

What is a conclusion in a research paper, summarizing conclusion, editorial conclusion, externalizing conclusion, importance of a good research paper conclusion, how to write a conclusion for your research paper, research paper conclusion examples.

• How to write a research paper conclusion with Paperpal? 

Frequently Asked Questions

A conclusion in a research paper is the final section where you summarize and wrap up your research, presenting the key findings and insights derived from your study. The research paper conclusion is not the place to introduce new information or data that was not discussed in the main body of the paper. When working on how to conclude a research paper, remember to stick to summarizing and interpreting existing content. The research paper conclusion serves the following purposes: 1

• Warn readers of the possible consequences of not attending to the problem.
• Recommend specific course(s) of action.
• Restate key ideas to drive home the ultimate point of your research paper.
• Provide a “take-home” message that you want the readers to remember about your study.

Types of conclusions for research papers

In research papers, the conclusion provides closure to the reader. The type of research paper conclusion you choose depends on the nature of your study, your goals, and your target audience. I provide you with three common types of conclusions:

A summarizing conclusion is the most common type of conclusion in research papers. It involves summarizing the main points, reiterating the research question, and restating the significance of the findings. This common type of research paper conclusion is used across different disciplines.

An editorial conclusion is less common but can be used in research papers that are focused on proposing or advocating for a particular viewpoint or policy. It involves presenting a strong editorial or opinion based on the research findings and offering recommendations or calls to action.

An externalizing conclusion is a type of conclusion that extends the research beyond the scope of the paper by suggesting potential future research directions or discussing the broader implications of the findings. This type of conclusion is often used in more theoretical or exploratory research papers.

Align your conclusion’s tone with the rest of your research paper. Start Writing with Paperpal Now!  

The conclusion in a research paper serves several important purposes:

• Offers Implications and Recommendations : Your research paper conclusion is an excellent place to discuss the broader implications of your research and suggest potential areas for further study. It’s also an opportunity to offer practical recommendations based on your findings.
• Provides Closure : A good research paper conclusion provides a sense of closure to your paper. It should leave the reader with a feeling that they have reached the end of a well-structured and thought-provoking research project.
• Leaves a Lasting Impression : Writing a well-crafted research paper conclusion leaves a lasting impression on your readers. It’s your final opportunity to leave them with a new idea, a call to action, or a memorable quote.

Writing a strong conclusion for your research paper is essential to leave a lasting impression on your readers. Here’s a step-by-step process to help you create and know what to put in the conclusion of a research paper: 2

• Research Statement : Begin your research paper conclusion by restating your research statement. This reminds the reader of the main point you’ve been trying to prove throughout your paper. Keep it concise and clear.
• Key Points : Summarize the main arguments and key points you’ve made in your paper. Avoid introducing new information in the research paper conclusion. Instead, provide a concise overview of what you’ve discussed in the body of your paper.
• Address the Research Questions : If your research paper is based on specific research questions or hypotheses, briefly address whether you’ve answered them or achieved your research goals. Discuss the significance of your findings in this context.
• Significance : Highlight the importance of your research and its relevance in the broader context. Explain why your findings matter and how they contribute to the existing knowledge in your field.
• Implications : Explore the practical or theoretical implications of your research. How might your findings impact future research, policy, or real-world applications? Consider the “so what?” question.
• Future Research : Offer suggestions for future research in your area. What questions or aspects remain unanswered or warrant further investigation? This shows that your work opens the door for future exploration.
• Closing Thought : Conclude your research paper conclusion with a thought-provoking or memorable statement. This can leave a lasting impression on your readers and wrap up your paper effectively. Avoid introducing new information or arguments here.
• Proofread and Revise : Carefully proofread your conclusion for grammar, spelling, and clarity. Ensure that your ideas flow smoothly and that your conclusion is coherent and well-structured.

Write your research paper conclusion 2x faster with Paperpal. Try it now!

Remember that a well-crafted research paper conclusion is a reflection of the strength of your research and your ability to communicate its significance effectively. It should leave a lasting impression on your readers and tie together all the threads of your paper. Now you know how to start the conclusion of a research paper and what elements to include to make it impactful, let’s look at a research paper conclusion sample.

How to write a research paper conclusion with Paperpal?

A research paper conclusion is not just a summary of your study, but a synthesis of the key findings that ties the research together and places it in a broader context. A research paper conclusion should be concise, typically around one paragraph in length. However, some complex topics may require a longer conclusion to ensure the reader is left with a clear understanding of the study’s significance. Paperpal, an AI writing assistant trusted by over 800,000 academics globally, can help you write a well-structured conclusion for your research paper. 

• Sign Up or Log In: Create a new Paperpal account or login with your details.  
• Navigate to Features : Once logged in, head over to the features’ side navigation pane. Click on Templates and you’ll find a suite of generative AI features to help you write better, faster.  
• Generate an outline: Under Templates, select ‘Outlines’. Choose ‘Research article’ as your document type.  
• Select your section: Since you’re focusing on the conclusion, select this section when prompted.  
• Choose your field of study: Identifying your field of study allows Paperpal to provide more targeted suggestions, ensuring the relevance of your conclusion to your specific area of research. 
• Provide a brief description of your study: Enter details about your research topic and findings. This information helps Paperpal generate a tailored outline that aligns with your paper’s content. 
• Generate the conclusion outline: After entering all necessary details, click on ‘generate’. Paperpal will then create a structured outline for your conclusion, to help you start writing and build upon the outline.  
• Write your conclusion: Use the generated outline to build your conclusion. The outline serves as a guide, ensuring you cover all critical aspects of a strong conclusion, from summarizing key findings to highlighting the research’s implications. 
• Refine and enhance: Paperpal’s ‘Make Academic’ feature can be particularly useful in the final stages. Select any paragraph of your conclusion and use this feature to elevate the academic tone, ensuring your writing is aligned to the academic journal standards. 

By following these steps, Paperpal not only simplifies the process of writing a research paper conclusion but also ensures it is impactful, concise, and aligned with academic standards. Sign up with Paperpal today and write your research paper conclusion 2x faster .  

The research paper conclusion is a crucial part of your paper as it provides the final opportunity to leave a strong impression on your readers. In the research paper conclusion, summarize the main points of your research paper by restating your research statement, highlighting the most important findings, addressing the research questions or objectives, explaining the broader context of the study, discussing the significance of your findings, providing recommendations if applicable, and emphasizing the takeaway message. The main purpose of the conclusion is to remind the reader of the main point or argument of your paper and to provide a clear and concise summary of the key findings and their implications. All these elements should feature on your list of what to put in the conclusion of a research paper to create a strong final statement for your work.

A strong conclusion is a critical component of a research paper, as it provides an opportunity to wrap up your arguments, reiterate your main points, and leave a lasting impression on your readers. Here are the key elements of a strong research paper conclusion: 1. Conciseness : A research paper conclusion should be concise and to the point. It should not introduce new information or ideas that were not discussed in the body of the paper. 2. Summarization : The research paper conclusion should be comprehensive enough to give the reader a clear understanding of the research’s main contributions. 3 . Relevance : Ensure that the information included in the research paper conclusion is directly relevant to the research paper’s main topic and objectives; avoid unnecessary details. 4 . Connection to the Introduction : A well-structured research paper conclusion often revisits the key points made in the introduction and shows how the research has addressed the initial questions or objectives. 5. Emphasis : Highlight the significance and implications of your research. Why is your study important? What are the broader implications or applications of your findings? 6 . Call to Action : Include a call to action or a recommendation for future research or action based on your findings.

The length of a research paper conclusion can vary depending on several factors, including the overall length of the paper, the complexity of the research, and the specific journal requirements. While there is no strict rule for the length of a conclusion, but it’s generally advisable to keep it relatively short. A typical research paper conclusion might be around 5-10% of the paper’s total length. For example, if your paper is 10 pages long, the conclusion might be roughly half a page to one page in length.

In general, you do not need to include citations in the research paper conclusion. Citations are typically reserved for the body of the paper to support your arguments and provide evidence for your claims. However, there may be some exceptions to this rule: 1. If you are drawing a direct quote or paraphrasing a specific source in your research paper conclusion, you should include a citation to give proper credit to the original author. 2. If your conclusion refers to or discusses specific research, data, or sources that are crucial to the overall argument, citations can be included to reinforce your conclusion’s validity.

The conclusion of a research paper serves several important purposes: 1. Summarize the Key Points 2. Reinforce the Main Argument 3. Provide Closure 4. Offer Insights or Implications 5. Engage the Reader. 6. Reflect on Limitations

Remember that the primary purpose of the research paper conclusion is to leave a lasting impression on the reader, reinforcing the key points and providing closure to your research. It’s often the last part of the paper that the reader will see, so it should be strong and well-crafted.

• Makar, G., Foltz, C., Lendner, M., & Vaccaro, A. R. (2018). How to write effective discussion and conclusion sections. Clinical spine surgery, 31(8), 345-346.
• Bunton, D. (2005). The structure of PhD conclusion chapters.  Journal of English for academic purposes ,  4 (3), 207-224.

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• 5 Reasons for Rejection After Peer Review
• Ethical Research Practices For Research with Human Subjects

7 Ways to Improve Your Academic Writing Process

• Paraphrasing in Academic Writing: Answering Top Author Queries

Preflight For Editorial Desk: The Perfect Hybrid (AI + Human) Assistance Against Compromised Manuscripts

You may also like, how to write the first draft of a..., mla works cited page: format, template & examples, how to write a high-quality conference paper, academic editing: how to self-edit academic text with..., measuring academic success: definition & strategies for excellence, phd qualifying exam: tips for success , ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., what are journal guidelines on using generative ai..., quillbot review: features, pricing, and free alternatives.

• Privacy Policy

Home » Research Paper Conclusion – Writing Guide and Examples

Research Paper Conclusion – Writing Guide and Examples

Table of Contents

Research Paper Conclusion

Definition:

A research paper conclusion is the final section of a research paper that summarizes the key findings, significance, and implications of the research. It is the writer’s opportunity to synthesize the information presented in the paper, draw conclusions, and make recommendations for future research or actions.

The conclusion should provide a clear and concise summary of the research paper, reiterating the research question or problem, the main results, and the significance of the findings. It should also discuss the limitations of the study and suggest areas for further research.

Parts of Research Paper Conclusion

The parts of a research paper conclusion typically include:

Restatement of the Thesis

The conclusion should begin by restating the thesis statement from the introduction in a different way. This helps to remind the reader of the main argument or purpose of the research.

Summary of Key Findings

The conclusion should summarize the main findings of the research, highlighting the most important results and conclusions. This section should be brief and to the point.

Implications and Significance

In this section, the researcher should explain the implications and significance of the research findings. This may include discussing the potential impact on the field or industry, highlighting new insights or knowledge gained, or pointing out areas for future research.

Limitations and Recommendations

It is important to acknowledge any limitations or weaknesses of the research and to make recommendations for how these could be addressed in future studies. This shows that the researcher is aware of the potential limitations of their work and is committed to improving the quality of research in their field.

Concluding Statement

The conclusion should end with a strong concluding statement that leaves a lasting impression on the reader. This could be a call to action, a recommendation for further research, or a final thought on the topic.

How to Write Research Paper Conclusion

Here are some steps you can follow to write an effective research paper conclusion:

• Restate the research problem or question: Begin by restating the research problem or question that you aimed to answer in your research. This will remind the reader of the purpose of your study.
• Summarize the main points: Summarize the key findings and results of your research. This can be done by highlighting the most important aspects of your research and the evidence that supports them.
• Discuss the implications: Discuss the implications of your findings for the research area and any potential applications of your research. You should also mention any limitations of your research that may affect the interpretation of your findings.
• Provide a conclusion : Provide a concise conclusion that summarizes the main points of your paper and emphasizes the significance of your research. This should be a strong and clear statement that leaves a lasting impression on the reader.
• Offer suggestions for future research: Lastly, offer suggestions for future research that could build on your findings and contribute to further advancements in the field.

Remember that the conclusion should be brief and to the point, while still effectively summarizing the key findings and implications of your research.

Example of Research Paper Conclusion

Here’s an example of a research paper conclusion:

Conclusion :

In conclusion, our study aimed to investigate the relationship between social media use and mental health among college students. Our findings suggest that there is a significant association between social media use and increased levels of anxiety and depression among college students. This highlights the need for increased awareness and education about the potential negative effects of social media use on mental health, particularly among college students.

Despite the limitations of our study, such as the small sample size and self-reported data, our findings have important implications for future research and practice. Future studies should aim to replicate our findings in larger, more diverse samples, and investigate the potential mechanisms underlying the association between social media use and mental health. In addition, interventions should be developed to promote healthy social media use among college students, such as mindfulness-based approaches and social media detox programs.

Overall, our study contributes to the growing body of research on the impact of social media on mental health, and highlights the importance of addressing this issue in the context of higher education. By raising awareness and promoting healthy social media use among college students, we can help to reduce the negative impact of social media on mental health and improve the well-being of young adults.

Purpose of Research Paper Conclusion

The purpose of a research paper conclusion is to provide a summary and synthesis of the key findings, significance, and implications of the research presented in the paper. The conclusion serves as the final opportunity for the writer to convey their message and leave a lasting impression on the reader.

The conclusion should restate the research problem or question, summarize the main results of the research, and explain their significance. It should also acknowledge the limitations of the study and suggest areas for future research or action.

Overall, the purpose of the conclusion is to provide a sense of closure to the research paper and to emphasize the importance of the research and its potential impact. It should leave the reader with a clear understanding of the main findings and why they matter. The conclusion serves as the writer’s opportunity to showcase their contribution to the field and to inspire further research and action.

When to Write Research Paper Conclusion

The conclusion of a research paper should be written after the body of the paper has been completed. It should not be written until the writer has thoroughly analyzed and interpreted their findings and has written a complete and cohesive discussion of the research.

Before writing the conclusion, the writer should review their research paper and consider the key points that they want to convey to the reader. They should also review the research question, hypotheses, and methodology to ensure that they have addressed all of the necessary components of the research.

Once the writer has a clear understanding of the main findings and their significance, they can begin writing the conclusion. The conclusion should be written in a clear and concise manner, and should reiterate the main points of the research while also providing insights and recommendations for future research or action.

Characteristics of Research Paper Conclusion

The characteristics of a research paper conclusion include:

• Clear and concise: The conclusion should be written in a clear and concise manner, summarizing the key findings and their significance.
• Comprehensive: The conclusion should address all of the main points of the research paper, including the research question or problem, the methodology, the main results, and their implications.
• Future-oriented : The conclusion should provide insights and recommendations for future research or action, based on the findings of the research.
• Impressive : The conclusion should leave a lasting impression on the reader, emphasizing the importance of the research and its potential impact.
• Objective : The conclusion should be based on the evidence presented in the research paper, and should avoid personal biases or opinions.
• Unique : The conclusion should be unique to the research paper and should not simply repeat information from the introduction or body of the paper.

Advantages of Research Paper Conclusion

The advantages of a research paper conclusion include:

• Summarizing the key findings : The conclusion provides a summary of the main findings of the research, making it easier for the reader to understand the key points of the study.
• Emphasizing the significance of the research: The conclusion emphasizes the importance of the research and its potential impact, making it more likely that readers will take the research seriously and consider its implications.
• Providing recommendations for future research or action : The conclusion suggests practical recommendations for future research or action, based on the findings of the study.
• Providing closure to the research paper : The conclusion provides a sense of closure to the research paper, tying together the different sections of the paper and leaving a lasting impression on the reader.
• Demonstrating the writer’s contribution to the field : The conclusion provides the writer with an opportunity to showcase their contribution to the field and to inspire further research and action.

Limitations of Research Paper Conclusion

While the conclusion of a research paper has many advantages, it also has some limitations that should be considered, including:

• I nability to address all aspects of the research: Due to the limited space available in the conclusion, it may not be possible to address all aspects of the research in detail.
• Subjectivity : While the conclusion should be objective, it may be influenced by the writer’s personal biases or opinions.
• Lack of new information: The conclusion should not introduce new information that has not been discussed in the body of the research paper.
• Lack of generalizability: The conclusions drawn from the research may not be applicable to other contexts or populations, limiting the generalizability of the study.
• Misinterpretation by the reader: The reader may misinterpret the conclusions drawn from the research, leading to a misunderstanding of the findings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Data Collection – Methods Types and Examples

Research Paper Format – Types, Examples and...

Scope of the Research – Writing Guide and...

References in Research – Types, Examples and...

Research Paper Abstract – Writing Guide and...

Research Techniques – Methods, Types and Examples

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 9. The Conclusion
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The conclusion is intended to help the reader understand why your research should matter to them after they have finished reading the paper. A conclusion is not merely a summary of the main topics covered or a re-statement of your research problem, but a synthesis of key points derived from the findings of your study and, if applicable based on your analysis, explain new areas for future research. For most college-level research papers, two or three well-developed paragraphs is sufficient for a conclusion, although in some cases, more paragraphs may be required in describing the key findings and highlighting their significance.

Conclusions. The Writing Center. University of North Carolina; Conclusions. The Writing Lab and The OWL. Purdue University.

Importance of a Good Conclusion

A well-written conclusion provides important opportunities to demonstrate to the reader your understanding of the research problem. These include:

• Presenting the last word on the issues you raised in your paper . Just as the introduction gives a first impression to your reader, the conclusion offers a chance to leave a lasting impression. Do this, for example, by highlighting key findings in your analysis that advance new understanding about the research problem, that are unusual or unexpected, or that have important implications applied to practice.
• Summarizing your thoughts and conveying the larger significance of your study . The conclusion is an opportunity to succinctly re-emphasize  your answer to the "So What?" question by placing the study within the context of how your research advances past studies about the topic.
• Identifying how a gap in the literature has been addressed . The conclusion can be where you describe how a previously identified gap in the literature [first identified in your literature review section] has been addressed by your research and why this contribution is significant.
• Demonstrating the importance of your ideas . Don't be shy. The conclusion offers an opportunity to elaborate on the impact and significance of your findings. This is particularly important if your study approached examining the research problem from an unusual or innovative perspective.
• Introducing possible new or expanded ways of thinking about the research problem . This does not refer to introducing new information [which should be avoided], but to offer new insight and creative approaches for framing or contextualizing the research problem based on the results of your study.

Bunton, David. “The Structure of PhD Conclusion Chapters.” Journal of English for Academic Purposes 4 (July 2005): 207–224; Conclusions. The Writing Center. University of North Carolina; Kretchmer, Paul. Twelve Steps to Writing an Effective Conclusion. San Francisco Edit, 2003-2008; Conclusions. The Writing Lab and The OWL. Purdue University; Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8.

Structure and Writing Style

I.  General Rules

The general function of your paper's conclusion is to restate the main argument . It reminds the reader of your main argument(s) strengths and reiterates the most important evidence supporting those argument(s). Do this by clearly summarizing the context, background, and the necessity of examining the research problem in relation to an issue, controversy, or a gap found in the literature. However, make sure that your conclusion is not simply a repetitive summary of the findings. This reduces the impact of the argument(s) you have developed in your paper.

When writing the conclusion to your paper, follow these general rules:

• Present your conclusions in clear, concise language. Re-state the purpose of your study, then describe how your findings differ or support those of other studies and why [i.e., describe what were the unique, new, or crucial contributions your study made to the overall research about your topic].
• Do not simply reiterate your findings or the discussion of your results. Provide a synthesis of arguments presented in the paper to show how these converge to address the research problem and the overall objectives of your study.
• Indicate opportunities for future research if you haven't already done so in the discussion section of your paper. Highlighting the need for further research provides the reader with evidence that you have an in-depth awareness of the research problem but that further analysis should take place beyond the scope of your investigation.

Consider the following points to help ensure your conclusion is presented well:

• If the argument or purpose of your paper is complex, you may need to summarize the argument for your reader.
• If, prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the end of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration that returns the topic to the context provided by the introduction or within a new context that emerges from the data [this is opposite of the introduction, which begins with general discussion of the context and ends with a detailed description of the research problem]. 

The conclusion also provides a place for you to persuasively and succinctly restate the research problem, given that the reader has now been presented with all the information about the topic . Depending on the discipline you are writing in, the concluding paragraph may contain your reflections on the evidence presented. However, the nature of being introspective about the research you have conducted will depend on the topic and whether your professor wants you to express your observations in this way. If asked to think introspectively about the topic, do not delve into idle speculation. Being introspective means looking within yourself as an author to try and understand an issue more deeply, not to guess at possible outcomes or make up scenarios not supported by the evidence.

II.  Developing a Compelling Conclusion

Although an effective conclusion needs to be clear and succinct, it does not need to be written passively or lack a compelling narrative. Strategies to help you move beyond merely summarizing the key points of your research paper may include any of the following:

• If your paper addresses a critical, contemporary problem, warn readers of the possible consequences of not attending to the problem proactively based on the evidence presented in your study.
• Recommend a specific course or courses of action that, if adopted, could address a specific problem in practice or in the development of new knowledge leading to positive change.
• Cite a relevant quotation or expert opinion already noted in your paper in order to lend authority and support to the conclusion(s) you have reached [a good source would be from a source cited in your literature review].
• Explain the consequences of your research in a way that elicits action or demonstrates urgency in seeking change.
• Restate a key statistic, fact, or visual image to emphasize the most important finding of your paper.
• If your discipline encourages personal reflection, illustrate your concluding point by drawing from your own life experiences.
• Return to an anecdote, an example, or a quotation that you presented in your introduction, but add further insight derived from the findings of your study; use your interpretation of results from your study to recast it in new or important ways.
• Provide a "take-home" message in the form of a succinct, declarative statement that you want the reader to remember about your study.

III. Problems to Avoid

Failure to be concise Your conclusion section should be concise and to the point. Conclusions that are too lengthy often have unnecessary information in them. The conclusion is not the place for details about your methodology or results. Although you should give a summary of what was learned from your research, this summary should be relatively brief, since the emphasis in the conclusion is on the implications, evaluations, insights, and other forms of analysis that you make. Strategies for writing concisely can be found here .

Failure to comment on larger, more significant issues In the introduction, your task was to move from the general [topic studied within the field of study] to the specific [the research problem]. However, in the conclusion, your task is to move the discussion from specific [your research problem] back to a general discussion framed around the implications and significance of your findings [i.e., how your research contributes new understanding or fills an important gap in the literature]. In short, the conclusion is where you should place your research within a larger context [visualize the structure of your paper as an hourglass--start with a broad introduction and review of the literature, move to the specific method of analysis and the discussion, conclude with a broad summary of the study's implications and significance].

Failure to reveal problems and negative results Negative aspects of the research process should never be ignored. These are problems, deficiencies, or challenges encountered during your study. They should be summarized as a way of qualifying your overall conclusions. If you encountered negative or unintended results [i.e., findings that are validated outside the research context in which they were generated], you must report them in the results section and discuss their implications in the discussion section of your paper. In the conclusion, use negative or surprising results as an opportunity to explain their possible significance and/or how they may form the basis for future research.

Failure to provide a clear summary of what was learned In order to discuss how your research fits within your field of study [and possibly the world at large], you need to summarize briefly and succinctly how it contributes to new knowledge or a new understanding about the research problem. This element of your conclusion may be only a few sentences long, but it often represents the key takeaway for your reader.

Failure to match the objectives of your research Often research objectives in the social and behavioral sciences change while the research is being carried out due to unforeseen factors or unanticipated variables. This is not a problem unless you forget to go back and refine the original objectives in your introduction. As these changes emerge they must be documented so that they accurately reflect what you were trying to accomplish in your research [not what you thought you might accomplish when you began].

Resist the urge to apologize If you've immersed yourself in studying the research problem, you presumably should know a good deal about it [perhaps even more than your professor!]. Nevertheless, by the time you have finished writing, you may be having some doubts about what you have produced. Repress those doubts! Don't undermine your authority as a researcher by saying something like, "This is just one approach to examining this problem; there may be other, much better approaches that...." The overall tone of your conclusion should convey confidence to the reader concerning the validity and realiability of your research.

Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8; Concluding Paragraphs. College Writing Center at Meramec. St. Louis Community College; Conclusions. The Writing Center. University of North Carolina; Conclusions. The Writing Lab and The OWL. Purdue University; Freedman, Leora  and Jerry Plotnick. Introductions and Conclusions. The Lab Report. University College Writing Centre. University of Toronto; Leibensperger, Summer. Draft Your Conclusion. Academic Center, the University of Houston-Victoria, 2003; Make Your Last Words Count. The Writer’s Handbook. Writing Center. University of Wisconsin Madison; Miquel, Fuster-Marquez and Carmen Gregori-Signes. “Chapter Six: ‘Last but Not Least:’ Writing the Conclusion of Your Paper.” In Writing an Applied Linguistics Thesis or Dissertation: A Guide to Presenting Empirical Research . John Bitchener, editor. (Basingstoke,UK: Palgrave Macmillan, 2010), pp. 93-105; Tips for Writing a Good Conclusion. Writing@CSU. Colorado State University; Kretchmer, Paul. Twelve Steps to Writing an Effective Conclusion. San Francisco Edit, 2003-2008; Writing Conclusions. Writing Tutorial Services, Center for Innovative Teaching and Learning. Indiana University; Writing: Considering Structure and Organization. Institute for Writing Rhetoric. Dartmouth College.

Writing Tip

Don't Belabor the Obvious!

Avoid phrases like "in conclusion...," "in summary...," or "in closing...." These phrases can be useful, even welcome, in oral presentations. But readers can see by the tell-tale section heading and number of pages remaining that they are reaching the end of your paper. You'll irritate your readers if you belabor the obvious.

Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8.

Another Writing Tip

New Insight, Not New Information!

Don't surprise the reader with new information in your conclusion that was never referenced anywhere else in the paper. This is why the conclusion rarely has citations to sources that haven't been referenced elsewhere in your paper. If you have new information to present, add it to the discussion or other appropriate section of the paper. Note that, although no new information is introduced, the conclusion, along with the discussion section, is where you offer your most "original" contributions in the paper; the conclusion is where you describe the value of your research, demonstrate that you understand the material that you have presented, and position your findings within the larger context of scholarship on the topic, including describing how your research contributes new insights to that scholarship.

Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8; Conclusions. The Writing Center. University of North Carolina.

• << Previous: Limitations of the Study
• Next: Appendices >>
• Last Updated: Jul 3, 2024 10:07 AM
• URL: https://libguides.usc.edu/writingguide

How to write a strong conclusion for your research paper

Last updated

17 February 2024

Reviewed by

Short on time? Get an AI generated summary of this article instead

Writing a research paper is a chance to share your knowledge and hypothesis. It's an opportunity to demonstrate your many hours of research and prove your ability to write convincingly.

Ideally, by the end of your research paper, you'll have brought your readers on a journey to reach the conclusions you've pre-determined. However, if you don't stick the landing with a good conclusion, you'll risk losing your reader’s trust.

Writing a strong conclusion for your research paper involves a few important steps, including restating the thesis and summing up everything properly.

Find out what to include and what to avoid, so you can effectively demonstrate your understanding of the topic and prove your expertise.

• Why is a good conclusion important?

A good conclusion can cement your paper in the reader’s mind. Making a strong impression in your introduction can draw your readers in, but it's the conclusion that will inspire them.

• What to include in a research paper conclusion

There are a few specifics you should include in your research paper conclusion. Offer your readers some sense of urgency or consequence by pointing out why they should care about the topic you have covered. Discuss any common problems associated with your topic and provide suggestions as to how these problems can be solved or addressed.

The conclusion should include a restatement of your initial thesis. Thesis statements are strengthened after you’ve presented supporting evidence (as you will have done in the paper), so make a point to reintroduce it at the end.

Finally, recap the main points of your research paper, highlighting the key takeaways you want readers to remember. If you've made multiple points throughout the paper, refer to the ones with the strongest supporting evidence.

• Steps for writing a research paper conclusion

Many writers find the conclusion the most challenging part of any research project . By following these three steps, you'll be prepared to write a conclusion that is effective and concise.

• Step 1: Restate the problem

Always begin by restating the research problem in the conclusion of a research paper. This serves to remind the reader of your hypothesis and refresh them on the main point of the paper. 

When restating the problem, take care to avoid using exactly the same words you employed earlier in the paper.

• Step 2: Sum up the paper

After you've restated the problem, sum up the paper by revealing your overall findings. The method for this differs slightly, depending on whether you're crafting an argumentative paper or an empirical paper.

Argumentative paper: Restate your thesis and arguments

Argumentative papers involve introducing a thesis statement early on. In crafting the conclusion for an argumentative paper, always restate the thesis, outlining the way you've developed it throughout the entire paper.

It might be appropriate to mention any counterarguments in the conclusion, so you can demonstrate how your thesis is correct or how the data best supports your main points.

Empirical paper: Summarize research findings

Empirical papers break down a series of research questions. In your conclusion, discuss the findings your research revealed, including any information that surprised you.

Be clear about the conclusions you reached, and explain whether or not you expected to arrive at these particular ones.

• Step 3: Discuss the implications of your research

Argumentative papers and empirical papers also differ in this part of a research paper conclusion. Here are some tips on crafting conclusions for argumentative and empirical papers.

Argumentative paper: Powerful closing statement

In an argumentative paper, you'll have spent a great deal of time expressing the opinions you formed after doing a significant amount of research. Make a strong closing statement in your argumentative paper's conclusion to share the significance of your work.

You can outline the next steps through a bold call to action, or restate how powerful your ideas turned out to be.

Empirical paper: Directions for future research

Empirical papers are broader in scope. They usually cover a variety of aspects and can include several points of view.

To write a good conclusion for an empirical paper, suggest the type of research that could be done in the future, including methods for further investigation or outlining ways other researchers might proceed.

If you feel your research had any limitations, even if they were outside your control, you could mention these in your conclusion.

After you finish outlining your conclusion, ask someone to read it and offer feedback. In any research project you're especially close to, it can be hard to identify problem areas. Having a close friend or someone whose opinion you value read the research paper and provide honest feedback can be invaluable. Take note of any suggested edits and consider incorporating them into your paper if they make sense.

• Things to avoid in a research paper conclusion

Keep these aspects to avoid in mind as you're writing your conclusion and refer to them after you've created an outline.

Dry summary

Writing a memorable, succinct conclusion is arguably more important than a strong introduction. Take care to avoid just rephrasing your main points, and don't fall into the trap of repeating dry facts or citations.

You can provide a new perspective for your readers to think about or contextualize your research. Either way, make the conclusion vibrant and interesting, rather than a rote recitation of your research paper’s highlights.

Clichéd or generic phrasing

Your research paper conclusion should feel fresh and inspiring. Avoid generic phrases like "to sum up" or "in conclusion." These phrases tend to be overused, especially in an academic context and might turn your readers off.

The conclusion also isn't the time to introduce colloquial phrases or informal language. Retain a professional, confident tone consistent throughout your paper’s conclusion so it feels exciting and bold.

New data or evidence

While you should present strong data throughout your paper, the conclusion isn't the place to introduce new evidence. This is because readers are engaged in actively learning as they read through the body of your paper.

By the time they reach the conclusion, they will have formed an opinion one way or the other (hopefully in your favor!). Introducing new evidence in the conclusion will only serve to surprise or frustrate your reader.

Ignoring contradictory evidence

If your research reveals contradictory evidence, don't ignore it in the conclusion. This will damage your credibility as an expert and might even serve to highlight the contradictions.

Be as transparent as possible and admit to any shortcomings in your research, but don't dwell on them for too long.

Ambiguous or unclear resolutions

The point of a research paper conclusion is to provide closure and bring all your ideas together. You should wrap up any arguments you introduced in the paper and tie up any loose ends, while demonstrating why your research and data are strong.

Use direct language in your conclusion and avoid ambiguity. Even if some of the data and sources you cite are inconclusive or contradictory, note this in your conclusion to come across as confident and trustworthy.

• Examples of research paper conclusions

Your research paper should provide a compelling close to the paper as a whole, highlighting your research and hard work. While the conclusion should represent your unique style, these examples offer a starting point:

Ultimately, the data we examined all point to the same conclusion: Encouraging a good work-life balance improves employee productivity and benefits the company overall. The research suggests that when employees feel their personal lives are valued and respected by their employers, they are more likely to be productive when at work. In addition, company turnover tends to be reduced when employees have a balance between their personal and professional lives. While additional research is required to establish ways companies can support employees in creating a stronger work-life balance, it's clear the need is there.

Social media is a primary method of communication among young people. As we've seen in the data presented, most young people in high school use a variety of social media applications at least every hour, including Instagram and Facebook. While social media is an avenue for connection with peers, research increasingly suggests that social media use correlates with body image issues. Young girls with lower self-esteem tend to use social media more often than those who don't log onto social media apps every day. As new applications continue to gain popularity, and as more high school students are given smartphones, more research will be required to measure the effects of prolonged social media use.

What are the different kinds of research paper conclusions?

There are no formal types of research paper conclusions. Ultimately, the conclusion depends on the outline of your paper and the type of research you’re presenting. While some experts note that research papers can end with a new perspective or commentary, most papers should conclude with a combination of both. The most important aspect of a good research paper conclusion is that it accurately represents the body of the paper.

Can I present new arguments in my research paper conclusion?

Research paper conclusions are not the place to introduce new data or arguments. The body of your paper is where you should share research and insights, where the reader is actively absorbing the content. By the time a reader reaches the conclusion of the research paper, they should have formed their opinion. Introducing new arguments in the conclusion can take a reader by surprise, and not in a positive way. It might also serve to frustrate readers.

How long should a research paper conclusion be?

There's no set length for a research paper conclusion. However, it's a good idea not to run on too long, since conclusions are supposed to be succinct. A good rule of thumb is to keep your conclusion around 5 to 10 percent of the paper's total length. If your paper is 10 pages, try to keep your conclusion under one page.

What should I include in a research paper conclusion?

A good research paper conclusion should always include a sense of urgency, so the reader can see how and why the topic should matter to them. You can also note some recommended actions to help fix the problem and some obstacles they might encounter. A conclusion should also remind the reader of the thesis statement, along with the main points you covered in the paper. At the end of the conclusion, add a powerful closing statement that helps cement the paper in the mind of the reader.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 6 February 2023

Last updated: 6 October 2023

Last updated: 5 February 2023

Last updated: 16 April 2023

Last updated: 7 March 2023

Last updated: 9 March 2023

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

Users report unexpectedly high data usage, especially during streaming sessions.

Users find it hard to navigate from the home page to relevant playlists in the app.

It would be great to have a sleep timer feature, especially for bedtime listening.

I need better filters to find the songs or artists I’m looking for.

Log in or sign up

Get started for free

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

How to Write a Conclusion for a Research Paper

3-minute read

• 29th August 2023

If you’re writing a research paper, the conclusion is your opportunity to summarize your findings and leave a lasting impression on your readers. In this post, we’ll take you through how to write an effective conclusion for a research paper and how you can:

·   Reword your thesis statement

·   Highlight the significance of your research

·   Discuss limitations

·   Connect to the introduction

·   End with a thought-provoking statement

Rewording Your Thesis Statement

Begin your conclusion by restating your thesis statement in a way that is slightly different from the wording used in the introduction. Avoid presenting new information or evidence in your conclusion. Just summarize the main points and arguments of your essay and keep this part as concise as possible. Remember that you’ve already covered the in-depth analyses and investigations in the main body paragraphs of your essay, so it’s not necessary to restate these details in the conclusion.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

Highlighting the Significance of Your Research

The conclusion is a good place to emphasize the implications of your research . Avoid ambiguous or vague language such as “I think” or “maybe,” which could weaken your position. Clearly explain why your research is significant and how it contributes to the broader field of study.

Here’s an example from a (fictional) study on the impact of social media on mental health:

Discussing Limitations

Although it’s important to emphasize the significance of your study, you can also use the conclusion to briefly address any limitations you discovered while conducting your research, such as time constraints or a shortage of resources. Doing this demonstrates a balanced and honest approach to your research.

Connecting to the Introduction

In your conclusion, you can circle back to your introduction , perhaps by referring to a quote or anecdote you discussed earlier. If you end your paper on a similar note to how you began it, you will create a sense of cohesion for the reader and remind them of the meaning and significance of your research.

Ending With a Thought-Provoking Statement

Consider ending your paper with a thought-provoking and memorable statement that relates to the impact of your research questions or hypothesis. This statement can be a call to action, a philosophical question, or a prediction for the future (positive or negative). Here’s an example that uses the same topic as above (social media and mental health):

Expert Proofreading Services

Ensure that your essay ends on a high note by having our experts proofread your research paper. Our team has experience with a wide variety of academic fields and subjects and can help make your paper stand out from the crowd – get started today and see the difference it can make in your work.

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

9-minute read

How to Use Infographics to Boost Your Presentation

Is your content getting noticed? Capturing and maintaining an audience’s attention is a challenge when...

8-minute read

Why Interactive PDFs Are Better for Engagement

Are you looking to enhance engagement and captivate your audience through your professional documents? Interactive...

7-minute read

Seven Key Strategies for Voice Search Optimization

Voice search optimization is rapidly shaping the digital landscape, requiring content professionals to adapt their...

4-minute read

Five Creative Ways to Showcase Your Digital Portfolio

Are you a creative freelancer looking to make a lasting impression on potential clients or...

How to Ace Slack Messaging for Contractors and Freelancers

Effective professional communication is an important skill for contractors and freelancers navigating remote work environments....

How to Insert a Text Box in a Google Doc

Google Docs is a powerful collaborative tool, and mastering its features can significantly enhance your...

Make sure your writing is the best it can be with our expert English proofreading and editing.

Conclusions

What this handout is about.

This handout will explain the functions of conclusions, offer strategies for writing effective ones, help you evaluate conclusions you’ve drafted, and suggest approaches to avoid.

About conclusions

Introductions and conclusions can be difficult to write, but they’re worth investing time in. They can have a significant influence on a reader’s experience of your paper.

Just as your introduction acts as a bridge that transports your readers from their own lives into the “place” of your analysis, your conclusion can provide a bridge to help your readers make the transition back to their daily lives. Such a conclusion will help them see why all your analysis and information should matter to them after they put the paper down.

Your conclusion is your chance to have the last word on the subject. The conclusion allows you to have the final say on the issues you have raised in your paper, to synthesize your thoughts, to demonstrate the importance of your ideas, and to propel your reader to a new view of the subject. It is also your opportunity to make a good final impression and to end on a positive note.

Your conclusion can go beyond the confines of the assignment. The conclusion pushes beyond the boundaries of the prompt and allows you to consider broader issues, make new connections, and elaborate on the significance of your findings.

Your conclusion should make your readers glad they read your paper. Your conclusion gives your reader something to take away that will help them see things differently or appreciate your topic in personally relevant ways. It can suggest broader implications that will not only interest your reader, but also enrich your reader’s life in some way. It is your gift to the reader.

Strategies for writing an effective conclusion

One or more of the following strategies may help you write an effective conclusion:

• Play the “So What” Game. If you’re stuck and feel like your conclusion isn’t saying anything new or interesting, ask a friend to read it with you. Whenever you make a statement from your conclusion, ask the friend to say, “So what?” or “Why should anybody care?” Then ponder that question and answer it. Here’s how it might go: You: Basically, I’m just saying that education was important to Douglass. Friend: So what? You: Well, it was important because it was a key to him feeling like a free and equal citizen. Friend: Why should anybody care? You: That’s important because plantation owners tried to keep slaves from being educated so that they could maintain control. When Douglass obtained an education, he undermined that control personally. You can also use this strategy on your own, asking yourself “So What?” as you develop your ideas or your draft.
• Return to the theme or themes in the introduction. This strategy brings the reader full circle. For example, if you begin by describing a scenario, you can end with the same scenario as proof that your essay is helpful in creating a new understanding. You may also refer to the introductory paragraph by using key words or parallel concepts and images that you also used in the introduction.
• Synthesize, don’t summarize. Include a brief summary of the paper’s main points, but don’t simply repeat things that were in your paper. Instead, show your reader how the points you made and the support and examples you used fit together. Pull it all together.
• Include a provocative insight or quotation from the research or reading you did for your paper.
• Propose a course of action, a solution to an issue, or questions for further study. This can redirect your reader’s thought process and help them to apply your info and ideas to their own life or to see the broader implications.
• Point to broader implications. For example, if your paper examines the Greensboro sit-ins or another event in the Civil Rights Movement, you could point out its impact on the Civil Rights Movement as a whole. A paper about the style of writer Virginia Woolf could point to her influence on other writers or on later feminists.

Strategies to avoid

• Beginning with an unnecessary, overused phrase such as “in conclusion,” “in summary,” or “in closing.” Although these phrases can work in speeches, they come across as wooden and trite in writing.
• Stating the thesis for the very first time in the conclusion.
• Introducing a new idea or subtopic in your conclusion.
• Ending with a rephrased thesis statement without any substantive changes.
• Making sentimental, emotional appeals that are out of character with the rest of an analytical paper.
• Including evidence (quotations, statistics, etc.) that should be in the body of the paper.

Four kinds of ineffective conclusions

• The “That’s My Story and I’m Sticking to It” Conclusion. This conclusion just restates the thesis and is usually painfully short. It does not push the ideas forward. People write this kind of conclusion when they can’t think of anything else to say. Example: In conclusion, Frederick Douglass was, as we have seen, a pioneer in American education, proving that education was a major force for social change with regard to slavery.
• The “Sherlock Holmes” Conclusion. Sometimes writers will state the thesis for the very first time in the conclusion. You might be tempted to use this strategy if you don’t want to give everything away too early in your paper. You may think it would be more dramatic to keep the reader in the dark until the end and then “wow” them with your main idea, as in a Sherlock Holmes mystery. The reader, however, does not expect a mystery, but an analytical discussion of your topic in an academic style, with the main argument (thesis) stated up front. Example: (After a paper that lists numerous incidents from the book but never says what these incidents reveal about Douglass and his views on education): So, as the evidence above demonstrates, Douglass saw education as a way to undermine the slaveholders’ power and also an important step toward freedom.
• The “America the Beautiful”/”I Am Woman”/”We Shall Overcome” Conclusion. This kind of conclusion usually draws on emotion to make its appeal, but while this emotion and even sentimentality may be very heartfelt, it is usually out of character with the rest of an analytical paper. A more sophisticated commentary, rather than emotional praise, would be a more fitting tribute to the topic. Example: Because of the efforts of fine Americans like Frederick Douglass, countless others have seen the shining beacon of light that is education. His example was a torch that lit the way for others. Frederick Douglass was truly an American hero.
• The “Grab Bag” Conclusion. This kind of conclusion includes extra information that the writer found or thought of but couldn’t integrate into the main paper. You may find it hard to leave out details that you discovered after hours of research and thought, but adding random facts and bits of evidence at the end of an otherwise-well-organized essay can just create confusion. Example: In addition to being an educational pioneer, Frederick Douglass provides an interesting case study for masculinity in the American South. He also offers historians an interesting glimpse into slave resistance when he confronts Covey, the overseer. His relationships with female relatives reveal the importance of family in the slave community.

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Douglass, Frederick. 1995. Narrative of the Life of Frederick Douglass, an American Slave, Written by Himself. New York: Dover.

Hamilton College. n.d. “Conclusions.” Writing Center. Accessed June 14, 2019. https://www.hamilton.edu//academics/centers/writing/writing-resources/conclusions .

Holewa, Randa. 2004. “Strategies for Writing a Conclusion.” LEO: Literacy Education Online. Last updated February 19, 2004. https://leo.stcloudstate.edu/acadwrite/conclude.html.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• How to Write Discussions and Conclusions

The discussion section contains the results and outcomes of a study. An effective discussion informs readers what can be learned from your experiment and provides context for the results.

What makes an effective discussion?

When you’re ready to write your discussion, you’ve already introduced the purpose of your study and provided an in-depth description of the methodology. The discussion informs readers about the larger implications of your study based on the results. Highlighting these implications while not overstating the findings can be challenging, especially when you’re submitting to a journal that selects articles based on novelty or potential impact. Regardless of what journal you are submitting to, the discussion section always serves the same purpose: concluding what your study results actually mean.

A successful discussion section puts your findings in context. It should include:

• the results of your research,
• a discussion of related research, and
• a comparison between your results and initial hypothesis.

Tip: Not all journals share the same naming conventions.

You can apply the advice in this article to the conclusion, results or discussion sections of your manuscript.

Our Early Career Researcher community tells us that the conclusion is often considered the most difficult aspect of a manuscript to write. To help, this guide provides questions to ask yourself, a basic structure to model your discussion off of and examples from published manuscripts. 

Questions to ask yourself:

• Was my hypothesis correct?
• If my hypothesis is partially correct or entirely different, what can be learned from the results? 
• How do the conclusions reshape or add onto the existing knowledge in the field? What does previous research say about the topic? 
• Why are the results important or relevant to your audience? Do they add further evidence to a scientific consensus or disprove prior studies? 
• How can future research build on these observations? What are the key experiments that must be done? 
• What is the “take-home” message you want your reader to leave with?

How to structure a discussion

Trying to fit a complete discussion into a single paragraph can add unnecessary stress to the writing process. If possible, you’ll want to give yourself two or three paragraphs to give the reader a comprehensive understanding of your study as a whole. Here’s one way to structure an effective discussion:

Writing Tips

While the above sections can help you brainstorm and structure your discussion, there are many common mistakes that writers revert to when having difficulties with their paper. Writing a discussion can be a delicate balance between summarizing your results, providing proper context for your research and avoiding introducing new information. Remember that your paper should be both confident and honest about the results! 

• Read the journal’s guidelines on the discussion and conclusion sections. If possible, learn about the guidelines before writing the discussion to ensure you’re writing to meet their expectations. 
• Begin with a clear statement of the principal findings. This will reinforce the main take-away for the reader and set up the rest of the discussion. 
• Explain why the outcomes of your study are important to the reader. Discuss the implications of your findings realistically based on previous literature, highlighting both the strengths and limitations of the research. 
• State whether the results prove or disprove your hypothesis. If your hypothesis was disproved, what might be the reasons? 
• Introduce new or expanded ways to think about the research question. Indicate what next steps can be taken to further pursue any unresolved questions. 
• If dealing with a contemporary or ongoing problem, such as climate change, discuss possible consequences if the problem is avoided. 
• Be concise. Adding unnecessary detail can distract from the main findings. 

Don’t

• Rewrite your abstract. Statements with “we investigated” or “we studied” generally do not belong in the discussion. 
• Include new arguments or evidence not previously discussed. Necessary information and evidence should be introduced in the main body of the paper. 
• Apologize. Even if your research contains significant limitations, don’t undermine your authority by including statements that doubt your methodology or execution. 
• Shy away from speaking on limitations or negative results. Including limitations and negative results will give readers a complete understanding of the presented research. Potential limitations include sources of potential bias, threats to internal or external validity, barriers to implementing an intervention and other issues inherent to the study design. 
• Overstate the importance of your findings. Making grand statements about how a study will fully resolve large questions can lead readers to doubt the success of the research. 

Snippets of Effective Discussions:

Consumer-based actions to reduce plastic pollution in rivers: A multi-criteria decision analysis approach

Identifying reliable indicators of fitness in polar bears

• How to Write a Great Title
• How to Write an Abstract
• How to Write Your Methods
• How to Report Statistics
• How to Edit Your Work

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

In a short paper—even a research paper—you don’t need to provide an exhaustive summary as part of your conclusion. But you do need to make some kind of transition between your final body paragraph and your concluding paragraph. This may come in the form of a few sentences of summary. Or it may come in the form of a sentence that brings your readers back to your thesis or main idea and reminds your readers where you began and how far you have traveled.

So, for example, in a paper about the relationship between ADHD and rejection sensitivity, Vanessa Roser begins by introducing readers to the fact that researchers have studied the relationship between the two conditions and then provides her explanation of that relationship. Here’s her thesis: “While socialization may indeed be an important factor in RS, I argue that individuals with ADHD may also possess a neurological predisposition to RS that is exacerbated by the differing executive and emotional regulation characteristic of ADHD.”

In her final paragraph, Roser reminds us of where she started by echoing her thesis: “This literature demonstrates that, as with many other conditions, ADHD and RS share a delicately intertwined pattern of neurological similarities that is rooted in the innate biology of an individual’s mind, a connection that cannot be explained in full by the behavioral mediation hypothesis.”  

Highlight the “so what”  

At the beginning of your paper, you explain to your readers what’s at stake—why they should care about the argument you’re making. In your conclusion, you can bring readers back to those stakes by reminding them why your argument is important in the first place. You can also draft a few sentences that put those stakes into a new or broader context.

In the conclusion to her paper about ADHD and RS, Roser echoes the stakes she established in her introduction—that research into connections between ADHD and RS has led to contradictory results, raising questions about the “behavioral mediation hypothesis.”

She writes, “as with many other conditions, ADHD and RS share a delicately intertwined pattern of neurological similarities that is rooted in the innate biology of an individual’s mind, a connection that cannot be explained in full by the behavioral mediation hypothesis.”  

Leave your readers with the “now what”  

After the “what” and the “so what,” you should leave your reader with some final thoughts. If you have written a strong introduction, your readers will know why you have been arguing what you have been arguing—and why they should care. And if you’ve made a good case for your thesis, then your readers should be in a position to see things in a new way, understand new questions, or be ready for something that they weren’t ready for before they read your paper.

In her conclusion, Roser offers two “now what” statements. First, she explains that it is important to recognize that the flawed behavioral mediation hypothesis “seems to place a degree of fault on the individual. It implies that individuals with ADHD must have elicited such frequent or intense rejection by virtue of their inadequate social skills, erasing the possibility that they may simply possess a natural sensitivity to emotion.” She then highlights the broader implications for treatment of people with ADHD, noting that recognizing the actual connection between rejection sensitivity and ADHD “has profound implications for understanding how individuals with ADHD might best be treated in educational settings, by counselors, family, peers, or even society as a whole.”

To find your own “now what” for your essay’s conclusion, try asking yourself these questions:

• What can my readers now understand, see in a new light, or grapple with that they would not have understood in the same way before reading my paper? Are we a step closer to understanding a larger phenomenon or to understanding why what was at stake is so important?  
• What questions can I now raise that would not have made sense at the beginning of my paper? Questions for further research? Other ways that this topic could be approached?  
• Are there other applications for my research? Could my questions be asked about different data in a different context? Could I use my methods to answer a different question?  
• What action should be taken in light of this argument? What action do I predict will be taken or could lead to a solution?  
• What larger context might my argument be a part of?  

What to avoid in your conclusion  

• a complete restatement of all that you have said in your paper.  
• a substantial counterargument that you do not have space to refute; you should introduce counterarguments before your conclusion.  
• an apology for what you have not said. If you need to explain the scope of your paper, you should do this sooner—but don’t apologize for what you have not discussed in your paper.  
• fake transitions like “in conclusion” that are followed by sentences that aren’t actually conclusions. (“In conclusion, I have now demonstrated that my thesis is correct.”)
• picture_as_pdf Conclusions

How to Write Conclusion in Research Paper (With Example)

Writing a strong conclusion is a crucial part of any research paper. It provides a final opportunity to summarize your key findings, restate your thesis, and leave a lasting impression on your reader. However, many students struggle with how to effectively write a conclusion that ties everything together.

In this article, we’ll provide some tips and strategies for writing a compelling conclusion, along with an example to help illustrate the process. By following these guidelines, you can ensure that your research paper ends on a high note and leaves a lasting impact on your audience.

Why Conclusion is Important in Research Paper

The conclusion is the final chapter of your research paper journey, sealing the deal on all your hard work. After thoroughly laying out your main points and arguments in the body paragraphs, the conclusion gives you a chance to tie everything together into a neat, cohesive package.

More than just summarizing your key ideas, an effective conclusion shows readers the bigger picture of your research and why it matters. It highlights the significance of your findings , explains how your work contributes to the field, and points to potential future directions stemming from your study.

The conclusion is your last chance to leave a lasting impact and compel readers to seriously consider your perspective. With the right phrasing and tone, you can amplify the power of your work. Choose your words wisely, be persuasive yet diplomatic, and guide readers to walk away feeling satisfied by your reasoning and conclusions.

Approach the conclusion thoughtfully, reflect deeply on the larger meaning of your research, and craft impactful final sentences that linger in the reader’s mind. Wield your conclusion skillfully to make your research paper transformative and memorable. A powerful, thoughtful conclusion inspires action, sparks curiosity, and showcases the valuable insights you bring to the academic conversation.

How to Write Conclusion for a Research Paper

Crafting an effective conclusion in research paper requires thoughtful consideration and deliberate effort. After presenting your findings and analysis, the conclusion allows you to close your work with a flourish.

Begin by briefly summarizing the main points of your paper, provide a quick recap of your thesis, methodology, and key findings without repeating too much details from the body. Use this as an opportunity to reinforce your main argument and position within the field.

Next, highlight the significance and implications of your research. What new insights or perspectives does your work contribute? Discuss how your findings can inform future studies or practical applications. Convey why your research matters and how it moves the needle forward in your discipline.

Address any limitations of the current study and propose potential next steps that could be taken by you or other scholars to further the research. This shows readers you have critically considered ways to continue expanding knowledge in this area.

Finally, close with a memorable statement that captures the essence of your work and leaves a lasting impression. This could be an apt metaphor, a call to action, or a thought provoking question for readers to ponder. Choose words that will resonate with your audience and demonstrate the impact of your research.

With care and creativity, your conclusion can elevate your paper and cement your scholarly authority. Revisit often as you write to ensure your conclusion accomplishes its purpose, to convince readers of the value of your study and ignite further progress in your field.

What Not to Include in a Research Paper Conclusion

1. New Data: In a research paper conclusion, avoid presenting new data or evidence that wasn’t discussed earlier in the paper. It’s the time to summarize, analyze, or explain the significance of data already provided, not to introduce new material.

2. Irrelevant Details: The conclusion is not the spot for extraneous details not directly related to your research or its findings. Be focused and concise, tying up the paper neatly without going off-target.

3. Personal Opinions: Try not to include personal beliefs or subjective opinions unless your paper calls for it. Stick to empirical evidence, facts, and objective interpretation of your research.

4. Vague Summarization: While summarizing is the essence of a conclusion, too much of a broad or vague narrative should be avoided. Your conclusion shouldn’t be a generalization of the research but should specifically state your significant findings and their implications.

5. Overstating Results: No matter how exhilarating your research may be, don’t exaggerate its implications or general applications. Remember to acknowledge limitations or potential areas for future exploration.

6. Procrastinating: Refrain from leaving unresolved issues for future research. The conclusion is meant to tie up loose ends, not create more.

7. Repetition: While some reiteration is necessary, completely repeating the same phrases and points made previously can make your conclusion sound boring and redundant. Instead, try to look at your argument from a fresh, summarized perspective.

8. Apologies: Do not apologize or discredit your research efforts. Avoid phrases like, “This research was only” or “Although the study wasn’t able to prove”. A conclusion should confidently present your research results even if they’re unexpected or differ from your hypothesis.

9. Impractical Recommendations: While it’s often good to suggest directions for future research, don’t go overboard by proposing impractical or unachievable goals. Keep your recommendations relevant to your findings and within the realm of possibility.

10. Too Much Jargon: While it’s appropriate to use technical language throughout your research paper, remember the conclusion might be what a layman reads. Stick with a happy medium of professional lingo intermixed with understandable, plain language.

Also Check:   Conclusion for Internship Report

Conclusion in research Example

Research: Impact of Social Media Use on Adolescent Mental Health.

In conclusion, this study has demonstrated the significant impact of social media use on adolescent mental health. Our findings indicate that frequent social media use is associated with higher levels of anxiety and depression, particularly among girls. These results underscore the need for continued research in this area, as well as the development of interventions and strategies to promote healthy social media use among young people. By addressing this issue, we can help to ensure the well-being and success of the next generation.

Conclusion in Research Paper Example

Research: Impact of climate change on coral reefs in Florida.

In conclusion, the effect of climate change on Florida’s coral reefs presents a significant concern for the state’s ecosystem and economy. The data collected during this investigation reveal a direct correlation between rising ocean temperatures and coral bleaching events. This pattern has increased over the past decade, indicating that coral reefs’ health directly correlates with climate change effects.

Example Conclusion in Research

Research: The Influence of Social Media on Consumer Buying Behavior

Social media significantly shapes consumer buying behavior. Its power to influence is seen through peer opinions, online advertising, and brand communication. However, with the potential for misinformation, the reliability and quality of information are areas for further study. Despite these concerns, businesses leveraging social media can effectively boost their market reach and sales.

Research Paper Conclusion

Research: Impacts of Remote Work on Employee Productivity

Remote work has been found to notably enhance employee productivity. The elimination of commuting time, flexible scheduling, and comforting environment contribute to this increase. However, factors like home distractions and technological difficulties offer room for further research. Yet, integrating remote work can be a strategic pathway towards improved efficiency and workforce satisfaction.

These examples demonstrate techniques for crafting an effective conclusion in a research paper, providing your thesis with a powerful final statement. Now it is your turn to compose a strong concluding paragraph that summarizes your findings, reinforces your central argument, and leaves readers with a memorable takeaway.

Remember to restate your thesis without repeating it verbatim, highlight your main points without introducing new evidence, and end on a note that conveys the significance of your research. With a clear structure and purpose, proper grammar, and impactful writing, you can give your paper the persuasive conclusion it deserves.

Writing an effective conclusion takes practice, but by honing these skills you will elevate your academic writing to new heights. Use the strategies outlined here as a guide, believe in your capabilities, and soon you will be adept at concluding research papers powerfully. The final paragraph is your last chance to impress readers, so make it count!

You May Also Like

Conclusion of earthquake example, conclusion of yoga and meditation, conclusion of social media, conclusion of waste management, conclusion for assignment, conclusion of artificial intelligence | how to write | with example, leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

How to Write a Conclusion for a Research Paper

By the time you write the conclusion, you should have pointed out in the body of your research paper why your topic is important to the reader, and you should have presented the reader with all your arguments. It is critical that you do not introduce new information or ideas in your conclusion. If you find that you have not yet made the arguments you wished to make or pointed out evidence you feel is crucial to your reader’s understanding of your subject, you are not yet ready to write the conclusion; add another body paragraph before writing the conclusion.

Academic Writing, Editing, Proofreading, And Problem Solving Services

Get 10% off with 24start discount code.

Your research paper should have a strong, succinct concluding section, where you draw together your findings. Think of it as a conclusion, not a summary. The difference is that you are reaching overall judgments about your topic, not summarizing everything you wrote about it. How to write a conclusion for a research paper? The focus should be on:

• Saying what your research has found, what the findings mean, and how well they support the argument of your thesis statement.
• Establishing the limits of your argument: How widely does it apply? What are the strengths and weaknesses of your method? How clear-cut are your findings?
• Explaining how your findings and argument fit into your field, relating them to answers others have given and to the existing literature.

You may also want to add some concise comments about possible future developments or what kind of research should come next, but don’t lay it on too thick. The place of honor goes to your own explanation. Don’t spend too much of your final section criticizing others. Don’t introduce any big new topics or ideas. You certainly don’t expect to see new characters in the last scene of a movie. For the same reasons, you shouldn’t find any big new topics being introduced in the last paragraphs of a research paper.

Your concluding statement should focus on what your findings mean. How do you interpret them? Are they just as easily explained by alternative theories or other perspectives? Here, you are returning to the questions that first animated you and answering them, based on your research. You not only want to give the answers; you also want to explain their significance. What do they mean for policy, theory, literary interpretation, moral action, or whatever? You are answering the old, hard question: “So what?”

Be wary of overreaching. You really need to do two things at the same time: explain the significance of your findings and stake out their limits. You may have a hunch that your findings apply widely but, as a social scientist, you need to assess whether you can say so confidently, based on your current research. Your reader needs to know: “Do these findings apply to all college students, to all adults, or only to white mice?” White mice don’t come up much in the humanities, but the reader still wants to know how far your approach reaches. Does your analysis apply only to this novel or this writer, or could it apply to a whole literary genre?

Make it a priority to discuss these conclusions with your professor or adviser. The main danger here is that students finally reach this final section with only a week or two left before the due date. They don’t have enough time to work through their conclusions and revise them. That leaves the research paper weakest at the end, precisely where it should be strongest, nailing down the most significant points.

Begin discussing your major findings with your adviser while you are still writing the heart of the research paper. Of course, your conclusions will be tentative at that stage, but it helps to begin talking about them. As always, a little writing helps. You could simply list your main findings or write out a few paragraphs about them. Either would serve as a launching pad for meetings with your adviser. You will find these discussions also shed light on the research that leads to these findings. That, in turn, will strengthen your middle sections. Later, when you draft the conclusion, review your notes on these talks and the short documents you wrote for them. They will serve as prewriting for the final section.

The opening sentence of the conclusion should flow smoothly and logically from the transition sentence in the previous paragraph and lead the reader to reflect on your thesis. A good conclusion however, does not simply restate the thesis. You want to remind the reader of the thesis in your conclusion but reword it in a stronger fashion so that it is interesting and memorable to your audience.After reminding the reader of the thesis, the conclusion should then reflect on the topics in the body of the paper and summarize the key findings of your research. If you are writing a persuasive paper, it should summarize your key arguments and logically point your readers to the conclusion you wish them to reach.

Phrases for Conclusions of Research Papers

• All this requires us to (propose the next action or an alternative idea).
• Altogether, these findings indicate (point out the logical result).
• Finally, it is important to note (make your strongest point and follow with a recommendation).
• In conclusion (restate your thesis with greater emphasis).
• It is evident that (point out the logical result or obvious next action).
• In light of the evidence, (restate your thesis with greater emphasis).
• In short, (summarize your findings).
• It should be evident that we need to (propose the next action or an alternative idea).
• In summary, (summarize your findings).
• Looking ahead, it is obvious that (propose the next action or an alternative idea).
• My conclusion is (restate your thesis with greater emphasis).
• One last word must be said. (Follow with your opinion and propose a next action.)
• One concludes that (give your opinion).
• Overall, (summarize your findings).
• Reflecting on these facts,we can see that …
• The evidence presented above shows that (give your opinion).
• The reader can conclude (make the point you wish to make).
• These facts and observations support the idea that (offer a theory).
• This analysis reveals (state your findings).
• To conclude, (give an opinion based on the findings presented in the paper).
• To sum up this discussion, (summarize your findings).
• To summarize, (summarize your findings).
• We arrive at the following conclusion: (give an opinion based on the findings presented in the paper).
• We cannot ignore the fact that (state an important concern and follow with a call to action).
• We can postulate (give your opinion or offer a theory).
• We come to the conclusion that (give your opinion or offer a theory).
• We can now present the theory that (give your opinion or offer a theory).

Examples of Strong Conclusions

As an example of how to end your research paper, let’s turn again to John Dower’s splendid book on postwar Japan, Embracing Defeat: Japan in the Wake of World War II . In the final pages, Dower pulls together his findings on war-ravaged Japan and its efforts to rebuild. He then judges the legacies of that period: its continuing impact on the country’s social, political, and economic life. Some insights are unexpected, at least to me. He argues that Japan has pursued trade protection as the only acceptable avenue for its persistent nationalism. America’s overwhelming power and Japan’s self-imposed restraints—the intertwined subjects of the book—blocked any political or military expression of Japan’s nationalist sentiment. Those avenues were simply too dangerous, he says, while economic nationalism was not. Dower ends with these paragraphs:

The Japanese economists and bureaucrats who drafted the informal 1946 blueprint for a planned economy were admirably clear on these objectives [of “demilitarization and democratization”]. They sought rapid recovery and maximum economic growth, of course—but they were just as concerned with achieving economic demilitarization and economic democracy. . . . Japan became wealthy. The standard of living rose impressively at every level of society. Income distribution was far more equitable than in the United States. Job security was assured. Growth was achieved without inordinate dependence on a military-industrial complex or a thriving trade in armaments. These are hardly trivial ideas, but they are now being discarded along with all the deservedly bankrupt aspects of the postwar system. The lessons and legacies of defeat have been many and varied indeed; and their end is not yet in sight. (John W. Dower, Embracing Defeat: Japan in the Wake of World War II . New York: W. W. Norton, 1999, pp. 563–64)

Remember the anecdotal opening of Herbert’s book Impressionism: Art, Leisure, and Parisian Society , with Henry Tuckerman’s 1867 arrival in a much-changed Paris? (see research paper introduction examples) Herbert strikes a completely different tone in his conclusion. It synthesizes the art history he has presented, offers a large judgment about where Impressionism fits among art movements, and suggests why exhibitions of Monet, Manet, and Renoir are still so popular. He manages to do all that in a few well-crafted sentences:

Although we credit [Impressionism] with being the gateway to modern art, we also treat it as the last of the great Western styles based upon a perception of harmony with natural vision. That harmony, long since lost to us in this century of urbanization, industrialization, and world wars, remains a longed-for idea, so we look back to Impressionism as the painting of a golden era. We flock into exhibitions of paintings that represent cafes, boating, promenading, and peaceful landscapes precisely because of our yearning for less troubled times. The only history that we feel deeply is the kind that is useful to us. Impressionism still looms large at the end of the twentieth century because we use its leisure-time subjects and its brilliantly colored surfaces to construct a desirable history. (Herbert, Impressionism , p. 306)

Robert Dallek offers similarly accessible, powerful judgments in his conclusion to Flawed Giant: Lyndon Johnson and His Times, 1961–1973 :

[Johnson’s] presidency was a story of great achievement and terrible failure, of lasting gains and unforgettable losses. . . . In a not so distant future, when coming generations have no direct experience of the man and the passions of the sixties are muted, Johnson will probably be remembered as a President who faithfully reflected the country’s greatness and limitations—a man notable for his successes and failures, for his triumphs and tragedy. Only one thing seems certain: Lyndon Johnson will not join the many obscure—almost nameless, faceless—Presidents whose terms of office register on most Americans as blank slates. He will not be forgotten. (Robert Dallek, Flawed Giant: Lyndon Johnson and His Times, 1961–1973 . New York: Oxford University Press, 1998, p. 628)

Some writers not only synthesize their findings or compare them to others; they use the conclusion to say what their work means for appropriate methods or subject matter in their field. That is what Robert Bruegmann does in his final statement in The Architects and the City: Holabird & Roche of Chicago, 1880–1918 . His conclusion goes beyond saying that this was a great architectural firm or that it designed buildings of lasting importance. Bruegmann tells us that Holabird & Roche helped shape modern Chicago and that its work, properly studied, helps us understand “the city as the ultimate human artifact”:

Traditional architectural history has tended to see the city less as a process than as a product, a collection of high art architectural objects in a setting dominated by mundane buildings of little interest. This tended to perpetuate a destructive and divisive attitude about the built environment, suggesting that only a few buildings are worthy of careful study and preservation while all others are mere backdrop. I hope that these explorations in the work of Holabird & Roche have shed light on parts of the city rarely visited by the architectural historian and on some little explored aspects of its history. If so, perhaps it has achieved its most basic goal: providing an insight into the city as the ultimate human artifact, our most complex and prodigious social creation, and the most tangible result of the actions over time of all its citizens. (Robert Bruegmann, The Architects and the City: Holabird & Roche of Chicago, 1880–1918 . Chicago: University of Chicago Press, 1997, p. 443)

These are powerful conclusions, ending major works of scholarship on a high note. What concluding paragraphs should never do is gaze off into the sunset, offer vague homilies, or claim you have found the meaning of human existence. Be concrete. Stick to your topic. Make sure your research paper conclusions stand on solid ground. Avoid vague platitudes in your conclusion. Your goal should be reaching strong, sound judgments, firmly grounded in your readings and research. Better to claim too little than too much. Best of all, claim what you’ve earned the right to say: what your research really means.

Having finished the main parts of a research paper you can write an abstract.

Back to  How To Write A Research Paper .

ORDER HIGH QUALITY CUSTOM PAPER

• Walden University
• Faculty Portal

Writing a Paper: Conclusions

Writing a conclusion.

A conclusion is an important part of the paper; it provides closure for the reader while reminding the reader of the contents and importance of the paper. It accomplishes this by stepping back from the specifics in order to view the bigger picture of the document. In other words, it is reminding the reader of the main argument. For most course papers, it is usually one paragraph that simply and succinctly restates the main ideas and arguments, pulling everything together to help clarify the thesis of the paper. A conclusion does not introduce new ideas; instead, it should clarify the intent and importance of the paper. It can also suggest possible future research on the topic.

An Easy Checklist for Writing a Conclusion

It is important to remind the reader of the thesis of the paper so he is reminded of the argument and solutions you proposed.

Think of the main points as puzzle pieces, and the conclusion is where they all fit together to create a bigger picture. The reader should walk away with the bigger picture in mind.

Make sure that the paper places its findings in the context of real social change.

Make sure the reader has a distinct sense that the paper has come to an end. It is important to not leave the reader hanging. (You don’t want her to have flip-the-page syndrome, where the reader turns the page, expecting the paper to continue. The paper should naturally come to an end.)

No new ideas should be introduced in the conclusion. It is simply a review of the material that is already present in the paper. The only new idea would be the suggesting of a direction for future research.

Conclusion Example

As addressed in my analysis of recent research, the advantages of a later starting time for high school students significantly outweigh the disadvantages. A later starting time would allow teens more time to sleep--something that is important for their physical and mental health--and ultimately improve their academic performance and behavior. The added transportation costs that result from this change can be absorbed through energy savings. The beneficial effects on the students’ academic performance and behavior validate this decision, but its effect on student motivation is still unknown. I would encourage an in-depth look at the reactions of students to such a change. This sort of study would help determine the actual effects of a later start time on the time management and sleep habits of students.

Related Webinar

Didn't find what you need? Email us at [email protected] .

• Previous Page: Thesis Statements
• Next Page: Writer's Block
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications
• College University and Postgraduate
• Academic Writing
• Research Papers

How to Write a Conclusion for a Research Paper

Last Updated: May 8, 2024 Approved

This article was co-authored by Christopher Taylor, PhD . Christopher Taylor is an Adjunct Assistant Professor of English at Austin Community College in Texas. He received his PhD in English Literature and Medieval Studies from the University of Texas at Austin in 2014. wikiHow marks an article as reader-approved once it receives enough positive feedback. This article received 43 testimonials and 83% of readers who voted found it helpful, earning it our reader-approved status. This article has been viewed 2,261,383 times.

The conclusion of a research paper needs to summarize the content and purpose of the paper without seeming too wooden or dry. Every basic conclusion must share several key elements, but there are also several tactics you can play around with to craft a more effective conclusion and several you should avoid to prevent yourself from weakening your paper's conclusion. Here are some writing tips to keep in mind when creating a conclusion for your next research paper.

Sample Conclusions

Writing a basic conclusion.

• Do not spend a great amount of time or space restating your topic.
• A good research paper will make the importance of your topic apparent, so you do not need to write an elaborate defense of your topic in the conclusion.
• Usually a single sentence is all you need to restate your topic.
• An example would be if you were writing a paper on the epidemiology of infectious disease, you might say something like "Tuberculosis is a widespread infectious disease that affects millions of people worldwide every year."
• Yet another example from the humanities would be a paper about the Italian Renaissance: "The Italian Renaissance was an explosion of art and ideas centered around artists, writers, and thinkers in Florence."

• A thesis is a narrowed, focused view on the topic at hand.
• This statement should be rephrased from the thesis you included in your introduction. It should not be identical or too similar to the sentence you originally used.
• Try re-wording your thesis statement in a way that complements your summary of the topic of your paper in your first sentence of your conclusion.
• An example of a good thesis statement, going back to the paper on tuberculosis, would be "Tuberculosis is a widespread disease that affects millions of people worldwide every year. Due to the alarming rate of the spread of tuberculosis, particularly in poor countries, medical professionals are implementing new strategies for the diagnosis, treatment, and containment of this disease ."

• A good way to go about this is to re-read the topic sentence of each major paragraph or section in the body of your paper.
• Find a way to briefly restate each point mentioned in each topic sentence in your conclusion. Do not repeat any of the supporting details used within your body paragraphs.
• Under most circumstances, you should avoid writing new information in your conclusion. This is especially true if the information is vital to the argument or research presented in your paper.
• For example, in the TB paper you could summarize the information. "Tuberculosis is a widespread disease that affects millions of people worldwide. Due to the alarming rate of the spread of tuberculosis, particularly in poor countries, medical professionals are implementing new strategies for the diagnosis, treatment, and containment of this disease. In developing countries, such as those in Africa and Southeast Asia, the rate of TB infections is soaring. Crowded conditions, poor sanitation, and lack of access to medical care are all compounding factors in the spread of the disease. Medical experts, such as those from the World Health Organization are now starting campaigns to go into communities in developing countries and provide diagnostic testing and treatments. However, the treatments for TB are very harsh and have many side effects. This leads to patient non-compliance and spread of multi-drug resistant strains of the disease."

• Note that this is not needed for all research papers.
• If you already fully explained what the points in your paper mean or why they are significant, you do not need to go into them in much detail in your conclusion. Simply restating your thesis or the significance of your topic should suffice.
• It is always best practice to address important issues and fully explain your points in the body of your paper. The point of a conclusion to a research paper is to summarize your argument for the reader and, perhaps, to call the reader to action if needed.

• Note that a call for action is not essential to all conclusions. A research paper on literary criticism, for instance, is less likely to need a call for action than a paper on the effect that television has on toddlers and young children.
• A paper that is more likely to call readers to action is one that addresses a public or scientific need. Let's go back to our example of tuberculosis. This is a very serious disease that is spreading quickly and with antibiotic-resistant forms.
• A call to action in this research paper would be a follow-up statement that might be along the lines of "Despite new efforts to diagnose and contain the disease, more research is needed to develop new antibiotics that will treat the most resistant strains of tuberculosis and ease the side effects of current treatments."

• For example, if you are writing a history paper, then you might discuss how the historical topic you discussed matters today. If you are writing about a foreign country, then you might use the conclusion to discuss how the information you shared may help readers understand their own country.

Making Your Conclusion as Effective as Possible

• Since this sort of conclusion is so basic, you must aim to synthesize the information rather than merely summarizing it.
• Instead of merely repeating things you already said, rephrase your thesis and supporting points in a way that ties them all together.
• By doing so, you make your research paper seem like a "complete thought" rather than a collection of random and vaguely related ideas.

• Ask a question in your introduction. In your conclusion, restate the question and provide a direct answer.
• Write an anecdote or story in your introduction but do not share the ending. Instead, write the conclusion to the anecdote in the conclusion of your paper.
• For example, if you wanted to get more creative and put a more humanistic spin on a paper on tuberculosis, you might start your introduction with a story about a person with the disease, and refer to that story in your conclusion. For example, you could say something like this before you re-state your thesis in your conclusion: "Patient X was unable to complete the treatment for tuberculosis due to severe side effects and unfortunately succumbed to the disease."
• Use the same concepts and images introduced in your introduction in your conclusion. The images may or may not appear at other points throughout the research paper.

• Include enough information about your topic to back the statement up but do not get too carried away with excess detail.
• If your research did not provide you with a clear-cut answer to a question posed in your thesis, do not be afraid to indicate as much.
• Restate your initial hypothesis and indicate whether you still believe it or if the research you performed has begun swaying your opinion.
• Indicate that an answer may still exist and that further research could shed more light on the topic at hand.

• This may not be appropriate for all types of research papers. Most research papers, such as one on effective treatment for diseases, will have the information to make the case for a particular argument already in the paper.
• A good example of a paper that might ask a question of the reader in the ending is one about a social issue, such as poverty or government policy.
• Ask a question that will directly get at the heart or purpose of the paper. This question is often the same question, or some version of it, that you may have started with when you began your research.
• Make sure that the question can be answered by the evidence presented in your paper.
• If desired you can briefly summarize the answer after stating the question. You could also leave the question hanging for the reader to answer, though.

• Even without a call to action, you can still make a recommendation to your reader.
• For instance, if you are writing about a topic like third-world poverty, you can various ways for the reader to assist in the problem without necessarily calling for more research.
• Another example would be, in a paper about treatment for drug-resistant tuberculosis, you could suggest donating to the World Health Organization or research foundations that are developing new treatments for the disease.

Avoiding Common Pitfalls

• These sayings usually sound stiff, unnatural, or trite when used in writing.
• Moreover, using a phrase like "in conclusion" to begin your conclusion is a little too straightforward and tends to lead to a weak conclusion. A strong conclusion can stand on its own without being labeled as such.

• Always state the main argument or thesis in the introduction. A research paper is an analytical discussion of an academic topic, not a mystery novel.
• A good, effective research paper will allow your reader to follow your main argument from start to finish.
• This is why it is best practice to start your paper with an introduction that states your main argument and to end the paper with a conclusion that re-states your thesis for re-iteration.

• All significant information should be introduced in the body of the paper.
• Supporting evidence expands the topic of your paper by making it appear more detailed. A conclusion should narrow the topic to a more general point.
• A conclusion should only summarize what you have already stated in the body of your paper.
• You may suggest further research or a call to action, but you should not bring in any new evidence or facts in the conclusion.

• Most often, a shift in tone occurs when a research paper with an academic tone gives an emotional or sentimental conclusion.
• Even if the topic of the paper is of personal significance for you, you should not indicate as much in your paper.
• If you want to give your paper a more humanistic slant, you could start and end your paper with a story or anecdote that would give your topic more personal meaning to the reader.
• This tone should be consistent throughout the paper, however.

• Apologetic statements include phrases like "I may not be an expert" or "This is only my opinion."
• Statements like this can usually be avoided by refraining from writing in the first-person.
• Avoid any statements in the first-person. First-person is generally considered to be informal and does not fit with the formal tone of a research paper.

Community Q&A

You Might Also Like

• ↑ http://owl.english.purdue.edu/owl/resource/724/04/
• ↑ http://www.crlsresearchguide.org/18_Writing_Conclusion.asp
• ↑ http://writing.wisc.edu/Handbook/PlanResearchPaper.html#conclusion
• ↑ http://writingcenter.unc.edu/handouts/conclusions/
• ↑ http://writing2.richmond.edu/writing/wweb/conclude.html

About This Article

To write a conclusion for a research paper, start by restating your thesis statement to remind your readers what your main topic is and bring everything full circle. Then, briefly summarize all of the main points you made throughout your paper, which will help remind your readers of everything they learned. You might also want to include a call to action if you think more research or work needs to be done on your topic by writing something like, "Despite efforts to contain the disease, more research is needed to develop antibiotics." Finally, end your conclusion by explaining the broader context of your topic and why your readers should care about it, which will help them understand why your topic is relevant and important. For tips from our Academic co-author, like how to avoid common pitfalls when writing your conclusion, scroll down! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Did this article help you?

Ummay Aimen

Sep 30, 2016

Oct 22, 2017

Sally Larrin

Mar 17, 2018

Maya Loeven

Jun 4, 2017

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

wikiHow Tech Help Pro:

Develop the tech skills you need for work and life

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

The research process, explore more of umgc.

• Online Guide to Writing

Planning and Writing a Research Paper

Draw Conclusions

As a writer, you are presenting your viewpoint, opinions, evidence, etc. for others to review, so you must take on this task with maturity, courage and thoughtfulness.  Remember, you are adding to the discourse community with every research paper that you write.  This is a privilege and an opportunity to share your point of view with the world at large in an academic setting.

Because research generates further research, the conclusions you draw from your research are important. As a researcher, you depend on the integrity of the research that precedes your own efforts, and researchers depend on each other to draw valid conclusions. 

To test the validity of your conclusions, you will have to review both the content of your paper and the way in which you arrived at the content. You may ask yourself questions, such as the ones presented below, to detect any weak areas in your paper, so you can then make those areas stronger.  Notice that some of the questions relate to your process, others to your sources, and others to how you arrived at your conclusions.

Checklist for Evaluating Your Conclusions

Key Takeaways

• Because research generates further research, the conclusions you draw from your research are important.
• To test the validity of your conclusions, you will have to review both the content of your paper and the way in which you arrived at the content.

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker
• APA Citation Generator
• MLA Citation Generator
• Chicago Citation Generator
• Vancouver Citation Generator
• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Write a Research Paper Conclusion Section

What is a conclusion in a research paper?

The conclusion in a research paper is the final paragraph or two in a research paper. In scientific papers, the conclusion usually follows the Discussion section , summarizing the importance of the findings and reminding the reader why the work presented in the paper is relevant.

However, it can be a bit confusing to distinguish the conclusion section/paragraph from a summary or a repetition of your findings, your own opinion, or the statement of the implications of your work. In fact, the conclusion should contain a bit of all of these other parts but go beyond it—but not too far beyond! 

The structure and content of the conclusion section can also vary depending on whether you are writing a research manuscript or an essay. This article will explain how to write a good conclusion section, what exactly it should (and should not) contain, how it should be structured, and what you should avoid when writing it.  

Table of Contents:

What does a good conclusion section do, what to include in a research paper conclusion.

• Conclusion in an Essay
• Research Paper Conclusion 
• Conclusion Paragraph Outline and Example
• What Not to Do When Writing a Conclusion

The conclusion of a research paper has several key objectives. It should:

• Restate your research problem addressed in the introduction section
• Summarize your main arguments, important findings, and broader implications
• Synthesize key takeaways from your study

The specific content in the conclusion depends on whether your paper presents the results of original scientific research or constructs an argument through engagement with previously published sources.

You presented your general field of study to the reader in the introduction section, by moving from general information (the background of your work, often combined with a literature review ) to the rationale of your study and then to the specific problem or topic you addressed, formulated in the form of the statement of the problem in research or the thesis statement in an essay.

In the conclusion section, in contrast, your task is to move from your specific findings or arguments back to a more general depiction of how your research contributes to the readers’ understanding of a certain concept or helps solve a practical problem, or fills an important gap in the literature. The content of your conclusion section depends on the type of research you are doing and what type of paper you are writing. But whatever the outcome of your work is, the conclusion is where you briefly summarize it and place it within a larger context. It could be called the “take-home message” of the entire paper.

What to summarize in the conclusion

Your conclusion section needs to contain a very brief summary of your work , a very brief summary of the main findings of your work, and a mention of anything else that seems relevant when you now look at your work from a bigger perspective, even if it was not initially listed as one of your main research questions. This could be a limitation, for example, a problem with the design of your experiment that either needs to be considered when drawing any conclusions or that led you to ask a different question and therefore draw different conclusions at the end of your study (compared to when you started out).

Once you have reminded the reader of what you did and what you found, you need to go beyond that and also provide either your own opinion on why your work is relevant (and for whom, and how) or theoretical or practical implications of the study , or make a specific call for action if there is one to be made.   

How to Write an Essay Conclusion

Academic essays follow quite different structures than their counterparts in STEM and the natural sciences. Humanities papers often have conclusion sections that are much longer and contain more detail than scientific papers. There are three main types of academic essay conclusions.

Summarizing conclusion

The most typical conclusion at the end of an analytical/explanatory/argumentative essay is a summarizing conclusion . This is, as the name suggests, a clear summary of the main points of your topic and thesis. Since you might have gone through a number of different arguments or subtopics in the main part of your essay, you need to remind the reader again what those were, how they fit into each other, and how they helped you develop or corroborate your hypothesis.

For an essay that analyzes how recruiters can hire the best candidates in the shortest time or on “how starving yourself will increase your lifespan, according to science”, a summary of all the points you discussed might be all you need. Note that you should not exactly repeat what you said earlier, but rather highlight the essential details and present those to your reader in a different way. 

Externalizing conclusion

If you think that just reminding the reader of your main points is not enough, you can opt for an externalizing conclusion instead, that presents new points that were not presented in the paper so far. These new points can be additional facts and information or they can be ideas that are relevant to the topic and have not been mentioned before.

Such a conclusion can stimulate your readers to think about your topic or the implications of your analysis in a whole new way. For example, at the end of a historical analysis of a specific event or development, you could direct your reader’s attention to some current events that were not the topic of your essay but that provide a different context for your findings.

Editorial conclusion

In an editorial conclusion , another common type of conclusion that you will find at the end of papers and essays, you do not add new information but instead present your own experiences or opinions on the topic to round everything up. What makes this type of conclusion interesting is that you can choose to agree or disagree with the information you presented in your paper so far. For example, if you have collected and analyzed information on how a specific diet helps people lose weight, you can nevertheless have your doubts on the sustainability of that diet or its practicability in real life—if such arguments were not included in your original thesis and have therefore not been covered in the main part of your paper, the conclusion section is the place where you can get your opinion across.    

How to Conclude an Empirical Research Paper

An empirical research paper is usually more concise and succinct than an essay, because, if it is written well, it focuses on one specific question, describes the method that was used to answer that one question, describes and explains the results, and guides the reader in a logical way from the introduction to the discussion without going on tangents or digging into not absolutely relevant topics.

Summarize the findings

In a scientific paper, you should include a summary of the findings. Don’t go into great detail here (you will have presented your in-depth  results  and  discussion  already), but do clearly express the answers to the  research questions  you investigated.

Describe your main findings, even if they weren’t necessarily the ones anticipated, and explain the conclusion they led you to. Explain these findings in as few words as possible.

Instead of beginning with “ In conclusion, in this study, we investigated the effect of stress on the brain using fMRI …”, you should try to find a way to incorporate the repetition of the essential (and only the essential) details into the summary of the key points. “ The findings of this fMRI study on the effect of stress on the brain suggest that …” or “ While it has been known for a long time that stress has an effect on the brain, the findings of this fMRI study show that, surprisingly… ” would be better ways to start a conclusion. 

You should also not bring up new ideas or present new facts in the conclusion of a research paper, but stick to the background information you have presented earlier, to the findings you have already discussed, and the limitations and implications you have already described. The one thing you can add here is a practical recommendation that you haven’t clearly stated before—but even that one needs to follow logically from everything you have already discussed in the discussion section.

Discuss the implications

After summing up your key arguments or findings, conclude the paper by stating the broader implications of the research , whether in methods , approach, or findings. Express practical or theoretical takeaways from your paper. This often looks like a “call to action” or a final “sales pitch” that puts an exclamation point on your paper.

If your research topic is more theoretical in nature, your closing statement should express the significance of your argument—for example, in proposing a new understanding of a topic or laying the groundwork for future research.

Future research example

Future research into education standards should focus on establishing a more detailed picture of how novel pedagogical approaches impact young people’s ability to absorb new and difficult concepts. Moreover, observational studies are needed to gain more insight into how specific teaching models affect the retention of relationships and facts—for instance, how inquiry-based learning and its emphasis on lateral thinking can be used as a jumping-off point for more holistic classroom approaches.

Research Conclusion Example and Outline

Let’s revisit the study on the effect of stress on the brain we mentioned before and see what the common structure for a conclusion paragraph looks like, in three steps. Following these simple steps will make it easy for you to wrap everything up in one short paragraph that contains all the essential information: 

One: Short summary of what you did, but integrated into the summary of your findings:

While it has been known for a long time that stress has an effect on the brain, the findings of this fMRI study in 25 university students going through mid-term exams show that, surprisingly, one’s attitude to the experienced stress significantly modulates the brain’s response to it. 

Note that you don’t need to repeat any methodological or technical details here—the reader has been presented with all of these before, they have read your results section and the discussion of your results, and even (hopefully!) a discussion of the limitations and strengths of your paper. The only thing you need to remind them of here is the essential outcome of your work. 

Two: Add implications, and don’t forget to specify who this might be relevant for: 

Students could be considered a specific subsample of the general population, but earlier research shows that the effect that exam stress has on their physical and mental health is comparable to the effects of other types of stress on individuals of other ages and occupations. Further research into practical ways of modulating not only one’s mental stress response but potentially also one’s brain activity (e.g., via neurofeedback training) are warranted.

This is a “research implication”, and it is nicely combined with a mention of a potential limitation of the study (the student sample) that turns out not to be a limitation after all (because earlier research suggests we can generalize to other populations). If there already is a lot of research on neurofeedback for stress control, by the way, then this should have been discussed in your discussion section earlier and you wouldn’t say such studies are “warranted” here but rather specify how your findings could inspire specific future experiments or how they should be implemented in existing applications. 

Three: The most important thing is that your conclusion paragraph accurately reflects the content of your paper. Compare it to your research paper title , your research paper abstract , and to your journal submission cover letter , in case you already have one—if these do not all tell the same story, then you need to go back to your paper, start again from the introduction section, and find out where you lost the logical thread. As always, consistency is key.    

Problems to Avoid When Writing a Conclusion 

• Do not suddenly introduce new information that has never been mentioned before (unless you are writing an essay and opting for an externalizing conclusion, see above). The conclusion section is not where you want to surprise your readers, but the take-home message of what you have already presented.
• Do not simply copy your abstract, the conclusion section of your abstract, or the first sentence of your introduction, and put it at the end of the discussion section. Even if these parts of your paper cover the same points, they should not be identical.
• Do not start the conclusion with “In conclusion”. If it has its own section heading, that is redundant, and if it is the last paragraph of the discussion section, it is inelegant and also not really necessary. The reader expects you to wrap your work up in the last paragraph, so you don’t have to announce that. Just look at the above example to see how to start a conclusion in a natural way.
• Do not forget what your research objectives were and how you initially formulated the statement of the problem in your introduction section. If your story/approach/conclusions changed because of methodological issues or information you were not aware of when you started, then make sure you go back to the beginning and adapt your entire story (not just the ending). 

Consider Receiving Academic Editing Services

When you have arrived at the conclusion of your paper, you might want to head over to Wordvice AI’s AI Writing Assistant to receive a free grammar check for any academic content. 

After drafting, you can also receive English editing and proofreading services , including paper editing services for your journal manuscript. If you need advice on how to write the other parts of your research paper , or on how to make a research paper outline if you are struggling with putting everything you did together, then head over to the Wordvice academic resources pages , where we have a lot more articles and videos for you.

Explore Jobs

• Jobs Near Me
• Remote Jobs
• Full Time Jobs
• Part Time Jobs
• Entry Level Jobs
• Work From Home Jobs

Find Specific Jobs

• $15 Per Hour Jobs
• $20 Per Hour Jobs
• Hiring Immediately Jobs
• High School Jobs
• H1b Visa Jobs

Explore Careers

• Business And Financial
• Architecture And Engineering
• Computer And Mathematical

Explore Professions

• What They Do
• Certifications
• Demographics

Best Companies

• Health Care
• Fortune 500

Explore Companies

• CEO And Executies
• Resume Builder
• Career Advice
• Explore Majors
• Questions And Answers
• Interview Questions
• How To Write A Conclusion For A Research Paper
• What Are Skills Employers Look For?
• What Are Inductive Reasoning?
• What Are Problem Solving Skills?
• What Are Active Listening Skills?
• What Are Management Skills?
• What Are Attention To Detail?
• What Are Detail Oriented Skills?
• What Are Domain Knowledge?
• What Is Professionalism?
• What Are Rhetorical Skills?
• What Is Integrity?
• What Are Persuasion Skills?
• How To Start A Conversation
• Team Player
• Visual Learner
• High Income Skills
• The Most Important Professional Skills

Find a Job You Really Want In

Writing a research paper is tedious, and after all that work, you’d think the conclusion would be the easy part. In reality, this is often one of the most difficult sections of a research paper to write, since you have to neatly tie up pages and pages of research in a short amount of time.

To help you with this, we’ve put together some instructions and tips on how to write a research paper conclusion. We’ll also talk about what conclusions are, why they’re important, and different ways you can format them.

Key Takeaways

Research paper conclusions serve to close the argument the introduction opened and restate the main points of the research paper.

There are three research paper conclusion formats: summarization, reflective, and projective.

Your research paper conclusion should be concise, straightforward, and accurate.

How To Write A Research Paper Conclusion

6 tips for writing a research paper conclusion, different formats of research paper conclusions, what is the conclusion of a research paper, why is writing a conclusion important for a research paper, research paper conclusion faq.

• Sign Up For More Advice and Jobs

Many students understand that the conclusion is a crucial part of their research paper, but they don’t know how to go about writing one.

Follow the steps below for how to write a research paper conclusion.

Open With The Research Topic. To begin a conclusion paragraph, use the first sentence to reiterate the comprehensive subject matter that your paper covered. Since this is just a sentence-long retelling of your research topic and why it’s important, it doesn’t have to be specific, but it does need clarity.

Dragonflies are a magnificently complex insect whose advanced physical mechanics and vast species differences make them a notable study in the scientific community.

Focus On Your Specific Thesis. Every research paper focuses on targetted intricacies within a larger topic. Now that the more extensive topic of the research paper has been mentioned, the next sentence or two highlights the specific thesis presented.

Don’t merely copy and paste the introduction of your thesis from the first paragraph. Restate it in different words that illicit a more in-depth understanding from the reader .

The overall characteristics found only within the Odonata family unites the dragonfly under a singular title. All species of dragonfly faced the same path towards the modern structure known today, and therefore, they are all similar in one way or another. However, there are also significant differences apparent to the naked eye between a species that shares so much of the same structure.

Summarize And Connect Main Points. Throughout a research paper, the writer presents points to support the initial thesis claim. Very briefly summarize and tie together these points in a way that supports your thesis. This is the place to restate your research findings.

By examining the striped meadowhawk and migrant hawker dragonflies, it is shown that habitat governs many aspects pertaining to that specific species’ lifestyle. It is also proven that color and patterns perceived on this insect serve a greater purpose of individualizing and distinguishing between these two species.

Bring It All Together. It sounds redundant to say you need to conclude your conclusion, but that’s the final step. You’ve done the mini recap of your research paper through the beginning sentences of your essay. Close the conclusion by making a final encouragement for an action, idea, or fact.

The dragonfly is a unique insect with uniting factors and specialization. However, the most attributed aspect to this insect as a whole is the enormity of their differences. The evolved genetic features attributed to various species of dragonflies both individualize them and apply unification to the insect as a whole.

Consider What Conclusion Format To Use Carefully. The way you structure a conclusion has a massive effect on how impactful it will be to a reader.

Some types of writing can work well with a variety of conclusion formats, but others will confuse a paper’s message. For example, using a reflective style conclusion on a scientific research paper comes across as too opinion-based for a topic that’s shrouded in measurable fact.

Don’t Make It Too Complex. It’s best to use plain language when summarizing the information presented in a research paper or making a claim. Many students are tempted to use impressive wording and complex writing in a research paper conclusion to present themselves as experts in the subject , but it only gives the reader a headache.

Conclusions Should Be Concise . Research papers give the writer pages of leeway to make all the drawn-out points that they need, but conclusions don’t offer as much room. An essay’s conclusion needs to be short by definition because it’s merely a last takeaway for the reader. A research paper conclusion is a final paragraph, not the entire page .

Double Check Your Information. There’s nothing worse for a research paper’s validity than confidently making a claim in the conclusion that turns out to be false. It’s fundamental that all the facts and information your detail in a research paper are backed up with credible sources listed neatly on the works cited page.

Empathize With The Reader. Whether you’re submitting a research paper for an introductory university class or publishing a scholarly journal, you still need to keep the reader in mind when writing a conclusion. Think about who you’re communicating with through your research paper and what you’re hoping to accomplish with it.

Do Research . One way to fix the problem if you’re unsure of what makes an essay conclusion compelling is researching the topic. Reading articles (like this one) is helpful because they give you a clear demonstration of how to create a conclusion, but applying this structure to your own work can be difficult. A case of easier said than done.

Based on the goal or subject of your research paper, the structure of your conclusion changes. Pick a type of conclusion that will strengthen the point of your essay. Below are examples of different formats to use when writing research paper conclusions.

Summarization. The summarization conclusion is most commonly used for research papers that are presenting a series of concrete facts.

It’s the form of conclusion that most people are familiar with. Using the summary technique requires a succinct compiling of the most critical points you’ve made in an essay.

Summarization Conclusion Formatting Works Best For:

Solution-Based Research

Persuasive Writing

History and Science Studies

Structuring An Argument

Reflective. A conclusion that uses a reflective structure takes the information outlined in the research paper to arrive at a grander insight about the topic at hand. This type of conclusion is popular when you’re attempting to change the reader’s viewpoint with a paper.

Reflective Conclusion Formatting Works Best For:

Persuasive Essays

English and Political Studies

Projective. When using a projective conclusion, the writer applies their work presented earlier in the thesis to eventual outcomes that can arise. It is called a projective conclusion because it is more results-based than summarizing facts or establishing an overarching lesson.

Projective Conclusion Formatting Works Best For:

Research Paper

Expository Essay

Narrative Works (Sometimes)

The conclusion of a research paper ties together all the prior information you’ve covered. It leaves the reader with a final thought about the research paper and the message it’s trying to convey.

Unlike the body paragraphs of a research paper, which aim at specificity and focus on developing a single concept or piece of information, conclusions are broader. The goal is to gloss over what’s already been stated earlier in the essay to solidify it with the reader.

The conclusion also serves a different purpose than the introduction . An introductory paragraph is for establishing what the reader will be learning more about. It opens the metaphorical door towards understanding a research endeavor or topic. The conclusion closes the argument that the introductory paragraph opens.

Including a conclusion is an important part of writing a research paper because it creates an organized summarization of information and outlines inferences about the subject studied. It provides an additional layer of clarity in a short written work.

Research papers are often lengthy and dull, so it’s easy for a reader’s attention to stray. A conclusion brings the reader back and offers them the most critical takeaways from the paper.

How long should a good conclusion be?

A good conclusion should be one paragraph or three to five sentences long. Your research paper conclusion should be concise, which means you don’t need to take up a whole page for just your conclusion. Instead, try to stick to about one paragraph in length.

What are the general rules in crafting conclusions in your research paper?

The general rules for crafting conclusions for your research paper include:

Choose the right conclusion format.

Keep it simple.

Be concise.

Be accurate.

Keep the reader’s needs (or requirements) in mind.

Remind the reader of your thesis.

Summarize and connect main points.

End with a concluding sentence.

What is a better way to say, “In conclusion”?

A better way to say, “In conclusion,” is “Therefore,” “Finally,” or “Lastly.” Other good words include, “As expressed” or “As a result.” You can also simply launch into your concluding paragraph if a transition isn’t needed.

How useful was this post?

Click on a star to rate it!

Average rating / 5. Vote count:

No votes so far! Be the first to rate this post.

Sky Ariella is a professional freelance writer, originally from New York. She has been featured on websites and online magazines covering topics in career, travel, and lifestyle. She received her BA in psychology from Hunter College.

Recent Job Searches

• Registered Nurse Jobs Resume Location
• Truck Driver Jobs Resume Location
• Call Center Representative Jobs Resume Location
• Customer Service Representative Jobs Resume
• Delivery Driver Jobs Resume Location
• Warehouse Worker Jobs Resume Location
• Account Executive Jobs Resume Location
• Sales Associate Jobs Resume Location
• Licensed Practical Nurse Jobs Resume Location
• Company Driver Jobs Resume

Related posts

50 Jobs That Use Html The Most

How To List Computer Skills On A Resume [Full List + Examples]

How To Give An Effective Presentation (With Examples)

Most Common Resume Items of Business Founders and CEOs

• Career Advice >
• Desired Traits >

How to Write a Conclusion for a Research Paper

When you're wrapping up a research paper, the conclusion is like the grand finale of a fireworks show – it's your chance to leave a lasting impression. In this article, we'll break down the steps to help you write a winning research paper conclusion that not only recaps your main points but also ties everything together. Consider it the "So what?" moment – why should people care about your research? Our professional essay writers will guide you through making your conclusion strong, clear, and something that sticks with your readers long after they've put down your paper. So, let's dive in and ensure your research ends on a high note!

What Is a Conclusion in a Research Paper

In a research paper, the conclusion serves as the final segment, where you summarize the main points and findings of your study. It's not just a repetition of what you've already said but rather a chance to tie everything together and highlight the significance of your research. As you learn how to start a research paper , a good conclusion also often discusses the implications of your findings, suggests potential areas for further research, and leaves the reader with a lasting impression of the importance and relevance of your work in the broader context of the field. Essentially, it's your last opportunity to make a strong impact and leave your readers with a clear understanding of the significance of your research. Here’s a research paper conclusion example:

In conclusion, this research paper has navigated the intricacies of sustainable urban development, shedding light on the pivotal role of community engagement and innovative planning strategies. Through applying qualitative and quantitative research methods, we've uncovered valuable insights into the challenges and opportunities inherent in fostering environmentally friendly urban spaces. The implications of these findings extend beyond the confines of this study, emphasizing the imperative for continued exploration in the realms of urban planning and environmental sustainability. By emphasizing both the practical applications and theoretical contributions, this research underscores the significance of community involvement and forward-thinking strategies in shaping the future of urban landscapes. As cities evolve, incorporating these insights into planning and development practices will create resilient and harmonious urban environments.

Conclusion Outline for Research Paper

This outline for a research paper conclusion provides a structured framework to ensure that your ending effectively summarizes the key elements of your research paper and leaves a lasting impression on your readers. Adjust the content based on the specific requirements and focus of your research.

Restate the Thesis Statement

• Briefly restate the main thesis or research question.
• Emphasize the core objective or purpose of the study.

Summarize Key Findings

• Recap the main points and key findings from each section of the paper.
• Provide a concise overview of the research journey.

Discuss Implications

• Explore the broader implications of the research findings.
• Discuss how the results contribute to the existing body of knowledge in the field.

Address Limitations

• Acknowledge any limitations or constraints encountered during the research process.
• Explain how these limitations may impact the interpretation of the findings.

Suggest Areas for Future Research

• Propose potential directions for future studies related to the topic.
• Identify gaps in the current research that warrant further exploration.

Reaffirm Significance

• Reaffirm the importance and relevance of the research in the broader context.
• Highlight the practical applications or real-world implications of the study.

Concluding Statement

• Craft a strong, memorable closing statement that leaves a lasting impression.
• Sum up the overall impact of the research and its potential contribution to the field.

Study the full guide on how to make a research paper outline here, which will also specify the conclusion writing specifics to improve your general prowess.

Tips on How to Make a Conclusion in Research

Here are key considerations regarding a conclusion for research paper to not only recap the primary ideas in your work but also delve deeper to earn a higher grade:

• Provide a concise recap of your main research outcomes.
• Remind readers of your research goals and their accomplishments.
• Stick to summarizing existing content; refrain from adding new details.
• Emphasize why your research matters and its broader implications.
• Clearly explain the practical or theoretical impact of your findings.
• Prompt readers to reflect on how your research influences their perspective.
• Briefly discuss the robustness of your research methods.
• End with a suggestion for future research or a practical application.
• Transparently address any constraints or biases in your study.
• End on a powerful note, leaving a memorable impression on your readers.

For your inspiration, we’ve also prepared this research proposal example APA , which dwells on another important aspect of research writing.

How to Write a Research Paper Conclusion

As you finish your research paper, the conclusion takes center stage. In this section, we've got five practical tips for writing a conclusion for a research paper. We'll guide you through summarizing your key findings, revisiting your research goals, discussing the bigger picture, addressing any limitations, and ending on a powerful note. Think of it as your roadmap to creating a conclusion that not only wraps up your research but also leaves a lasting impact on your readers. Let's dive in and make sure your conclusion stands out for all the right reasons!

Synthesize Core Discoveries. Initiate your conclusion by synthesizing the essential discoveries of your research. Offer a succinct recapitulation of the primary points and outcomes you have elucidated in your paper. This aids in reinforcing the gravity of your work and reiterates the pivotal information you have presented.

Revisit Research Objectives. Revisit the research objectives or questions you outlined at the beginning of your paper. Assess whether you have successfully addressed these objectives and if your findings align with the initial goals of your research. This reflection helps tie your conclusion back to the purpose of your study.

Discuss Implications and Contributions. Discuss the broader implications of your research and its potential contributions to the field. Consider how your findings might impact future research, applications, or understanding of the subject matter. This demonstrates the significance of your work and places it within a larger context.

Address Limitations and Future Research. Acknowledge any limitations in your study, such as constraints in data collection or potential biases. Briefly discuss how these limitations might have affected your results. Additionally, suggest areas for future research that could build upon your work, addressing any unanswered questions or unexplored aspects. This demonstrates a thoughtful approach to your research.

End with a Strong Conclusion Statement. Conclude your research paper with a strong and memorable statement that reinforces the key message you want readers to take away. This could be a call to action, a proposal for further investigation, or a reflection on the broader significance of your findings. Leave your readers with a lasting impression that emphasizes the importance of your research. Remember that you can buy a research paper anytime if you lack time or get stuck in writer’s block.

Want to Enjoy the Benefits of Academic Freedom?

With our top-notch writers and 24/7 support, you can relax and focus on what really matters

Stylistic Devices to Use in a Conclusion

Discover distinctive stylistic insights that you can apply when writing a conclusion for a research paper:

• Rhetorical Questions. When using rhetorical questions, strategically place them to engage readers' minds. For instance, you might pose a question that prompts reflection on the broader implications of your findings, leaving your audience with something to ponder.
• Powerful Language. Incorporate strong language to convey a sense of conviction and importance. Choose words that resonate with the overall tone of your research and amplify the significance of your conclusions. This adds weight to your key messages.
• Repetitions. Repetitions can be employed to reinforce essential ideas. Reiterate key phrases or concepts in a way that emphasizes their importance without sounding redundant. This technique serves to drive home your main points.
• Anecdotes. Integrating anecdotes into your conclusion can provide a human touch. Share a brief and relevant story that connects with your research, making the information more relatable and memorable for your audience.
• Vivid Imagery. Lastly, use vivid imagery to paint a picture in the minds of your readers. Appeal to their senses by describing scenarios or outcomes related to your research. This creates a more immersive and lasting impression.

If you have a larger paper to write, for example a thesis, use our custom dissertation writing can help you in no time.

How to Make a Conclusion Logically Appealing

Knowing how to write a conclusion for a research paper that is logically appealing is important for leaving a lasting impression on your readers. Here are some tips to achieve this:

Logical Sequencing

• Present your conclusion in a structured manner, following the natural flow of your paper. Readers should effortlessly follow your thought process, making your conclusion more accessible and persuasive.

Reinforce Main Arguments

• Emphasize the core arguments and findings from your research. By reinforcing key points, you solidify your stance and provide a logical culmination to your paper.

Address Counterarguments

• Acknowledge and address potential counterarguments or limitations in your research. Demonstrate intellectual honesty and strengthen your conclusion by preemptively addressing potential doubts.

Connect with Introduction

• Revisit themes or concepts introduced in your introduction to create a cohesive narrative, allowing readers to trace the logical progression of your research from start to finish.

Propose Actionable Insights

• Suggest practical applications or recommendations based on your findings. This will add a forward-looking dimension, making your conclusion more relevant and compelling.

Highlight Significance

• Clearly articulate the broader implications of your research to convey the importance of your work and its potential impact on the field, making your conclusion logically compelling.

Are you ready to produce an A-grade assignment? If not, opt for a custom research paper from our skilled writers across various disciplines.

Avoid These Things When Writing a Research Paper Conclusion

As you write your conclusion of research paper, there’s a list of things professional writers don’t recommend doing. Consider these issues carefully:

• Repetition of Exact Phrases
• Repetitively using the same phrases or sentences from the main body. Repetition can make your conclusion seem redundant and less engaging.
• Overly Lengthy Summaries
• Providing excessively detailed summaries of each section of your paper. Readers may lose interest if the conclusion becomes too long and detailed.
• Unclear Connection to the Introduction
• Failing to connect the conclusion back to the introduction. A lack of continuity may make the paper feel disjointed.
• Adding New Arguments or Ideas
• Introducing new arguments or ideas that were not addressed in the body. This can confuse the reader and disrupt the coherence of your paper.
• Overuse of Complex Jargon
• Using excessively complex or technical language without clarification. Clear communication is essential in the conclusion, ensuring broad understanding.
• Apologizing or Undermining Confidence
• Apologizing for limitations or expressing doubt about your work. Maintain a confident tone; if limitations exist, present them objectively without undermining your research.
• Sweeping Generalizations
• Making overly broad or unsupported generalizations. Such statements can weaken the credibility of your conclusion.
• Neglecting the Significance
• Failing to emphasize the broader significance of your research. Readers need to understand why your findings matter in a larger context.
• Abrupt Endings
• Concluding abruptly without a strong closing statement. A powerful ending leaves a lasting impression; avoid a sudden or weak conclusion.

Research Paper Conclusion Example

That covers the essential aspects of summarizing a research paper. The only remaining step is to review the conclusion examples for research paper provided by our team.

Like our examples? Order our research proposal writing service to write paper according to your instructions to avoid plagiarizing and to keep your academic integrity strong.

Final Thoughts

In conclusion, the knowledge of how to write the conclusion of a research paper is pivotal for presenting your findings and leaving a lasting impression on your readers. By summarizing the key points, reiterating the significance of your research, and offering avenues for future exploration, you can create a conclusion that not only reinforces the value of your study but also encourages further academic discourse. Remember to balance brevity and completeness, ensuring your conclusion is concise yet comprehensive. Emphasizing the practical implications of your research and connecting it to the broader academic landscape will help solidify the impact of your work. Pay someone to write a research paper if you are having a hard time finishing your coursework on time.

Tired of Stressing Over Endless Essays and Deadlines? 

Let us take the load off your shoulders! Order your custom research paper today and experience the relief of knowing that your assignment is in expert hands

How To Write A Conclusion For A Research Paper?

What should the conclusion of a research paper contain, how to start a conclusion paragraph for a research paper.

Daniel Parker

is a seasoned educational writer focusing on scholarship guidance, research papers, and various forms of academic essays including reflective and narrative essays. His expertise also extends to detailed case studies. A scholar with a background in English Literature and Education, Daniel’s work on EssayPro blog aims to support students in achieving academic excellence and securing scholarships. His hobbies include reading classic literature and participating in academic forums.

is an expert in nursing and healthcare, with a strong background in history, law, and literature. Holding advanced degrees in nursing and public health, his analytical approach and comprehensive knowledge help students navigate complex topics. On EssayPro blog, Adam provides insightful articles on everything from historical analysis to the intricacies of healthcare policies. In his downtime, he enjoys historical documentaries and volunteering at local clinics.

Frequently asked questions

What should i include in a research paper conclusion.

The conclusion of a research paper has several key elements you should make sure to include:

• A restatement of the research problem
• A summary of your key arguments and/or findings
• A short discussion of the implications of your research

Frequently asked questions: Writing a research paper

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

The best way to remember the difference between a research plan and a research proposal is that they have fundamentally different audiences. A research plan helps you, the researcher, organize your thoughts. On the other hand, a dissertation proposal or research proposal aims to convince others (e.g., a supervisor, a funding body, or a dissertation committee) that your research topic is relevant and worthy of being conducted.

Formulating a main research question can be a difficult task. Overall, your question should contribute to solving the problem that you have defined in your problem statement .

However, it should also fulfill criteria in three main areas:

• Researchability
• Feasibility and specificity
• Relevance and originality

Research questions anchor your whole project, so it’s important to spend some time refining them.

In general, they should be:

• Focused and researchable
• Answerable using credible sources
• Complex and arguable
• Feasible and specific
• Relevant and original

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

A research aim is a broad statement indicating the general purpose of your research project. It should appear in your introduction at the end of your problem statement , before your research objectives.

Research objectives are more specific than your research aim. They indicate the specific ways you’ll address the overarching aim.

Once you’ve decided on your research objectives , you need to explain them in your paper, at the end of your problem statement .

Keep your research objectives clear and concise, and use appropriate verbs to accurately convey the work that you will carry out for each one.

I will compare …

Your research objectives indicate how you’ll try to address your research problem and should be specific:

Research objectives describe what you intend your research project to accomplish.

They summarize the approach and purpose of the project and help to focus your research.

Your objectives should appear in the introduction of your research paper , at the end of your problem statement .

The main guidelines for formatting a paper in Chicago style are to:

• Use a standard font like 12 pt Times New Roman
• Use 1 inch margins or larger
• Apply double line spacing
• Indent every new paragraph ½ inch
• Include a title page
• Place page numbers in the top right or bottom center
• Cite your sources with author-date citations or Chicago footnotes
• Include a bibliography or reference list

To automatically generate accurate Chicago references, you can use Scribbr’s free Chicago reference generator .

The main guidelines for formatting a paper in MLA style are as follows:

• Use an easily readable font like 12 pt Times New Roman
• Set 1 inch page margins
• Include a four-line MLA heading on the first page
• Center the paper’s title
• Use title case capitalization for headings
• Cite your sources with MLA in-text citations
• List all sources cited on a Works Cited page at the end

To format a paper in APA Style , follow these guidelines:

• Use a standard font like 12 pt Times New Roman or 11 pt Arial
• If submitting for publication, insert a running head on every page
• Apply APA heading styles
• Cite your sources with APA in-text citations
• List all sources cited on a reference page at the end

No, it’s not appropriate to present new arguments or evidence in the conclusion . While you might be tempted to save a striking argument for last, research papers follow a more formal structure than this.

All your findings and arguments should be presented in the body of the text (more specifically in the results and discussion sections if you are following a scientific structure). The conclusion is meant to summarize and reflect on the evidence and arguments you have already presented, not introduce new ones.

Don’t feel that you have to write the introduction first. The introduction is often one of the last parts of the research paper you’ll write, along with the conclusion.

This is because it can be easier to introduce your paper once you’ve already written the body ; you may not have the clearest idea of your arguments until you’ve written them, and things can change during the writing process .

The way you present your research problem in your introduction varies depending on the nature of your research paper . A research paper that presents a sustained argument will usually encapsulate this argument in a thesis statement .

A research paper designed to present the results of empirical research tends to present a research question that it seeks to answer. It may also include a hypothesis —a prediction that will be confirmed or disproved by your research.

The introduction of a research paper includes several key elements:

• A hook to catch the reader’s interest
• Relevant background on the topic
• Details of your research problem

and your problem statement

• A thesis statement or research question
• Sometimes an overview of the paper

Ask our team

Want to contact us directly? No problem.  We  are always here for you.

• Email [email protected]
• Start live chat
• Call +1 (510) 822-8066
• WhatsApp +31 20 261 6040

Our team helps students graduate by offering:

• A world-class citation generator
• Plagiarism Checker software powered by Turnitin
• Innovative Citation Checker software
• Professional proofreading services
• Over 300 helpful articles about academic writing, citing sources, plagiarism, and more

Scribbr specializes in editing study-related documents . We proofread:

• PhD dissertations
• Research proposals
• Personal statements
• Admission essays
• Motivation letters
• Reflection papers
• Journal articles
• Capstone projects

Scribbr’s Plagiarism Checker is powered by elements of Turnitin’s Similarity Checker , namely the plagiarism detection software and the Internet Archive and Premium Scholarly Publications content databases .

The add-on AI detector is powered by Scribbr’s proprietary software.

The Scribbr Citation Generator is developed using the open-source Citation Style Language (CSL) project and Frank Bennett’s citeproc-js . It’s the same technology used by dozens of other popular citation tools, including Mendeley and Zotero.

You can find all the citation styles and locales used in the Scribbr Citation Generator in our publicly accessible repository on Github .

Free All-in-One Office Suite with PDF Editor

Edit Word, Excel, and PPT for FREE.

Read, edit, and convert PDFs with the powerful PDF toolkit.

Microsoft-like interface, easy to use.

Windows • MacOS • Linux • iOS • Android

Select areas that need to improve

• Didn't match my interface
• Too technical or incomprehensible
• Incorrect operation instructions
• Incomplete instructions on this function

Fields marked * are required please

Please leave your suggestions below

• Quick Tutorials
• Practical Skills

How to Write a Conclusion - Steps with Examples

I remember my college days when one of the most dreadful assignments was writing a research paper. It made me wonder if there was an easier way to help me through it. The worst part was writing the conclusion, which meant wrapping up the entire paper and finally drawing conclusions. It sounds pretty intimidating, doesn't it? How are you supposed to fit all that information into such a short space, and what else might you be missing? In this guide, I will show you how to write a conclusion so you can spare yourself from the distress of it all.

What to Include/ Not Include in a Conclusion?

Professors often stress a lot on writing a good conclusion that includes a wrap-up for your paper or essay. These are some factors you must consider to include in your conclusion:

Restate Your Thesis:

Begin by restating the main argument or thesis of your paper. This reinforces the central point you have been arguing throughout your work.

Summarize Key Points:

Provide a concise summary of the key points and findings from your paper. Highlight the most significant pieces of evidence that support your thesis.

Discuss the Implications:

Explain the broader implications of your findings. How do they contribute to the field of study? What practical applications or theoretical advancements arise from your research?

Address Limitations:

Acknowledge any limitations or weaknesses in your study. This demonstrates a critical and reflective approach to your research and provides a foundation for future work.

Suggest Future Research:

Propose areas for future research. What questions remain unanswered? What further investigations could build on your findings?

End with a Strong Closing Statement:

Conclude with a strong, impactful statement that leaves a lasting impression on your reader. This could be a call to action, a prediction, or a thought-provoking question related to your research topic.

There may also be certain things you would unknowingly add in your conclusion that would ultimately leave a bad impression on the reader. Keep these factors in mind so you may avoid when writing your conclusion for your paper:

New Information:

Avoid introducing new information or ideas that were not covered in the body of the paper. The conclusion is for synthesizing and reflecting on the information already presented.

Detailed Methodology:

Do not include detailed descriptions of your research methods. This information belongs in the methodology section of your paper.

Repetitive Summaries:

Refrain from simply reiterating points that were already made in the results or discussion sections. Instead, focus on synthesizing the information and highlighting its significance.

Speculative Statements:

Avoid idle speculation or guesswork about potential outcomes or implications that are not supported by your research findings.

Apologies or Undermining Your Work:

Do not undermine your work by apologizing for any perceived shortcomings. Present your conclusions confidently and assert the value of your research.

Excessive Length:

Keep the conclusion concise and to the point. Long, drawn-out conclusions can dilute the impact of your final statements.

To put things into perspective, here's what a good and bad conclusion example look like:

Good Example:

Bad Example:

Types of Conclusion

Summarizing conclusion:.

This type is the most common and involves summarizing the main points of the research, reiterating the research question, and restating the significance of the findings.

It is broadly used across different disciplines.

Example: If a study investigated the impact of social media on adolescents' mental health, a summarizing conclusion would reiterate key findings, such as the association between high social media use and increased anxiety and depression levels among adolescents, and emphasize the importance of these findings for developing effective interventions.

Editorial Conclusion:

This type is used less frequently and is suited for research papers that advocate for a particular viewpoint or policy. It presents a strong editorial opinion based on the research findings and offers recommendations or calls to action.

It is suitable for papers focusing on policy recommendations or advocating a specific viewpoint.

Example: For a study on the environmental impact of plastic waste, an editorial conclusion might call for a comprehensive ban on single-use plastics and increased recycling initiatives, urging governments, businesses, and individuals to take immediate action to protect the environment.

Externalizing Conclusion:

This type extends the research beyond the scope of the paper by suggesting future research directions or discussing broader implications of the findings. It is often used in theoretical or exploratory research papers.

It is Ideal for theoretical or exploratory studies.

Example: In a study exploring AI applications in healthcare, an externalizing conclusion might suggest future research into the ethical, legal, and social implications of AI in healthcare and emphasize the need for interdisciplinary collaboration to harness AI's potential while addressing its challenges.

How to Write a Conclusion in 4 Steps [With Examples]

Writing a conclusion may seem a bit tricky, but once you fully understand the essence of what goes into a conclusion, it will become much easier. To demonstrate how to write a conclusion, I will be using WPS Office , a tool designed to be convenient for students, thanks to its easy-to-use interface and free features. You can also utilize WPS AI, as I am in these simple 4 steps, to make the entire process smoother for yourself.

Step 1: Restate The Thesis Statement

Start your conclusion by restating the thesis statement of your research paper. This reminds the reader of the main focus and purpose of your study.

Example: If your thesis statement is "This study investigates the impact of social media on adolescents' mental health, revealing a significant association between high social media usage and increased levels of anxiety and depression.", you can use WPS AI to help improve and rewrite your thesis statement.

Here's how WPS AI can assist you with your thesis statement.

Write your thesis statement in WPS Writer and select the entire text using your mouse.

After selecting the text, a small hover menu will appear. Click on the "WPS AI" icon in this menu.

This will open a list of AI assistance options you can choose from. To ask WPS AI to improve your thesis statement, click on "Improve Writing".

WPS AI will process and return an improved thesis statement. If you don’t like the improved version, click on "Rewrite", or click on "Accept" to replace your text with the improved version.

Step 2: Review Main Supporting Points

Next, we need to summarize the key points of our research. When summarizing the key findings of your research, it’s important to highlight the most significant results and their implications.

Example: Let's say that from our research the most important findings were:

The study found that high social media usage negatively affects adolescents' self-esteem due to constant exposure to idealized images and lifestyles.

Excessive use of social media, particularly before bedtime, was linked to disrupted sleep patterns and insufficient rest, contributing to mental health issues.

Despite being a tool for connection, high social media usage can lead to feelings of loneliness and social isolation as face-to-face interactions decrease.

Here's how WPS AI can assist you summarize the key points of your research for your conclusion.

Let's switch to WPS Office again, and this time let's select the key points that we have written down from our research.

Click on the WPS AI icon from the hover menu to open the list of options you can choose from.

From the list, let's click on "Summarize" to shorten and summarize the key points from our research.

You can now choose to either accept or ask WPS AI to rewrite this summary of key points again.

Step 3: Show Why It Matters

Now that you have laid out all the findings from your paper and WPS AI has effectively summarized them, you can further prompt it to broaden the implications of your findings and follow up with real-world problems.

To get real-world insights using WPS AI, follow these steps:

Click on the WPS AI widget at the top right corner of the WPS Writer interface.

The WPS AI pane will open on the right. Here, simply type in your prompt. Here is an example of a prompt:

"Explain the significance of high social media usage leading to increased anxiety and depression in adolescents, and discuss potential real-world problems and solutions."

WPS AI will display the results, which can now be a part of your summary or can be further summarized or improved with the help of WPS AI.

Step 4: Offer Meaningful Insights

Lastly, provide some final thoughts or insights that will leave a lasting impression on your reader. This can include suggestions for future research, practical applications of your findings, or a call to action based on your conclusions.

Example: Here is an example of how Meaningful Insights can be presented:

Further research is needed to explore the long-term effects of social media usage on adolescent mental health and to identify effective interventions.

Developing and promoting apps that encourage healthy social media use and provide mental health support could mitigate the negative effects identified in the study.

Stakeholders, including policymakers, educators, and parents, should collaborate to create environments that foster healthy digital habits and support adolescents' mental health.

Now, with the help of WPS AI, these points can simply be summarized to get more concise and structured Meaningful Insights for our conclusion.

Bonus Tips: How to Polish your Conclusion with WPS AI

Writing a strong conclusion for your research paper is crucial, and WPS Office is designed to be exceptionally student-friendly. It offers accessible options and advanced features for free, making it an excellent tool for students. One of the standout features is WPS AI, which integrates AI into its writing and proofreading abilities.

Draft Generation: WPS AI can assist you in writing a conclusion by generating an initial draft. This draft serves as a solid foundation, ensuring that all essential elements are included and properly structured.

Grammar and Style Check: WPS AI can identify grammar errors, awkward phrasing, and inconsistencies in your conclusion paragraph. This ensures that your writing is polished and professional.

Sentence Structure Enhancement: The AI can suggest improvements to sentence structures, helping you to vary sentence lengths and styles for better readability and flow. This makes your conclusion more engaging and easier to read.

Vocabulary Enhancement: WPS AI offers synonyms and alternative word choices to enhance the vocabulary in your conclusion, making your writing more sophisticated and engaging.

Clarity and Conciseness: WPS AI can help you refine your conclusion to ensure it effectively summarizes your main points without unnecessary repetition or tangents. This keeps your conclusion focused and impactful.

Refinement and Customization: Once WPS AI has generated the draft, you can refine and personalize it to align with your research and style. This step allows you to inject your voice and insights into the conclusion, making it uniquely yours.

Polishing and Proofreading: After refining the draft, you can use WPS AI to polish the conclusion further. WPS AI's advanced proofreading capabilities ensure that your conclusion is not only coherent and concise but also free of grammatical errors and stylistic inconsistencies.

ByIncorporating WPS AI into your writing routine you can significantly improve your efficiency and the overall quality of your academic work. You can streamline the process of writing your research paper conclusion, saving time and effort while ensuring a high-quality result. Whether you’re summarizing key findings, making policy recommendations, or suggesting future research directions, WPS AI helps you create a compelling and impactful conclusion.

So we have seen how WPS AI can help us write more effective and accurate conclusions, but is this all the help it offers? Absolutely not! With the help of WPS AI, you can further improve your conclusion by making it more fluent and easier to read.

Furthermore, WPS AI is not just a writing tool; it also offers AI spell check features, which can help students proofread their work according to their academic style such as APA, MLA, or Chicago style.

WPS Office has a lot to offer and is a perfect tool for students who need help writing not just effective conclusions but also effective research papers. So if you are stuck with a conclusion or a research paper, consider turning to WPS AI for help.

FAQs about writing a conclusion for paper/ essay

1. how long should a conclusion be.

A well-constructed conclusion typically constitutes approximately 10% of your document's total word count. For instance, in a 1,500-word paper, aim for a conclusion of about 150 words. This provides sufficient space to summarize key points and offer a final overview of the main ideas discussed.

2. How can I make my Conclusion impactful?

Here are some effective strategies for creating an impactful conclusion:

Utilize compelling language to engage the reader effectively.

Ensure the conclusion remains clear and concise, omitting insignificant specifics.

Conclude with a stimulating statement, a call to action, or a reflection on the broader implications of your research findings to make a lasting impact.

3. How do I avoid simply repeating what I've already said in the Conclusion?

To avoid repeating yourself in your conclusion, focus on cohesively summarizing your main ideas rather than reiterating them. Additionally, consider exploring the wider impact of your arguments or suggesting directions for future research on your topic. This approach ensures your conclusion provides fresh perspectives and maintains reader interest.

Perfect Your Conclusion With WPS Office

Your research paper is not complete without a strong conclusion. The person who reads your paper should feel like they have taken away significant key insights from your work. Writing an effective conclusion can sometimes be challenging, but WPS Office, with its AI capabilities, can assist you in helping you with how to write a conclusion to perfection. Incorporate WPS AI into your writing routine to significantly improve your efficiency and the overall quality of your academic work. Try WPS Office today and experience the benefits of AI-assisted writing firsthand.

• 1. How to Write an Essay in MLA Format | For Students
• 2. How to Write a Cover Letter [Tips with Examples]
• 3. How to write matrix in Word
• 4. How to write meter square in powerpoint
• 5. 10 Professional Company Presentation Templates: To Write a Great Company Profile
• 6. How to Write an Abstract - Steps with Examples

15 years of office industry experience, tech lover and copywriter. Follow me for product reviews, comparisons, and recommendations for new apps and software.

• Systematic review
• Open access
• Published: 24 June 2024

A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022

• Laura Ellen Ashcraft   ORCID: orcid.org/0000-0001-9957-0617 1 , 2 ,
• David E. Goodrich 3 , 4 , 5 ,
• Joachim Hero 6 ,
• Angela Phares 3 ,
• Rachel L. Bachrach 7 , 8 ,
• Deirdre A. Quinn 3 , 4 ,
• Nabeel Qureshi 6 ,
• Natalie C. Ernecoff 6 ,
• Lisa G. Lederer 5 ,
• Leslie Page Scheunemann 9 , 10 ,
• Shari S. Rogal 3 , 11   na1 &
• Matthew J. Chinman 3 , 4 , 6   na1  

Implementation Science volume  19 , Article number:  43 ( 2024 ) Cite this article

1958 Accesses

18 Altmetric

Metrics details

Studies of implementation strategies range in rigor, design, and evaluated outcomes, presenting interpretation challenges for practitioners and researchers. This systematic review aimed to describe the body of research evidence testing implementation strategies across diverse settings and domains, using the Expert Recommendations for Implementing Change (ERIC) taxonomy to classify strategies and the Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) framework to classify outcomes.

We conducted a systematic review of studies examining implementation strategies from 2010-2022 and registered with PROSPERO (CRD42021235592). We searched databases using terms “implementation strategy”, “intervention”, “bundle”, “support”, and their variants. We also solicited study recommendations from implementation science experts and mined existing systematic reviews. We included studies that quantitatively assessed the impact of at least one implementation strategy to improve health or health care using an outcome that could be mapped to the five evaluation dimensions of RE-AIM. Only studies meeting prespecified methodologic standards were included. We described the characteristics of studies and frequency of implementation strategy use across study arms. We also examined common strategy pairings and cooccurrence with significant outcomes.

Our search resulted in 16,605 studies; 129 met inclusion criteria. Studies tested an average of 6.73 strategies (0-20 range). The most assessed outcomes were Effectiveness ( n =82; 64%) and Implementation ( n =73; 56%). The implementation strategies most frequently occurring in the experimental arm were Distribute Educational Materials ( n =99), Conduct Educational Meetings ( n =96), Audit and Provide Feedback ( n =76), and External Facilitation ( n =59). These strategies were often used in combination. Nineteen implementation strategies were frequently tested and associated with significantly improved outcomes. However, many strategies were not tested sufficiently to draw conclusions.

This review of 129 methodologically rigorous studies built upon prior implementation science data syntheses to identify implementation strategies that had been experimentally tested and summarized their impact on outcomes across diverse outcomes and clinical settings. We present recommendations for improving future similar efforts.

Peer Review reports

Contributions to the literature

While many implementation strategies exist, it has been challenging to compare their effectiveness across a wide range of trial designs and practice settings

This systematic review provides a transdisciplinary evaluation of implementation strategies across population, practice setting, and evidence-based interventions using a standardized taxonomy of strategies and outcomes.

Educational strategies were employed ubiquitously; nineteen other commonly used implementation strategies, including External Facilitation and Audit and Provide Feedback, were associated with positive outcomes in these experimental trials.

This review offers guidance for scholars and practitioners alike in selecting implementation strategies and suggests a roadmap for future evidence generation.

Implementation strategies are “methods or techniques used to enhance the adoption, implementation, and sustainment of evidence-based practices or programs” (EBPs) [ 1 ]. In 2015, the Expert Recommendations for Implementing Change (ERIC) study organized a panel of implementation scientists to compile a standardized set of implementation strategy terms and definitions [ 2 , 3 , 4 ]. These 73 strategies were then organized into nine “clusters” [ 5 ]. The ERIC taxonomy has been widely adopted and further refined [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 ]. However, much of the evidence for individual or groups of ERIC strategies remains narrowly focused. Prior systematic reviews and meta-analyses have assessed strategy effectiveness, but have generally focused on a specific strategy, (e.g., Audit and Provide Feedback) [ 14 , 15 , 16 ], subpopulation, disease (e.g., individuals living with dementia) [ 16 ], outcome [ 15 ], service setting (e.g., primary care clinics) [ 17 , 18 , 19 ] or geography [ 20 ]. Given that these strategies are intended to have broad applicability, there remains a need to understand how well implementation strategies work across EBPs and settings and the extent to which implementation knowledge is generalizable.

There are challenges in assessing the evidence of implementation strategies across many EBPs, populations, and settings. Heterogeneity in population characteristics, study designs, methods, and outcomes have made it difficult to quantitatively compare which strategies work and under which conditions [ 21 ]. Moreover, there remains significant variability in how researchers operationalize, apply, and report strategies (individually or in combination) and outcomes [ 21 , 22 ]. Still, synthesizing data related to using individual strategies would help researchers replicate findings and better understand possible mediating factors including the cost, timing, and delivery by specific types of health providers or key partners [ 23 , 24 , 25 ]. Such an evidence base would also aid practitioners with implementation planning such as when and how to deploy a strategy for optimal impact.

Building upon previous efforts, we therefore conducted a systematic review to evaluate the level of evidence supporting the ERIC implementation strategies across a broad array of health and human service settings and outcomes, as organized by the evaluation framework, RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) [ 26 , 27 , 28 ]. A secondary aim of this work was to identify patterns in scientific reporting of strategy use that could not only inform reporting standards for strategies but also the methods employed in future. The current study was guided by the following research questions Footnote 1 :

What implementation strategies have been most commonly and rigorously tested in health and human service settings?

Which implementation strategies were commonly paired?

What is the evidence supporting commonly tested implementation strategies?

We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-P) model [ 29 , 30 , 31 ] to develop and report on the methods for this systematic review (Additional File 1). This study was considered to be non-human subjects research by the RAND institutional review board.

Registration

The protocol was registered with PROSPERO (PROSPERO 2021 CRD42021235592).

Eligibility criteria

This review sought to synthesize evidence for implementation strategies from research studies conducted across a wide range of health-related settings and populations. Inclusion criteria required studies to: 1) available in English; 2) published between January 1, 2010 and September 20, 2022; 3) based on experimental research (excluded protocols, commentaries, conference abstracts, or proposed frameworks); 4) set in a health or human service context (described below); 5) tested at least one quantitative outcome that could be mapped to the RE-AIM evaluation framework [ 26 , 27 , 28 ]; and 6) evaluated the impact of an implementation strategy that could be classified using the ERIC taxonomy [ 2 , 32 ]. We defined health and human service setting broadly, including inpatient and outpatient healthcare settings, specialty clinics, mental health treatment centers, long-term care facilities, group homes, correctional facilities, child welfare or youth services, aging services, and schools, and required that the focus be on a health outcome. We excluded hybrid type I trials that primarily focused on establishing EBP effectiveness, qualitative studies, studies that described implementation barriers and facilitators without assessing implementation strategy impact on an outcome, and studies not meeting standardized rigor criteria defined below.

Information sources

Our three-pronged search strategy included searching academic databases (i.e., CINAHL, PubMed, and Web of Science for replicability and transparency), seeking recommendations from expert implementation scientists, and assessing existing, relevant systematic reviews and meta-analyses.

Search strategy

Search terms included “implementation strateg*” OR “implementation intervention*” OR “implementation bundl*” OR “implementation support*.” The search, conducted on September 20, 2022, was limited to English language and publication between 2010 and 2022, similar to other recent implementation science reviews [ 22 ]. This timeframe was selected to coincide with the advent of Implementation Science and when the term “implementation strategy” became conventionally used [ 2 , 4 , 33 ]. A full search strategy can be found in Additional File 2.

Title and abstract screening process

Each study’s title and abstract were read by two reviewers, who dichotomously scored studies on each of the six eligibility criteria described above as yes=1 or no=0, resulting in a score ranging from 1 to 6. Abstracts receiving a six from both reviewers were included in the full text review. Those with only one score of six were adjudicated by a senior member of the team (MJC, SSR, DEG). The study team held weekly meetings to troubleshoot and resolve any ongoing issues noted through the abstract screening process.

Full text screening

During the full text screening process, we reviewed, in pairs, each article that had progressed through abstract screening. Conflicts between reviewers were adjudicated by a senior member of the team for a final inclusion decision (MJC, SSR, DEG).

Review of study rigor

After reviewing published rigor screening tools [ 34 , 35 , 36 ], we developed an assessment of study rigor that was appropriate for the broad range of reviewed implementation studies. Reviewers evaluated studies on the following: 1) presence of a concurrent comparison or control group (=2 for traditional randomized controlled trial or stepped wedge cluster randomized trial and =1 for pseudo-randomized and other studies with concurrent control); 2) EBP standardization by protocol or manual (=1 if present); 3) EBP fidelity tracking (=1 if present); 4) implementation strategy standardization by operational description, standard training, or manual (=1 if present); 5) length of follow-up from full implementation of intervention (=2 for twelve months or longer, =1 for six to eleven months, or =0 for less than six months); and 6) number of sites (=1 for more than one site). Rigor scores ranged from 0 to 8, with 8 indicating the most rigorous. Articles were included if they 1) included a concurrent control group, 2) had an experimental design, and 3) received a score of 7 or 8 from two independent reviewers.

Outside expert consultation

We contacted 37 global implementation science experts who were recognized by our study team as leaders in the field or who were commonly represented among first or senior authors in the included abstracts. We asked each expert for recommendations of publications meeting study inclusion criteria (i.e., quantitatively evaluating the effectiveness of an implementation strategy). Recommendations were recorded and compared to the full abstract list.

Systematic reviews

Eighty-four systematic reviews were identified through the initial search strategy (See Additional File 3). Systematic reviews that examined the effectiveness of implementation strategies were reviewed in pairs for studies that were not found through our initial literature search.

Data abstraction and coding

Data from the full text review were abstracted in pairs, with conflicts resolved by senior team members (DEG, MJC) using a standard Qualtrics abstraction form. The form captured the setting, number of sites and participants studied, evidence-based practice/program of focus, outcomes assessed (based on RE-AIM), strategies used in each study arm, whether the study took place in the U.S. or outside of the U.S., and the findings (i.e., was there significant improvement in the outcome(s)?). We coded implementation strategies used in the Control and Experimental Arms. We defined the Control Arm as receiving the lowest number of strategies (which could mean zero strategies or care as usual) and the Experimental Arm as the most intensive arm (i.e., receiving the highest number of strategies). When studies included multiple Experimental Arms, the Experimental Arm with the least intensive implementation strategy(ies) was classified as “Control” and the Experimental Arm with the most intensive implementation strategy(ies) was classified as the “Experimental” Arm.

Implementation strategies were classified using standard definitions (MJC, SSR, DEG), based on minor modifications to the ERIC taxonomy [ 2 , 3 , 4 ]. Modifications resulted in 70 named strategies and were made to decrease redundancy and improve clarity. These modifications were based on input from experts, cognitive interview data, and team consensus [ 37 ] (See Additional File 4). Outcomes were then coded into RE-AIM outcome domains following best practices as recommended by framework experts [ 26 , 27 , 28 ]. We coded the RE-AIM domain of Effectiveness as either an assessment of the effectiveness of the EBP or the implementation strategy. We did not assess implementation strategy fidelity or effects on health disparities as these are recently adopted reporting standards [ 27 , 28 ] and not yet widely implemented in current publications. Further, we did not include implementation costs as an outcome because reporting guidelines have not been standardized [ 38 , 39 ].

Assessment and minimization of bias

Assessment and minimization of bias is an important component of high-quality systematic reviews. The Cochrane Collaboration guidance for conducting high-quality systematic reviews recommends including a specific assessment of bias for individual studies by assessing the domains of randomization, deviations of intended intervention, missing data, measurement of the outcome, and selection of the reported results (e.g., following a pre-specified analysis plan) [ 40 , 41 ]. One way we addressed bias was by consolidating multiple publications from the same study into a single finding (i.e., N =1), so-as to avoid inflating estimates due to multiple publications on different aspects of a single trial. We also included high-quality studies only, as described above. However, it was not feasible to consistently apply an assessment of bias tool due to implementation science’s broad scope and the heterogeneity of study design, context, outcomes, and variable measurement, etc. For example, most implementation studies reviewed had many outcomes across the RE-AIM framework, with no one outcome designated as primary, precluding assignment of a single score across studies.

We used descriptive statistics to present the distribution of health or healthcare area, settings, outcomes, and the median number of included patients and sites per study, overall and by country (classified as U.S. vs. non-U.S.). Implementation strategies were described individually, using descriptive statistics to summarize the frequency of strategy use “overall” (in any study arm), and the mean number of strategies reported in the Control and Experimental Arms. We additionally described the strategies that were only in the experimental (and not control) arm, defining these as strategies that were “tested” and may have accounted for differences in outcomes between arms.

We described frequencies of pair-wise combinations of implementation strategies in the Experimental Arm. To assess the strength of the evidence supporting implementation strategies that were used in the Experimental Arm, study outcomes were categorized by RE-AIM and coded based on whether the association between use of the strategies resulted in a significantly positive effect (yes=1; no=0). We then created an indicator variable if at least one RE-AIM outcome in the study was significantly positive (yes=1; no=0). We plotted strategies on a graph with quadrants based on the combination of median number of studies in which a strategy appears and the median percent of studies in which a strategy was associated with at least one positive RE-AIM outcome. The upper right quadrant—higher number of studies overall and higher percent of studies with a significant RE-AIM outcome—represents a superior level of evidence. For implementation strategies in the upper right quadrant, we describe each RE-AIM outcome and the proportion of studies which have a significant outcome.

Search results

We identified 14,646 articles through the initial literature search, 17 articles through expert recommendation (three of which were not included in the initial search), and 1,942 articles through reviewing prior systematic reviews (Fig. 1 ). After removing duplicates, 9,399 articles were included in the initial abstract screening. Of those, 48% ( n =4,075) abstracts were reviewed in pairs for inclusion. Articles with a score of five or six were reviewed a second time ( n =2,859). One quarter of abstracts that scored lower than five were reviewed for a second time at random. We screened the full text of 1,426 articles in pairs. Common reasons for exclusion were 1) study rigor, including no clear delineation between the EBP and implementation strategy, 2) not testing an implementation strategy, and 3) article type that did not meet inclusion criteria (e.g., commentary, protocol, etc.). Six hundred seventeen articles were reviewed for study rigor with 385 excluded for reasons related to study design and rigor, and 86 removed for other reasons (e.g., not a research article). Among the three additional expert-recommended articles, one met inclusion criteria and was added to the analysis. The final number of studies abstracted was 129 representing 143 publications.

Expanded PRISMA Flow Diagram

The expanded PRISMA flow diagram provides a description of each step in the review and abstraction process for the systematic review

Descriptive results

Of 129 included studies (Table 1 ; see also Additional File 5 for Summary of Included Studies), 103 (79%) were conducted in a healthcare setting. EBP health care setting varied and included primary care ( n =46; 36%), specialty care ( n =27; 21%), mental health ( n =11; 9%), and public health ( n =30; 23%), with 64 studies (50%) occurring in an outpatient health care setting. Studies included a median of 29 sites and 1,419 target population (e.g., patients or students). The number of strategies varied widely across studies, with Control Arms averaging approximately two strategies (Range = 0-20, including studies with no strategy in the comparison group) and Experimental Arms averaging eight strategies (Range = 1-21). Non-US studies ( n =73) included more sites and target population on average, with an overall median of 32 sites and 1,531 patients assessed in each study.

Organized by RE-AIM, the most evaluated outcomes were Effectiveness ( n = 82, 64%) and Implementation ( n = 73, 56%); followed by Maintenance ( n =40; 31%), Adoption ( n =33; 26%), and Reach ( n =31; 24%). Most studies ( n = 98, 76%) reported at least one significantly positive outcome. Adoption and Implementation outcomes showed positive change in three-quarters of studies ( n =78), while Reach ( n =18; 58%), Effectiveness ( n =44; 54%), and Maintenance ( n =23; 58%) outcomes evidenced positive change in approximately half of studies.

The following describes the results for each research question.

Table 2 shows the frequency of studies within which an implementation strategy was used in the Control Arm, Experimental Arm(s), and tested strategies (those used exclusively in the Experimental Arm) grouped by strategy type, as specified by previous ERIC reports [ 2 , 6 ].

Control arm

In about half the studies (53%; n =69), the Control Arms were “active controls” that included at least one strategy, with an average of 1.64 (and up to 20) strategies reported in control arms. The two most common strategies used in Control Arms were: Distribute Educational Materials ( n =52) and Conduct Educational Meetings ( n =30).

Experimental arm

Experimental conditions included an average of 8.33 implementation strategies per study (Range = 1-21). Figure 2 shows a heat map of the strategies that were used in the Experimental Arms in each study. The most common strategies in the Experimental Arm were Distribute Educational Materials ( n =99), Conduct Educational Meetings ( n =96), Audit and Provide Feedback ( n =76), and External Facilitation ( n =59).

Implementation strategies used in the Experimental Arm of included studies. Explore more here: https://public.tableau.com/views/Figure2_16947070561090/Figure2?:language=en-US&:display_count=n&:origin=viz_share_link

Tested strategies

The average number of implementation strategies that were included in the Experimental Arm only (and not in the Control Arm) was 6.73 (Range = 0-20). Footnote 2 Overall, the top 10% of tested strategies included Conduct Educational Meetings ( n =68), Audit and Provide Feedback ( n =63), External Facilitation ( n =54), Distribute Educational Materials ( n =49), Tailor Strategies ( n =41), Assess for Readiness and Identify Barriers and Facilitators ( n =38) and Organize Clinician Implementation Team Meetings ( n =37). Few studies tested a single strategy ( n =9). These strategies included, Audit and Provide Feedback, Conduct Educational Meetings, Conduct Ongoing Training, Create a Learning Collaborative, External Facilitation ( n =2), Facilitate Relay of Clinical Data To Providers, Prepare Patients/Consumers to be Active Participants, and Use Other Payment Schemes. Three implementation strategies were included in the Control or Experimental Arms but were not Tested including, Use Mass Media, Stage Implementation Scale Up, and Fund and Contract for the Clinical Innovation.

Table 3  shows the five most used strategies in Experimental Arms with their top ten most frequent pairings, excluding Distribute Educational Materials and Conduct Educational Meetings, as these strategies were included in almost all Experimental and half of Control Arms. The five most used strategies in the Experimental Arm included Audit and Provide Feedback ( n =76), External Facilitation ( n =59), Tailor Strategies ( n =43), Assess for Readiness and Identify Barriers and Facilitators ( n =43), and Organize Implementation Teams ( n =42).

Strategies frequently paired with these five strategies included two educational strategies: Distribute Educational Materials and Conduct Educational Meetings. Other commonly paired strategies included Develop a Formal Implementation Blueprint, Promote Adaptability, Conduct Ongoing Training, Purposefully Reexamine the Implementation, and Develop and Implement Tools for Quality Monitoring.

We classified the strength of evidence for each strategy by evaluating both the number of studies in which each strategy appeared in the Experimental Arm and the percentage of times there was at least one significantly positive RE-AIM outcome. Using these factors, Fig. 3 shows the number of studies in which individual strategies were evaluated (on the y axis) compared to the percentage of times that studies including those strategies had at least one positive outcome (on the x axis). Due to the non-normal distribution of both factors, we used the median (rather than the mean) to create four quadrants. Strategies in the lower left quadrant were tested in fewer than the median number of studies (8.5) and were less frequently associated with a significant RE-AIM outcome (75%). The upper right quadrant included strategies that occurred in more than the median number of studies (8.5) and had more than the median percent of studies with a significant RE-AIM outcome (75%); thus those 19 strategies were viewed as having stronger evidence. Of those 19 implementation strategies, Conduct Educational Meetings, Distribute Educational Materials, External Facilitation, and Audit and Provide Feedback continued to occur frequently, appearing in 59-99 studies.

Experimental Arm Implementation Strategies with significant RE-AIM outcome. Explore more here: https://public.tableau.com/views/Figure3_16947017936500/Figure3?:language=en-US&publish=yes&:display_count=n&:origin=viz_share_link

Figure 4 graphically illustrates the proportion of significant outcomes for each RE-AIM outcome for the 19 commonly used and evidence-based implementation strategies in the upper right quadrant. These findings again show the widespread use of Conduct Educational Meetings and Distribute Educational Materials. Implementation and Effectiveness outcomes were assessed most frequently, with Implementation being the mostly commonly reported significantly positive outcome.

RE-AIM outcomes for the 19 Top-Right Quadrant Implementation Strategies . The y-axis is the number of studies and the x-axis is a stacked bar chart for each RE-AIM outcome with R=Reach, E=Effectiveness, A=Adoption, I=Implementation, M=Maintenance. Blue denotes at least one significant RE-AIM outcome; Light blue denotes studies which used the given implementation strategy and did not have a significant RE-AIM . Explore more here: https://public.tableau.com/views/Figure4_16947017112150/Figure4?:language=en-US&publish=yes&:display_count=n&:origin=viz_share_link

This systematic review identified 129 experimental studies examining the effectiveness of implementation strategies across a broad range of health and human service studies. Overall, we found that evidence is lacking for most ERIC implementation strategies, that most studies employed combinations of strategies, and that implementation outcomes, categorized by RE-AIM dimensions, have not been universally defined or applied. Accordingly, other researchers have described the need for universal outcomes definitions and descriptions across implementation research studies [ 28 , 42 ]. Our findings have important implications not only for the current state of the field but also for creating guidance to help investigators determine which strategies and in what context to examine.

The four most evaluated strategies were Distribute Educational Materials, Conduct Educational Meetings, External Facilitation, and Audit and Provide Feedback. Conducting Educational Meetings and Distributing Educational Materials were surprisingly the most common. This may reflect the fact that education strategies are generally considered to be “necessary but not sufficient” for successful implementation [ 43 , 44 ]. Because education is often embedded in interventions, it is critical to define the boundary between the innovation and the implementation strategies used to support the innovation. Further specification as to when these strategies are EBP core components or implementation strategies (e.g., booster trainings or remediation) is needed [ 45 , 46 ].

We identified 19 implementation strategies that were tested in at least 8 studies (more than the median) and were associated with positive results at least 75% of the time. These strategies can be further categorized as being used in early or pre-implementation versus later in implementation. Preparatory activities or pre-implementation, strategies that had strong evidence included educational activities (Meetings, Materials, Outreach visits, Train for Leadership, Use Train the Trainer Strategies) and site diagnostic activities (Assess for Readiness, Identify Barriers and Facilitators, Conduct Local Needs Assessment, Identify and Prepare Champions, and Assess and Redesign Workflows). Strategies that target the implementation phase include those that provide coaching and support (External and Internal Facilitation), involve additional key partners (Intervene with Patients to Enhance Uptake and Adherence), and engage in quality improvement activities (Audit and Provide Feedback, Facilitate the Relay of Clinical Data to Providers, Purposefully Reexamine the Implementation, Conduct Cyclical Small Tests of Change, Develop and Implement Tools for Quality Monitoring).

There were many ERIC strategies that were not represented in the reviewed studies, specifically the financial and policy strategies. Ten strategies were not used in any studies, including: Alter Patient/Consumer Fees, Change Liability Laws, Change Service Sites, Develop Disincentives, Develop Resource Sharing Agreements, Identify Early Adopters, Make Billing Easier, Start a Dissemination Organization, Use Capitated Payments, and Use Data Experts. One of the limitations of this investigation was that not all individual strategies or combinations were investigated. Reasons for the absence of these strategies in our review may include challenges with testing certain strategies experimentally (e.g., changing liability laws), limitations in our search terms, and the relative paucity of implementation strategy trials compared to clinical trials. Many “untested” strategies require large-scale structural changes with leadership support (see [ 47 ] for policy experiment example). Recent preliminary work has assessed the feasibility of applying policy strategies and described the challenges with doing so [ 48 , 49 , 50 ]. While not impossible in large systems like VA (for example: the randomized evaluation of the VA Stratification Tool for Opioid Risk Management) the large size, structure, and organizational imperative makes these initiatives challenging to experimentally evaluate. Likewise, the absence of these ten strategies may have been the result of our inclusion criteria, which required an experimental design. Thus, creative study designs may be needed to test high-level policy or financial strategies experimentally.

Some strategies that were likely under-represented in our search strategy included electronic medical record reminders and clinical decision support tools and systems. These are often considered “interventions” when used by clinical trialists and may not be indexed as studies involving ‘implementation strategies’ (these tools have been reviewed elsewhere [ 51 , 52 , 53 ]). Thus, strategies that are also considered interventions in the literature (e.g., education interventions) were not sought or captured. Our findings do not imply that these strategies are ineffective, rather that more study is needed. Consistent with prior investigations [ 54 ], few studies meeting inclusion criteria tested financial strategies. Accordingly, there are increasing calls to track and monitor the effects of financial strategies within implementation science to understand their effectiveness in practice [ 55 , 56 ]. However, experts have noted that the study of financial strategies can be a challenge given that they are typically implemented at the system-level and necessitate research designs for studying policy-effects (e.g., quasi-experimental methods, systems-science modeling methods) [ 57 ]. Yet, there have been some recent efforts to use financial strategies to support EBPs that appear promising [ 58 ] and could be a model for the field moving forward.

The relationship between the number of strategies used and improved outcomes has been described inconsistently in the literature. While some studies have found improved outcomes with a bundle of strategies that were uniquely combined or a standardized package of strategies (e.g., Replicating Effective Programs [ 59 , 60 ] and Getting To Outcomes [ 61 , 62 ]), others have found that “more is not always better” [ 63 , 64 , 65 ]. For example, Rogal and colleagues documented that VA hospitals implementing a new evidence-based hepatitis C treatment chose >20 strategies, when multiple years of data linking strategies to outcomes showed that 1-3 specific strategies would have yielded the same outcome [ 39 ]. Considering that most studies employed multiple or multifaceted strategies, it seems that there is a benefit of using a targeted bundle of strategies that are purposefully aligns with site/clinic/population norms, rather than simply adding more strategies [ 66 ].

It is difficult to assess the effectiveness of any one implementation strategy in bundles where multiple strategies are used simultaneously. Even a ‘single’ strategy like External Facilitation is, in actuality, a bundle of narrowly constructed strategies (e.g., Conduct Educational Meetings, Identify and Prepare Champions, and Develop a Formal Implementation Blueprint). Thus, studying External Facilitation does not allow for a test of the individual strategies that comprise it, potentially masking the effectiveness of any individual strategy. While we cannot easily disaggregate the effects of multifaceted strategies, doing so may not yield meaningful results. Because strategies often synergize, disaggregated results could either underestimate the true impact of individual strategies or conversely, actually undermine their effectiveness (i.e., when their effectiveness comes from their combination with other strategies). The complexity of health and human service settings, imperative to improve public health outcomes, and engagement with community partners often requires the use of multiple strategies simultaneously. Therefore, the need to improve real-world implementation may outweigh the theoretical need to identify individual strategy effectiveness. In situations where it would be useful to isolate the impact of single strategies, we suggest that the same methods for documenting and analyzing the critical components (or core functions) of complex interventions [ 67 , 68 , 69 , 70 ] may help to identify core components of multifaceted implementation strategies [ 71 , 72 , 73 , 74 ].

In addition, to truly assess the impacts of strategies on outcomes, it may be necessary to track fidelity to implementation strategies (not just the EBPs they support). While this can be challenging, without some degree of tracking and fidelity checks, one cannot determine whether a strategy’s apparent failure to work was because it 1) was ineffective or 2) was not applied well. To facilitate this tracking there are pragmatic tools to support researchers. For example, the Longitudinal Implementation Strategy Tracking System (LISTS) offers a pragmatic and feasible means to assess fidelity to and adaptations of strategies [ 75 ].

Implications for implementation science: four recommendations

Based on our findings, we offer four recommended “best practices” for implementation studies.

Prespecify strategies using standard nomenclature. This study reaffirmed the need to apply not only a standard naming convention (e.g., ERIC) but also a standard reporting of for implementation strategies. While reporting systems like those by Proctor [ 1 ] or Pinnock [ 75 ] would optimize learning across studies, few manuscripts specify strategies as recommended [ 76 , 77 ]. Pre-specification allows planners and evaluators to assess the feasibility and acceptability of strategies with partners and community members [ 24 , 78 , 79 ] and allows evaluators and implementers to monitor and measure the fidelity, dose, and adaptations to strategies delivered over the course of implementation [ 27 ]. In turn, these data can be used to assess the costs, analyze their effectiveness [ 38 , 80 , 81 ], and ensure more accurate reporting [ 82 , 83 , 84 , 85 ]. This specification should include, among other data, the intensity, stage of implementation, and justification for the selection. Information regarding why strategies were selected for specific settings would further the field and be of great use to practitioners. [ 63 , 65 , 69 , 79 , 86 ].

Ensure that standards for measuring and reporting implementation outcomes are consistently applied and account for the complexity of implementation studies. Part of improving standardized reporting must include clearly defining outcomes and linking each outcome to particular implementation strategies. It was challenging in the present review to disentangle the impact of the intervention(s) (i.e., the EBP) versus the impact of the implementation strategy(ies) for each RE-AIM dimension. For example, often fidelity to the EBP was reported but not for the implementation strategies. Similarly, Reach and Adoption of the intervention would be reported for the Experimental Arm but not for the Control Arm, prohibiting statistical comparisons of strategies on the relative impact of the EBP between study arms. Moreover, there were many studies evaluating numerous outcomes, risking data dredging. Further, the significant heterogeneity in the ways in which implementation outcomes are operationalized and reported is a substantial barrier to conducting large-scale meta-analytic approaches to synthesizing evidence for implementation strategies [ 67 ]. The field could look to others in the social and health sciences for examples in how to test, validate, and promote a common set of outcome measures to aid in bringing consistency across studies and real-world practice (e.g., the NIH-funded Patient-Reported Outcomes Measurement Information System [PROMIS], https://www.healthmeasures.net/explore-measurement-systems/promis ).

Develop infrastructure to learn cross-study lessons in implementation science. Data repositories, like those developed by NCI for rare diseases, U.S. HIV Implementation Science Coordination Initiative [ 87 ], and the Behavior Change Technique Ontology [ 88 ], could allow implementation scientists to report their findings in a more standardized manner, which would promote ease of communication and contextualization of findings across studies. For example, the HIV Implementation Science Coordination Initiative requested all implementation projects use common frameworks, developed user friendly databases to enable practitioners to match strategies to determinants, and developed a dashboard of studies that assessed implementation determinants [ 89 , 90 , 91 , 92 , 93 , 94 ].

Develop and apply methods to rigorously study common strategies and bundles. These findings support prior recommendations for improved empirical rigor in implementation studies [ 46 , 95 ]. Many studies were excluded from our review based on not meeting methodological rigor standards. Understanding the effectiveness of discrete strategies deployed alone or in combination requires reliable and low burden tracking methods to collect information about strategy use and outcomes. For example, frameworks like the Implementation Replication Framework [ 96 ] could help interpret findings across studies using the same strategy bundle. Other tracking approaches may leverage technology (e.g., cell phones, tablets, EMR templates) [ 78 , 97 ] or find novel, pragmatic approaches to collect recommended strategy specifications over time (e.g.., dose, deliverer, and mechanism) [ 1 , 9 , 27 , 98 , 99 ]. Rigorous reporting standards could inform more robust analyses and conclusions (e.g., moving toward the goal of understanding causality, microcosting efforts) [ 24 , 38 , 100 , 101 ]. Such detailed tracking is also required to understand how site-level factors moderate implementation strategy effects [ 102 ]. In some cases, adaptive trial designs like sequential multiple assignment randomized trials (SMARTs) and just-in-time adaptive interventions (JITAIs) can be helpful for planning strategy escalation.

Limitations

Despite the strengths of this review, there were certain notable limitations. For one, we only included experimental studies, omitting many informative observational investigations that cover the range of implementation strategies. Second, our study period was centered on the creation of the journal Implementation Science and not on the standardization and operationalization of implementation strategies in the publication of the ERIC taxonomy (which came later). This, in conjunction with latency in reporting study results and funding cycles, means that the employed taxonomy was not applied in earlier studies. To address this limitation, we retroactively mapped strategies to ERIC, but it is possible that some studies were missed. Additionally, indexing approaches used by academic databases may have missed relevant studies. We addressed this particular concern by reviewing other systematic reviews of implementation strategies and soliciting recommendations from global implementation science experts.

Another potential limitation comes from the ERIC taxonomy itself—i.e., strategy listings like ERIC are only useful when they are widely adopted and used in conjunction with guidelines for specifying and reporting strategies [ 1 ] in protocol and outcome papers. Although the ERIC paper has been widely cited (over three thousand times, accessed about 186 thousand times), it is still not universally applied, making tracking the impact of specific strategies more difficult. However, our experience with this review seemed to suggest that ERIC’s use was increasing over time. Also, some have commented that ERIC strategies can be unclear and are missing key domains. Thus, researchers are making definitions clearer for lay users [ 37 , 103 ], increasing the number of discrete strategies for specific domains like HIV treatment, acknowledging strategies for new functions (e.g., de-implementation [ 104 ], local capacity building), accounting for phases of implementation (dissemination, sustainment [ 13 ], scale-up), addressing settings [ 12 , 20 ], actors roles in the process, and making mechanisms of change to select strategies more user-friendly through searchable databases [ 9 , 10 , 54 , 73 , 104 , 105 , 106 ]. In sum, we found the utility of the ERIC taxonomy to outweigh any of the taxonomy’s current limitations.

As with all reviews, the search terms influenced our findings. As such, the broad terms for implementation strategies (e.g., “evidence-based interventions”[ 7 ] or “behavior change techniques” [ 107 ]) may have led to inadvertent omissions of studies of specific strategies. For example, the search terms may not have captured tests of policies, financial strategies, community health promotion initiatives, or electronic medical record reminders, due to differences in terminology used in corresponding subfields of research (e.g., health economics, business, health information technology, and health policy). To manage this, we asked experts to inform us about any studies that they would include and cross-checked their lists with what was identified through our search terms, which yielded very few additional studies. We included standard coding using the ERIC taxonomy, which was a strength, but future work should consider including the additional strategies that have been recommended to augment ERIC, around sustainment [ 13 , 79 , 106 , 108 ], community and public health research [ 12 , 109 , 110 , 111 ], consumer or service user engagement [ 112 ], de-implementation [ 104 , 113 , 114 , 115 , 116 , 117 ] and related terms [ 118 ].

We were unable to assess the bias of studies due to non-standard reporting across the papers and the heterogeneity of study designs, measurement of implementation strategies and outcomes, and analytic approaches. This could have resulted in over- or underestimating the results of our synthesis. We addressed this limitation by being cautious in our reporting of findings, specifically in identifying “effective” implementation strategies. Further, we were not able to gather primary data to evaluate effect sizes across studies in order to systematically evaluate bias, which would be fruitful for future study.

Conclusions

This novel review of 129 studies summarized the body of evidence supporting the use of ERIC-defined implementation strategies to improve health or healthcare. We identified commonly occurring implementation strategies, frequently used bundles, and the strategies with the highest degree of supportive evidence, while simultaneously identifying gaps in the literature. Additionally, we identified several key areas for future growth and operationalization across the field of implementation science with the goal of improved reporting and assessment of implementation strategies and related outcomes.

Availability and materials

All data for this study are included in this published article and its supplementary information files.

We modestly revised the following research questions from our PROSPERO registration after reading the articles and better understanding the nature of the literature: 1) What is the available evidence regarding the effectiveness of implementation strategies in supporting the uptake and sustainment of evidence intended to improve health and healthcare outcomes? 2) What are the current gaps in the literature (i.e., implementation strategies that do not have sufficient evidence of effectiveness) that require further exploration?

Tested strategies are those which exist in the Experimental Arm but not in the Control Arm. Comparative effectiveness or time staggered trials may not have any unique strategies in the Experimental Arm and therefore in our analysis would have no Tested Strategies.

Abbreviations

Centers for Disease Control

Cumulated Index to Nursing and Allied Health Literature

Dissemination and Implementation

Evidence-based practices or programs

Expert Recommendations for Implementing Change

Multiphase Optimization Strategy

National Cancer Institute

National Institutes of Health

The Pittsburgh Dissemination and Implementation Science Collaborative

Sequential Multiple Assignment Randomized Trial

United States

Department of Veterans Affairs

Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139.

Article   PubMed   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

Waltz TJ, Powell BJ, Chinman MJ, Smith JL, Matthieu MM, Proctor EK, et al. Expert recommendations for implementing change (ERIC): protocol for a mixed methods study. Implement Sci IS. 2014;9:39.

Article   PubMed   Google Scholar  

Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A Compilation of Strategies for Implementing Clinical Innovations in Health and Mental Health. Med Care Res Rev. 2012;69:123–57.

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:109.

Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ. Specifying and comparing implementation strategies across seven large implementation interventions: a practical application of theory. Implement Sci. 2019;14(1):32.

Community Preventive Services Task Force. Community Preventive Services Task Force: All Active Findings June 2023 [Internet]. 2023 [cited 2023 Aug 7]. Available from: https://www.thecommunityguide.org/media/pdf/CPSTF-All-Findings-508.pdf

Solberg LI, Kuzel A, Parchman ML, Shelley DR, Dickinson WP, Walunas TL, et al. A Taxonomy for External Support for Practice Transformation. J Am Board Fam Med JABFM. 2021;34:32–9.

Leeman J, Birken SA, Powell BJ, Rohweder C, Shea CM. Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. Implement Sci. 2017;12:1–9.

Article   Google Scholar  

Leeman J, Calancie L, Hartman MA, Escoffery CT, Herrmann AK, Tague LE, et al. What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective?: a systematic review. Implement Sci. 2015;10:1–15.

Nathan N, Shelton RC, Laur CV, Hailemariam M, Hall A. Editorial: Sustaining the implementation of evidence-based interventions in clinical and community settings. Front Health Serv. 2023;3:1176023.

Balis LE, Houghtaling B, Harden SM. Using implementation strategies in community settings: an introduction to the Expert Recommendations for Implementing Change (ERIC) compilation and future directions. Transl Behav Med. 2022;12:965–78.

Nathan N, Powell BJ, Shelton RC, Laur CV, Wolfenden L, Hailemariam M, et al. Do the Expert Recommendations for Implementing Change (ERIC) strategies adequately address sustainment? Front Health Serv. 2022;2:905909.

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.

Google Scholar  

Moore L, Guertin JR, Tardif P-A, Ivers NM, Hoch J, Conombo B, et al. Economic evaluations of audit and feedback interventions: a systematic review. BMJ Qual Saf. 2022;31:754–67.

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35.

Barnes C, McCrabb S, Stacey F, Nathan N, Yoong SL, Grady A, et al. Improving implementation of school-based healthy eating and physical activity policies, practices, and programs: a systematic review. Transl Behav Med. 2021;11:1365–410.

Tomasone JR, Kauffeldt KD, Chaudhary R, Brouwers MC. Effectiveness of guideline dissemination and implementation strategies on health care professionals’ behaviour and patient outcomes in the cancer care context: a systematic review. Implement Sci. 2020;15:1–18.

Seda V, Moles RJ, Carter SR, Schneider CR. Assessing the comparative effectiveness of implementation strategies for professional services to community pharmacy: A systematic review. Res Soc Adm Pharm. 2022;18:3469–83.

Lovero KL, Kemp CG, Wagenaar BH, Giusto A, Greene MC, Powell BJ, et al. Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review. Implement Sci. 2023;18:56.

Chapman A, Rankin NM, Jongebloed H, Yoong SL, White V, Livingston PM, et al. Overcoming challenges in conducting systematic reviews in implementation science: a methods commentary. Syst Rev. 2023;12:1–6.

Article   CAS   Google Scholar  

Proctor EK, Bunger AC, Lengnick-Hall R, Gerke DR, Martin JK, Phillips RJ, et al. Ten years of implementation outcomes research: a scoping review. Implement Sci. 2023;18:1–19.

Michaud TL, Pereira E, Porter G, Golden C, Hill J, Kim J, et al. Scoping review of costs of implementation strategies in community, public health and healthcare settings. BMJ Open. 2022;12:e060785.

Sohn H, Tucker A, Ferguson O, Gomes I, Dowdy D. Costing the implementation of public health interventions in resource-limited settings: a conceptual framework. Implement Sci. 2020;15:1–8.

Peek C, Glasgow RE, Stange KC, Klesges LM, Purcell EP, Kessler RS. The 5 R’s: an emerging bold standard for conducting relevant research in a changing world. Ann Fam Med. 2014;12:447–55.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7.

Shelton RC, Chambers DA, Glasgow RE. An Extension of RE-AIM to Enhance Sustainability: Addressing Dynamic Context and Promoting Health Equity Over Time. Front Public Health. 2020;8:134.

Holtrop JS, Estabrooks PA, Gaglio B, Harden SM, Kessler RS, King DK, et al. Understanding and applying the RE-AIM framework: Clarifications and resources. J Clin Transl Sci. 2021;5:e126.

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ [Internet]. 2021;372. Available from: https://www.bmj.com/content/372/bmj.n71

Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A Glossary for Dissemination and Implementation Research in Health. J Public Health Manag Pract. 2008;14:117–23.

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1.

Article   PubMed Central   Google Scholar  

Miller WR, Wilbourne PL. Mesa Grande: a methodological analysis of clinical trials of treatments for alcohol use disorders. Addict Abingdon Engl. 2002;97:265–77.

Miller WR, Brown JM, Simpson TL, Handmaker NS, Bien TH, Luckie LF, et al. What works? A methodological analysis of the alcohol treatment outcome literature. Handb Alcohol Treat Approaches Eff Altern 2nd Ed. Needham Heights, MA, US: Allyn & Bacon; 1995:12–44.

Wells S, Tamir O, Gray J, Naidoo D, Bekhit M, Goldmann D. Are quality improvement collaboratives effective? A systematic review BMJ Qual Saf. 2018;27:226–40.

Yakovchenko V, Chinman MJ, Lamorte C, Powell BJ, Waltz TJ, Merante M, et al. Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers. Implement Sci Commun. 2023;4:1–14.

Wagner TH, Yoon J, Jacobs JC, So A, Kilbourne AM, Yu W, et al. Estimating costs of an implementation intervention. Med Decis Making. 2020;40:959–67.

Gold HT, McDermott C, Hoomans T, Wagner TH. Cost data in implementation science: categories and approaches to costing. Implement Sci. 2022;17:11.

Boutron I, Page MJ, Higgins JP, Altman DG, Lundh A, Hróbjartsson A. Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions. 2019. https://doi.org/10.1002/9781119536604.ch7 . 

Higgins JP, Savović J, Page MJ, Elbers RG, Sterne J. Assessing risk of bias in a randomized trial. Cochrane Handb Syst Rev Interv. 2019;6:205–28.

Reilly KL, Kennedy S, Porter G, Estabrooks P. Comparing, Contrasting, and Integrating Dissemination and Implementation Outcomes Included in the RE-AIM and Implementation Outcomes Frameworks. Front Public Health [Internet]. 2020 [cited 2024 Apr 24];8. Available from: https://www.frontiersin.org/journals/public-health/articles/ https://doi.org/10.3389/fpubh.2020.00430/full

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess Winch Engl. 2004;8:iii–iv 1-72.

CAS   Google Scholar  

Beidas RS, Kendall PC. Training Therapists in Evidence-Based Practice: A Critical Review of Studies From a Systems-Contextual Perspective. Clin Psychol Publ Div Clin Psychol Am Psychol Assoc. 2010;17:1–30.

Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to Improve the Selection and Tailoring of Implementation Strategies. J Behav Health Serv Res. 2017;44:177–94.

Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health [Internet]. 2019 [cited 2021 Mar 31];7. Available from: https://www.frontiersin.org/articles/ https://doi.org/10.3389/fpubh.2019.00003/full

Frakt AB, Prentice JC, Pizer SD, Elwy AR, Garrido MM, Kilbourne AM, et al. Overcoming Challenges to Evidence-Based Policy Development in a Large. Integrated Delivery System Health Serv Res. 2018;53:4789–807.

PubMed   Google Scholar  

Crable EL, Lengnick-Hall R, Stadnick NA, Moullin JC, Aarons GA. Where is “policy” in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example. Implement Sci. 2022;17:80.

Crable EL, Grogan CM, Purtle J, Roesch SC, Aarons GA. Tailoring dissemination strategies to increase evidence-informed policymaking for opioid use disorder treatment: study protocol. Implement Sci Commun. 2023;4:16.

Bond GR. Evidence-based policy strategies: A typology. Clin Psychol Sci Pract. 2018;25:e12267.

Loo TS, Davis RB, Lipsitz LA, Irish J, Bates CK, Agarwal K, et al. Electronic Medical Record Reminders and Panel Management to Improve Primary Care of Elderly Patients. Arch Intern Med. 2011;171:1552–8.

Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J. Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ Can Med Assoc J. 2010;182:E216-25.

Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al. A Randomized Trial of Electronic Clinical Reminders to Improve Quality of Care for Diabetes and Coronary Artery Disease. J Am Med Inform Assoc JAMIA. 2005;12:431–7.

Dopp AR, Kerns SEU, Panattoni L, Ringel JS, Eisenberg D, Powell BJ, et al. Translating economic evaluations into financing strategies for implementing evidence-based practices. Implement Sci. 2021;16:1–12.

Dopp AR, Hunter SB, Godley MD, Pham C, Han B, Smart R, et al. Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study. Implement Sci Commun. 2022;3:51.

Proctor EK, Toker E, Tabak R, McKay VR, Hooley C, Evanoff B. Market viability: a neglected concept in implementation science. Implement Sci. 2021;16:98.

Dopp AR, Narcisse M-R, Mundey P, Silovsky JF, Smith AB, Mandell D, et al. A scoping review of strategies for financing the implementation of evidence-based practices in behavioral health systems: State of the literature and future directions. Implement Res Pract. 2020;1:2633489520939980.

PubMed   PubMed Central   Google Scholar  

Dopp AR, Kerns SEU, Panattoni L, Ringel JS, Eisenberg D, Powell BJ, et al. Translating economic evaluations into financing strategies for implementing evidence-based practices. Implement Sci IS. 2021;16:66.

Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care:application of the replicating effective programs framework. Implement Sci. 2007;2:42–51.

Kegeles SM, Rebchook GM, Hays RB, Terry MA, O’Donnell L, Leonard NR, et al. From science to application: the development of an intervention package. AIDS Educ Prev Off Publ Int Soc AIDS Educ. 2000;12:62–74.

Wandersman A, Imm P, Chinman M, Kaftarian S. Getting to outcomes: a results-based approach to accountability. Eval Program Plann. 2000;23:389–95.

Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol. 2012;50:445–59.

Rogal SS, Yakovchenko V, Waltz TJ, Powell BJ, Kirchner JE, Proctor EK, et al. The association between implementation strategy use and the uptake of hepatitis C treatment in a national sample. Implement Sci. 2017;12:1–13.

Smith SN, Almirall D, Prenovost K, Liebrecht C, Kyle J, Eisenberg D, et al. Change in patient outcomes after augmenting a low-level implementation strategy in community practices that are slow to adopt a collaborative chronic care model: a cluster randomized implementation trial. Med Care. 2019;57:503.

Rogal SS, Yakovchenko V, Waltz TJ, Powell BJ, Gonzalez R, Park A, et al. Longitudinal assessment of the association between implementation strategy use and the uptake of hepatitis C treatment: Year 2. Implement Sci. 2019;14:1–12.

Harvey G, Kitson A. Translating evidence into healthcare policy and practice: Single versus multi-faceted implementation strategies – is there a simple answer to a complex question? Int J Health Policy Manag. 2015;4:123–6.

Engell T, Stadnick NA, Aarons GA, Barnett ML. Common Elements Approaches to Implementation Research and Practice: Methods and Integration with Intervention Science. Glob Implement Res Appl. 2023;3:1–15.

Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci IS. 2009;4:40.

Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci IS. 2020;15:84.

Perez Jolles M, Lengnick-Hall R, Mittman BS. Core Functions and Forms of Complex Health Interventions: a Patient-Centered Medical Home Illustration. JGIM J Gen Intern Med. 2019;34:1032–8.

Schroeck FR, Ould Ismail AA, Haggstrom DA, Sanchez SL, Walker DR, Zubkoff L. Data-driven approach to implementation mapping for the selection of implementation strategies: a case example for risk-aligned bladder cancer surveillance. Implement Sci IS. 2022;17:58.

Frank HE, Kemp J, Benito KG, Freeman JB. Precision Implementation: An Approach to Mechanism Testing in Implementation Research. Adm Policy Ment Health. 2022;49:1084–94.

Lewis CC, Klasnja P, Lyon AR, Powell BJ, Lengnick-Hall R, Buchanan G, et al. The mechanics of implementation strategies and measures: advancing the study of implementation mechanisms. Implement Sci Commun. 2022;3:114.

Geng EH, Baumann AA, Powell BJ. Mechanism mapping to advance research on implementation strategies. PLoS Med. 2022;19:e1003918.

Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ. 2017;356:i6795.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38:65–76.

Hooley C, Amano T, Markovitz L, Yaeger L, Proctor E. Assessing implementation strategy reporting in the mental health literature: a narrative review. Adm Policy Ment Health Ment Health Serv Res. 2020;47:19–35.

Proctor E, Ramsey AT, Saldana L, Maddox TM, Chambers DA, Brownson RC. FAST: a framework to assess speed of translation of health innovations to practice and policy. Glob Implement Res Appl. 2022;2:107–19.

Cullen L, Hanrahan K, Edmonds SW, Reisinger HS, Wagner M. Iowa Implementation for Sustainability Framework. Implement Sci IS. 2022;17:1.

Saldana L, Ritzwoller DP, Campbell M, Block EP. Using economic evaluations in implementation science to increase transparency in costs and outcomes for organizational decision-makers. Implement Sci Commun. 2022;3:40.

Eisman AB, Kilbourne AM, Dopp AR, Saldana L, Eisenberg D. Economic evaluation in implementation science: making the business case for implementation strategies. Psychiatry Res. 2020;283:112433.

Akiba CF, Powell BJ, Pence BW, Nguyen MX, Golin C, Go V. The case for prioritizing implementation strategy fidelity measurement: benefits and challenges. Transl Behav Med. 2022;12:335–42.

Akiba CF, Powell BJ, Pence BW, Muessig K, Golin CE, Go V. “We start where we are”: a qualitative study of barriers and pragmatic solutions to the assessment and reporting of implementation strategy fidelity. Implement Sci Commun. 2022;3:117.

Rudd BN, Davis M, Doupnik S, Ordorica C, Marcus SC, Beidas RS. Implementation strategies used and reported in brief suicide prevention intervention studies. JAMA Psychiatry. 2022;79:829–31.

Painter JT, Raciborski RA, Matthieu MM, Oliver CM, Adkins DA, Garner KK. Engaging stakeholders to retrospectively discern implementation strategies to support program evaluation: Proposed method and case study. Eval Program Plann. 2024;103:102398.

Bunger AC, Powell BJ, Robertson HA, MacDowell H, Birken SA, Shea C. Tracking implementation strategies: a description of a practical approach and early findings. Health Res Policy Syst. 2017;15:1–12.

Mustanski B, Smith JD, Keiser B, Li DH, Benbow N. Supporting the growth of domestic HIV implementation research in the united states through coordination, consultation, and collaboration: how we got here and where we are headed. JAIDS J Acquir Immune Defic Syndr. 2022;90:S1-8.

Marques MM, Wright AJ, Corker E, Johnston M, West R, Hastings J, et al. The Behaviour Change Technique Ontology: Transforming the Behaviour Change Technique Taxonomy v1. Wellcome Open Res. 2023;8:308.

Merle JL, Li D, Keiser B, Zamantakis A, Queiroz A, Gallo CG, et al. Categorising implementation determinants and strategies within the US HIV implementation literature: a systematic review protocol. BMJ Open. 2023;13:e070216.

Glenshaw MT, Gaist P, Wilson A, Cregg RC, Holtz TH, Goodenow MM. Role of NIH in the Ending the HIV Epidemic in the US Initiative: Research Improving Practice. J Acquir Immune Defic Syndr. 1999;2022(90):S9-16.

Purcell DW, Namkung Lee A, Dempsey A, Gordon C. Enhanced Federal Collaborations in Implementation Science and Research of HIV Prevention and Treatment. J Acquir Immune Defic Syndr. 1999;2022(90):S17-22.

Queiroz A, Mongrella M, Keiser B, Li DH, Benbow N, Mustanski B. Profile of the Portfolio of NIH-Funded HIV Implementation Research Projects to Inform Ending the HIV Epidemic Strategies. J Acquir Immune Defic Syndr. 1999;2022(90):S23-31.

Zamantakis A, Li DH, Benbow N, Smith JD, Mustanski B. Determinants of Pre-exposure Prophylaxis (PrEP) Implementation in Transgender Populations: A Qualitative Scoping Review. AIDS Behav. 2023;27:1600–18.

Li DH, Benbow N, Keiser B, Mongrella M, Ortiz K, Villamar J, et al. Determinants of Implementation for HIV Pre-exposure Prophylaxis Based on an Updated Consolidated Framework for Implementation Research: A Systematic Review. J Acquir Immune Defic Syndr. 1999;2022(90):S235-46.

Chambers DA, Emmons KM. Navigating the field of implementation science towards maturity: challenges and opportunities. Implement Sci. 2024;19:26, s13012-024-01352–0.

Chinman M, Acosta J, Ebener P, Shearer A. “What we have here, is a failure to [replicate]”: Ways to solve a replication crisis in implementation science. Prev Sci. 2022;23:739–50.

Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117.

Lengnick-Hall R, Gerke DR, Proctor EK, Bunger AC, Phillips RJ, Martin JK, et al. Six practical recommendations for improved implementation outcomes reporting. Implement Sci. 2022;17:16.

Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci IS. 2021;16:36.

Xu X, Lazar CM, Ruger JP. Micro-costing in health and medicine: a critical appraisal. Health Econ Rev. 2021;11:1.

Barnett ML, Dopp AR, Klein C, Ettner SL, Powell BJ, Saldana L. Collaborating with health economists to advance implementation science: a qualitative study. Implement Sci Commun. 2020;1:82.

Lengnick-Hall R, Williams NJ, Ehrhart MG, Willging CE, Bunger AC, Beidas RS, et al. Eight characteristics of rigorous multilevel implementation research: a step-by-step guide. Implement Sci. 2023;18:52.

Riley-Gibson E, Hall A, Shoesmith A, Wolfenden L, Shelton RC, Doherty E, et al. A systematic review to determine the effect of strategies to sustain chronic disease prevention interventions in clinical and community settings: study protocol. Res Sq [Internet]. 2023 [cited 2024 Apr 19]; Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10312971/

Ingvarsson S, Hasson H, von Thiele Schwarz U, Nilsen P, Powell BJ, Lindberg C, et al. Strategies for de-implementation of low-value care—a scoping review. Implement Sci IS. 2022;17:73.

Lewis CC, Powell BJ, Brewer SK, Nguyen AM, Schriger SH, Vejnoska SF, et al. Advancing mechanisms of implementation to accelerate sustainable evidence-based practice integration: protocol for generating a research agenda. BMJ Open. 2021;11:e053474.

Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implement Sci. 2019;14:N.PAG-N.PAG.

Michie S, Atkins L, West R. The behaviour change wheel. Guide Des Interv 1st Ed G B Silverback Publ. 2014;1003:1010.

Birken SA, Haines ER, Hwang S, Chambers DA, Bunger AC, Nilsen P. Advancing understanding and identifying strategies for sustaining evidence-based practices: a review of reviews. Implement Sci IS. 2020;15:88.

Metz A, Jensen T, Farley A, Boaz A, Bartley L, Villodas M. Building trusting relationships to support implementation: A proposed theoretical model. Front Health Serv. 2022;2:894599.

Rabin BA, Cain KL, Watson P, Oswald W, Laurent LC, Meadows AR, et al. Scaling and sustaining COVID-19 vaccination through meaningful community engagement and care coordination for underserved communities: hybrid type 3 effectiveness-implementation sequential multiple assignment randomized trial. Implement Sci IS. 2023;18:28.

Gyamfi J, Iwelunmor J, Patel S, Irazola V, Aifah A, Rakhra A, et al. Implementation outcomes and strategies for delivering evidence-based hypertension interventions in lower-middle-income countries: Evidence from a multi-country consortium for hypertension control. PLOS ONE. 2023;18:e0286204.

Woodward EN, Ball IA, Willging C, Singh RS, Scanlon C, Cluck D, et al. Increasing consumer engagement: tools to engage service users in quality improvement or implementation efforts. Front Health Serv. 2023;3:1124290.

Norton WE, Chambers DA. Unpacking the complexities of de-implementing inappropriate health interventions. Implement Sci IS. 2020;15:2.

Norton WE, McCaskill-Stevens W, Chambers DA, Stella PJ, Brawley OW, Kramer BS. DeImplementing Ineffective and Low-Value Clinical Practices: Research and Practice Opportunities in Community Oncology Settings. JNCI Cancer Spectr. 2021;5:pkab020.

McKay VR, Proctor EK, Morshed AB, Brownson RC, Prusaczyk B. Letting Go: Conceptualizing Intervention De-implementation in Public Health and Social Service Settings. Am J Community Psychol. 2018;62:189–202.

Patey AM, Grimshaw JM, Francis JJ. Changing behaviour, ‘more or less’: do implementation and de-implementation interventions include different behaviour change techniques? Implement Sci IS. 2021;16:20.

Rodriguez Weno E, Allen P, Mazzucca S, Farah Saliba L, Padek M, Moreland-Russell S, et al. Approaches for Ending Ineffective Programs: Strategies From State Public Health Practitioners. Front Public Health. 2021;9:727005.

Gnjidic D, Elshaug AG. De-adoption and its 43 related terms: harmonizing low-value care terminology. BMC Med. 2015;13:273.

Download references

Acknowledgements

The authors would like to acknowledge the early contributions of the Pittsburgh Dissemination and Implementation Science Collaborative (Pitt DISC). LEA would like to thank Dr. Billie Davis for analytical support. The authors would like to acknowledge the implementation science experts who recommended articles for our review, including Greg Aarons, Mark Bauer, Rinad Beidas, Geoffrey Curran, Laura Damschroder, Rani Elwy, Amy Kilbourne, JoAnn Kirchner, Jennifer Leeman, Cara Lewis, Dennis Li, Aaron Lyon, Gila Neta, and Borsika Rabin.

Dr. Rogal’s time was funded in part by a University of Pittsburgh K award (K23-DA048182) and by a VA Health Services Research and Development grant (PEC 19-207). Drs. Bachrach and Quinn were supported by VA HSR Career Development Awards (CDA 20-057, PI: Bachrach; CDA 20-224, PI: Quinn). Dr. Scheunemann’s time was funded by the US Agency for Healthcare Research and Quality (K08HS027210). Drs. Hero, Chinman, Goodrich, Ernecoff, and Mr. Qureshi were funded by the Patient-Centered Outcomes Research Institute (PCORI) AOSEPP2 Task Order 12 to conduct a landscape review of US studies on the effectiveness of implementation strategies with results reported here ( https://www.pcori.org/sites/default/files/PCORI-Implementation-Strategies-for-Evidence-Based-Practice-in-Health-and-Health-Care-A-Review-of-the-Evidence-Full-Report.pdf and https://www.pcori.org/sites/default/files/PCORI-Implementation-Strategies-for-Evidence-Based-Practice-in-Health-and-Health-Care-Brief-Report-Summary.pdf ). Dr. Ashcraft and Ms. Phares were funded by the Center for Health Equity Research and Promotion, (CIN 13-405). The funders had no involvement in this study.

Author information

Shari S. Rogal and Matthew J. Chinman are co-senior authors.

Authors and Affiliations

Center for Health Equity Research and Promotion, Corporal Michael Crescenz VA Medical Center, Philadelphia, PA, USA

Laura Ellen Ashcraft

Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA

David E. Goodrich, Angela Phares, Deirdre A. Quinn, Shari S. Rogal & Matthew J. Chinman

Division of General Internal Medicine, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, USA

David E. Goodrich, Deirdre A. Quinn & Matthew J. Chinman

Clinical & Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA

David E. Goodrich & Lisa G. Lederer

RAND Corporation, Pittsburgh, PA, USA

Joachim Hero, Nabeel Qureshi, Natalie C. Ernecoff & Matthew J. Chinman

Center for Clinical Management Research, VA Ann Arbor Healthcare System, Ann Arbor, Michigan, USA

Rachel L. Bachrach

Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI, USA

Division of Geriatric Medicine, University of Pittsburgh, Department of Medicine, Pittsburgh, PA, USA

Leslie Page Scheunemann

Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, University of Pittsburgh, Department of Medicine, Pittsburgh, PA, USA

Departments of Medicine and Surgery, University of Pittsburgh, Pittsburgh, Pennsylvania, USA

Shari S. Rogal

You can also search for this author in PubMed   Google Scholar

Contributions

LEA, SSR, and MJC conceptualized the study. LEA, SSR, MJC, and JOH developed the study design. LEA and JOH acquired the data. LEA, DEG, AP, RLB, DAQ, LGL, LPS, SSR, NQ, and MJC conducted the abstract, full text review, and rigor assessment. LEA, DEG, JOH, AP, RLB, DAQ, NQ, NCE, SSR, and MJC conducted the data abstraction. DEG, SSR, and MJC adjudicated conflicts. LEA and SSR analyzed the data. LEA, SSR, JOH, and MJC interpreted the data. LEA, SSR, and MJC drafted the work. All authors substantially revised the work. All authors approved the submitted version and agreed to be personally accountable for their contributions and the integrity of the work.

Corresponding author

Correspondence to Laura Ellen Ashcraft .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

The manuscript does not contain any individual person’s data.

Competing interests

Additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., supplementary material 5., supplementary material 6., supplementary material 7., supplementary material 8., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ashcraft, L.E., Goodrich, D.E., Hero, J. et al. A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022. Implementation Sci 19 , 43 (2024). https://doi.org/10.1186/s13012-024-01369-5

Download citation

Received : 09 November 2023

Accepted : 27 May 2024

Published : 24 June 2024

DOI : https://doi.org/10.1186/s13012-024-01369-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Implementation strategy
• Health-related outcomes

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Student Opportunities

About Hoover

Located on the campus of Stanford University and in Washington, DC, the Hoover Institution is the nation’s preeminent research center dedicated to generating policy ideas that promote economic prosperity, national security, and democratic governance. 

• The Hoover Story
• Hoover Timeline & History
• Mission Statement
• Vision of the Institution Today
• Key Focus Areas
• About our Fellows
• Research Programs
• Annual Reports
• Hoover in DC
• Fellowship Opportunities
• Visit Hoover
• David and Joan Traitel Building & Rental Information
• Newsletter Subscriptions
• Connect With Us

Hoover scholars form the Institution’s core and create breakthrough ideas aligned with our mission and ideals. What sets Hoover apart from all other policy organizations is its status as a center of scholarly excellence, its locus as a forum of scholarly discussion of public policy, and its ability to bring the conclusions of this scholarship to a public audience.

• Peter Berkowitz
• Ross Levine
• Michael McFaul
• Timothy Garton Ash
• China's Global Sharp Power Project
• Economic Policy Group
• History Working Group
• Hoover Education Success Initiative
• National Security Task Force
• National Security, Technology & Law Working Group
• Middle East and the Islamic World Working Group
• Military History/Contemporary Conflict Working Group
• Renewing Indigenous Economies Project
• State & Local Governance
• Strengthening US-India Relations
• Technology, Economics, and Governance Working Group
• Taiwan in the Indo-Pacific Region

Books by Hoover Fellows

Economics Working Papers

Hoover Education Success Initiative | The Papers

• Hoover Fellows Program
• National Fellows Program
• Student Fellowship Program
• Veteran Fellowship Program
• Congressional Fellowship Program
• Media Fellowship Program
• Silas Palmer Fellowship
• Economic Fellowship Program

Throughout our over one-hundred-year history, our work has directly led to policies that have produced greater freedom, democracy, and opportunity in the United States and the world.

• Determining America’s Role in the World
• Answering Challenges to Advanced Economies
• Empowering State and Local Governance
• Revitalizing History
• Confronting and Competing with China
• Revitalizing American Institutions
• Reforming K-12 Education
• Understanding Public Opinion
• Understanding the Effects of Technology on Economics and Governance
• Energy & Environment
• Health Care
• Immigration
• International Affairs
• Key Countries / Regions
• Law & Policy
• Politics & Public Opinion
• Science & Technology
• Security & Defense
• State & Local
• Books by Fellows
• Published Works by Fellows
• Working Papers
• Congressional Testimony
• Hoover Press
• PERIODICALS
• The Caravan
• China's Global Sharp Power
• Economic Policy
• History Lab
• Hoover Education
• Global Policy & Strategy
• Middle East and the Islamic World
• Military History & Contemporary Conflict
• Renewing Indigenous Economies
• State and Local Governance
• Technology, Economics, and Governance

Hoover scholars offer analysis of current policy challenges and provide solutions on how America can advance freedom, peace, and prosperity.

• China Global Sharp Power Weekly Alert
• Email newsletters
• Hoover Daily Report
• Subscription to Email Alerts
• Periodicals
• California on Your Mind
• Defining Ideas
• Hoover Digest
• Video Series
• Uncommon Knowledge
• Battlegrounds
• GoodFellows
• Hoover Events
• Capital Conversations
• Hoover Book Club
• AUDIO PODCASTS
• Matters of Policy & Politics
• Economics, Applied
• Free Speech Unmuted
• Secrets of Statecraft
• Pacific Century
• Libertarian
• Library & Archives

Support Hoover

Learn more about joining the community of supporters and scholars working together to advance Hoover’s mission and values.

What is MyHoover?

MyHoover delivers a personalized experience at  Hoover.org . In a few easy steps, create an account and receive the most recent analysis from Hoover fellows tailored to your specific policy interests.

Watch this video for an overview of MyHoover.

Log In to MyHoover

Forgot Password

Don't have an account? Sign up

Have questions? Contact us

• Support the Mission of the Hoover Institution
• Subscribe to the Hoover Daily Report
• Follow Hoover on Social Media

Make a Gift

Your gift helps advance ideas that promote a free society.

• About Hoover Institution
• Meet Our Fellows
• Focus Areas
• Research Teams
• Library & Archives

Library & archives

Events, news & press.

Policy Lessons From Recent Economic Crises

With the benefit of hindsight, one can now draw many conclusions about the US policy responses to the two biggest economic crises of the past decade and a half: the 2008 financial crisis and its aftermath, and the COVID-19 pandemic. Both show what can happen when larger political projects hijack the immediate task at hand.

View the discussion thread.

Join the Hoover Institution’s community of supporters in ideas advancing freedom.

This paper is in the following e-collection/theme issue:

Published on 2.7.2024 in Vol 10 (2024)

A Comprehensive Youth Diabetes Epidemiological Data Set and Web Portal: Resource Development and Case Studies

Authors of this article:

Original Paper

• Catherine McDonough 1 * , MS   ; 
• Yan Chak Li 1 * , MPhil   ; 
• Nita Vangeepuram 2, 3 , MPH, MD   ; 
• Bian Liu 3 , PhD   ; 
• Gaurav Pandey 1 , PhD  

1 Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, United States

2 Department of Pediatrics, Icahn School of Medicine at Mount Sinai, New York, NY, United States

3 Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, United States

*these authors contributed equally

Corresponding Author:

Gaurav Pandey, PhD

Department of Genetics and Genomic Sciences

Icahn School of Medicine at Mount Sinai

1 Gustave L. Levy Pl

New York, NY, 10029

United States

Phone: 1 212 241 6500

Email: [email protected]

Background: The prevalence of type 2 diabetes mellitus (DM) and pre–diabetes mellitus (pre-DM) has been increasing among youth in recent decades in the United States, prompting an urgent need for understanding and identifying their associated risk factors. Such efforts, however, have been hindered by the lack of easily accessible youth pre-DM/DM data.

Objective: We aimed to first build a high-quality, comprehensive epidemiological data set focused on youth pre-DM/DM. Subsequently, we aimed to make these data accessible by creating a user-friendly web portal to share them and the corresponding codes. Through this, we hope to address this significant gap and facilitate youth pre-DM/DM research.

Methods: Building on data from the National Health and Nutrition Examination Survey (NHANES) from 1999 to 2018, we cleaned and harmonized hundreds of variables relevant to pre-DM/DM (fasting plasma glucose level ≥100 mg/dL or glycated hemoglobin  ≥5.7%) for youth aged 12-19 years (N=15,149). We identified individual factors associated with pre-DM/DM risk using bivariate statistical analyses and predicted pre-DM/DM status using our Ensemble Integration (EI) framework for multidomain machine learning. We then developed a user-friendly web portal named Prediabetes/diabetes in youth Online Dashboard (POND) to share the data and codes.

Results: We extracted 95 variables potentially relevant to pre-DM/DM risk organized into 4 domains (sociodemographic, health status, diet, and other lifestyle behaviors). The bivariate analyses identified 27 significant correlates of pre-DM/DM ( P <.001, Bonferroni adjusted), including race or ethnicity, health insurance, BMI, added sugar intake, and screen time. Among these factors, 16 factors were also identified based on the EI methodology (Fisher P of overlap=7.06×10 6 ). In addition to those, the EI approach identified 11 additional predictive variables, including some known (eg, meat and fruit intake and family income) and less recognized factors (eg, number of rooms in homes). The factors identified in both analyses spanned across all 4 of the domains mentioned. These data and results, as well as other exploratory tools, can be accessed on POND.

Conclusions: Using NHANES data, we built one of the largest public epidemiological data sets for studying youth pre-DM/DM and identified potential risk factors using complementary analytical approaches. Our results align with the multifactorial nature of pre-DM/DM with correlates across several domains. Also, our data-sharing platform, POND, facilitates a wide range of applications to inform future youth pre-DM/DM studies.

Introduction

Type 2 diabetes mellitus (DM) is a complex disease influenced by several biological and epidemiological factors [ 1 , 2 ], such as obesity [ 3 ], family history [ 4 ], diet [ 1 , 5 ], physical activity level [ 1 , 6 - 8 ], and socioeconomic status [ 9 - 11 ]. Prediabetes, characterized by elevated blood glucose levels below the diabetes threshold, is a precursor condition to DM [ 12 ]. There has been an alarming increasing trend in the prevalence of youth with pre–diabetes mellitus (pre-DM) and DM both in the United States [ 13 - 19 ] and worldwide [ 20 , 21 ], and the numbers of newly diagnosed youth living with pre-DM/DM are also expected to increase [ 14 , 20 , 22 ]. The latest estimate based on nationally representative data showed that the prevalence of pre-DM among youth increased from 11.6% in 1999-2002 to 28.2% in 2015-2018 in the United States [ 13 ]. This growth is particularly concerning because pre-DM/DM disproportionately affects racial and ethnic minority groups and those with low socioeconomic status [ 9 - 11 , 22 - 24 ], leading to significant health disparities. Having pre-DM/DM at a younger age also confers a higher health and economic burden resulting from living with the condition for more years and a higher risk of developing other cardiometabolic diseases [ 25 - 30 ]. This serious challenge calls for increased translational research into factors associated with pre-DM/DM among youth and how they can collectively affect disease risk and inform prevention strategies.

In particular, the most critically needed research in this direction is exploring the collective impact of various risk factors across multiple health-related domains. While clinical factors, such as obesity, have been mechanistically linked to insulin resistance [ 31 ], it is important to consider the broader perspective. There is an increasing recognition that social determinants of health (SDoH) play a significant role in amplifying the risk of pre-DM/DM and their related disparities. For example, factors such as limited access to health care, food and housing insecurity, and the neighborhood-built environment have been identified as influential contributors [ 9 - 11 , 32 ]. However, to gain a comprehensive understanding, it is essential to delve into other less studied variables, such as screen time, acculturation, or frequency of eating out, and examine how they interact to increase the risk of pre-DM/DM among youth [ 2 ].

One of the major challenges that has limited translational research into youth pre-DM/DM risk factors is that there are not publicly available, easily accessible data comprehensively profiling interrelated epidemiological factors for young individuals [ 2 ]. Specifically, most available public diabetes data portals focus on providing aggregated descriptive trends, such as pre-DM/DM prevalence for the entire population or subgroups stratified by race and ethnicity [ 33 - 36 ], which does not allow in-depth examination of the relationships between multiple risk factors and pre-DM/DM risk using individual-level data. While there do exist a few individual-level public diabetes data sets [ 37 - 41 ], they include mainly clinical measurements, while other important risk factors such as those related to diet, physical activity, and SDoH are limited. In addition, these data sets are not available for youth populations, as they focus exclusively on adult populations and not on youth specifically [ 37 , 39 - 41 ]. Furthermore, these data sets are not accompanied by any user-friendly web-based portals that can help explore or analyze these data to reveal interesting knowledge about youth pre-DM/DM. This shows that there is a lack of a comprehensive data set that includes multiple epidemiological variables to study youth pre-DM/DM and easily usable functionalities to explore and analyze data.

To directly address this data gap, we turned to the National Health and Nutrition Examination Survey (NHANES), which offers a promising path for examining pre-DM/DM among the US youth population by providing a rich source of individual- and household-level epidemiological factors. As a result, NHANES has been a prominent data source for studying youth pre-DM/DM trends and associated factors [ 18 , 42 - 45 ]. However, the use of NHANES data requires extensive data processing that is laborious and time-intensive [ 46 ]. This represents a major challenge for the widespread use of these high-quality and extensive data for studying youth pre-DM/DM.

In this work, we directly addressed the above challenges by processing NHANES data from 1999 to 2018 into a large-scale, youth diabetes–focused data set that covers a variety of relevant variable domains, namely, sociodemographic factors, health status indicators, diet, and other lifestyle behaviors. We also provided public access to this high-quality comprehensive youth pre-DM/DM data set, as well as functionalities to explore and analyze it, through the user-friendly Prediabetes/diabetes in youth Online Dashboard (POND) [ 47 ]. We demonstrated the data set’s use and potential through 2 case studies that used statistical analyses and machine learning (ML) approaches, respectively, to identify important epidemiological factors that are associated with youth pre-DM/DM.

Through this work, we aim to advance youth diabetes research by providing the most comprehensive epidemiological data set available through a public web portal and illustrating the value of these resources through our example case studies based on statistical analyses and ML. Our overarching goal is to enable researchers to investigate the multifactorial variables associated with youth pre-DM/DM, which may drive translational advances in prevention and management strategies.

Figure 1 [ 48 ] shows the overall study design and workflow. In the following subsections, we detail the components of the workflow.

Data Source and Study Population

We built the youth pre-DM/DM data set based on publicly available NHANES data [ 49 ] spanning the years from 1999 to 2018. Developed by the Centers for Disease Control and Prevention, NHANES is a serial cross-sectional survey that gathers comprehensive health-related information from nationally representative samples of the noninstitutionalized population in the United States. The survey uses a multistage probability sampling method and collects data through questionnaires, physical examinations, and biomarker analysis. Each year, approximately 5000 individuals are included in the survey, and the data are publicly released in 2-year cycles.

Figure 2 details the process used to define our study population. Briefly, of the total 101,316 participants in 1999-2018 NHANES, we excluded individuals who (1) were not within the 12-19 years age range, (2) did not have either of the biomarkers used to define pre-DM/DM status, and (3) answered “Yes” to “Have you ever been told by a doctor or health professional that you have diabetes?” The youth pre-DM/DM outcome of this work was derived as follows: youth were considered at risk of pre-DM/DM if their fasting plasma glucose (FPG) was at or greater than 100 mg/dL, or their glycated hemoglobin (HbA 1c ) was at or greater than 5.7%, according to the current American Diabetes Association (ADA) pediatric clinical guidelines [ 2 ].

Validation of the Study Population

We estimated pre-DM/DM prevalence across the 10 survey cycles (1999-2018) by incorporating the NHANES design elements in the analysis and compared the general trend with those reported in the literature [ 18 , 19 ]. We also specifically applied the analytical methods reported in a recent study [ 13 ] based on NHANES data to our study population to replicate the trends in pre-DM among youth in the United States from 1999 to 2018 reported in that analysis. Specifically, that study selected a youth population from 12-19 years of age with positive sampling weight from the fasting subsample (ie, nonzero and nonmissing Fasting Subsample 2 Year Mobile Examination Centers Weight [“WTSAF2YR”]; personal communication) without a self-reported physician-diagnosed DM. In addition, that study focused only on pre-DM, which was defined as an HbA 1c level between 5.7% and 6.4% or an FPG level between 100 mg/dL and 125 mg/dL [ 13 ].

Development of Youth Pre-DM/DM Data Set

Based on the most recent ADA standard of care recommendations including factors related to pre-DM/DM risk and management [ 2 ], we selected 27 potentially relevant NHANES questionnaires and grouped them into 4 domains: sociodemographic, health status, diet, and other lifestyle behaviors. For example, under the health status domain, BMI was included as a potential risk factor for youth pre-DM/DM [ 2 ]. Similarly, lifestyle and behavioral variables included factors, such as diet and physical activity, that have been shown to be critical for pre-DM/DM prevention in both observational studies and randomized clinical trials [ 50 - 52 ]. Our sociodemographic domain included demographic, socioeconomic, and SDoH variables (eg, age, gender, poverty status, and food security). Except for commonly available clinical measurements, such as blood pressure and total cholesterol, we did not include laboratory data (eg, triglycerides, transferrin, C-reactive protein, interleukin-6, and white blood cells), since these measurements were not collected for all NHANES participants and were not commonly accessible for the general population.

From the selected questionnaires, we identified a list of 95 variables based on the aforementioned methodology. The complete list of variables is provided in Table S1 in Section S1 of Multimedia Appendix 1 [ 13 , 49 , 53 - 62 ] and on our POND web portal [ 47 ]. All the code developed, processed data, and detailed description of variables are also available on the web portal [ 47 ]. The process of extracting these variables involved extensive examination of the questions that were asked, consultation of the literature, and discussions to reach consensus within the study team. The details of this process are provided in Figure S1 and Section S2 of Multimedia Appendix 1 . We used SAS (version 9.4; SAS Institute) and R (version 4.2.2; R Core Team, 2022) in R Studio (version 4.2.2; R Core Team, 2022) for data processing and data set development.

Building the POND

To facilitate other researchers’ use of our youth pre-DM/DM data set and make our methodology transparent and reproducible, we developed POND to share our processed data set and enable users to understand and explore the data on their own. The web portal was developed using R markdown and the flexdashboard package [ 63 ] and was published as a Shiny application [ 64 ]. Table S2 and Section S3 in Multimedia Appendix 1 provide details of all the R packages used to develop POND, and the related code is available on the portal’s download page.

Case Studies in Using the Data Set to Better Understand Youth Pre-DM/DM

To examine the validity and use of our data set for advancing translational research on youth pre-DM/DM, we conducted 2 complementary data analyses. We first conducted bivariate analyses to assess the statistical associations between each of the 95 variables and youth pre-DM/DM status. In the second analysis, we used ML methods to examine the ability to predict pre-DM/DM status of youth based on the 95 variables. The methodological details of these analyses are provided in the following subsections.

Bivariate Analyses to Identify Variables Associated With Pre-DM/DM Status

We examined associations between individual variables and youth pre-DM/DM status using chi-square and Wilcoxon rank sum tests for categorical and continuous variables, respectively. Cell sizes were checked for sufficient size (≥5) prior to chi-square tests. Independence and equal variance were assessed for continuous variables. Distribution normality was ensured through adequate sample size in accordance with Central Limit Theorem [ 65 ]. We applied Bonferroni correction for multiple hypothesis testing (n=95 tests) at an α level of .05 to determine the statistical significance of each association at the adjusted α level of .0005 (ie, approximately 0.05/95). We used Cramer V and Wilcoxon R values [ 66 ] as the effect size measures for categorical and continuous variables, respectively. To better compare with results from the ML approach, the main bivariate analyses did not account for NHANES survey design; thus, the results were applicable only to the study population included in the analytical sample and were not generalizable to the entire US youth population. For completeness, we provide the survey-weighted analyses using NHANES examination weights (“WTMEC2YR”) in Section S4 of Multimedia Appendix 1 .

Prediction of Pre-DM/DM Status Using ML Algorithms

Several ML algorithms have been used to predict adult pre-DM/DM status using NHANES data [ 67 - 69 ], and we have previously used these algorithms to predict pre-DM/DM status specifically among youth in a subsample of our current study population [ 42 ]. We expanded these existing analyses by taking into account the multidomain nature of our data set with the goal of building an effective and interpretable predictive model of youth pre-DM/DM. To that end, we leveraged our recently developed ML framework, Ensemble Integration (EI) [ 53 , 54 ], with all 4 domains and their variables in our data set. EI incorporates both consensus and complementarity in our data set by first inferring local predictive models from the individual domains, that is, sociodemographic, health status, diet, and other lifestyle behaviors, that are expected to capture information and interactions specific to the domains. These local models and information are then integrated into a global pre-DM/DM, comprehensive pre-DM/DM prediction model using heterogeneous ensemble algorithms [ 70 ] (Figure S2, Table S3, and Table S4 under Section S5 in Multimedia Appendix 1 ). These algorithms, such as stacking, allow the integration of an unrestricted number and variety of local models into the global predictive model, thus offering improved performance and robustness. EI also enables the identification of the most predictive variables in the final model, thus offering deeper insights into the outcome being predicted.

We used both the aforementioned capabilities of EI to build and interpret a predictive model of youth pre-DM/DM status based on our data set. We also compared the predictive performance of the model with three alternative approaches: (1) a modified form of the ADA screening guideline [ 55 ], which is based on BMI, total cholesterol level, hypertension, and race or ethnicity, to assess the use of data-driven screening for youth pre-DM/DM (Table S5 in Multimedia Appendix 1 ); (2) EI applied to individual variable domains, namely, sociodemographic, health status, diet and other lifestyle behaviors, to assess the value of multidomain data for youth pre-DM/DM prediction; and (3) extreme gradient boosting (XGBoost) [ 71 ] applied to our combined multidomain data set as a representative alternate ML algorithm. This alternative was chosen as XGBoost is considered the most effective classification algorithm for tabular data [ 72 ], since it can potentially capture feature interactions across different domains [ 73 , 74 ]. The prediction performance of EI and all the alternative approaches were assessed in terms of the commonly used area under the receiver operating characteristic curve (AUROC) [ 75 ] and balanced accuracy (BA; average of specificity and sensitivity) [ 76 ] measures. The performance of the ML-based prediction approaches, namely, multi- and single-domain EI and XGBoost, was evaluated in a 5-fold cross-validation setting repeated 10 times [ 77 ]. These performance scores were statistically compared using the Wilcoxon rank sum test, and the resultant P values were corrected for multiple hypothesis testing using the Benjamini-Hochberg procedure to yield false discovery rates (FDRs) [ 78 ]. More details of ML model building; the alternative approaches; and the evaluation methodology, including cross-validation, model selection, and comparison, are available in section S5 in Multimedia Appendix 1 . Finally, we used EI’s interpretation capabilities [ 53 , 54 ] to identify the variables in our data set that were the most predictive of youth pre-DM/DM status and compare them with the variables identified from the bivariate analyses described in the above subsection.

Ethical Considerations

This study used existing deidentified and anonymized data in the public domain directly downloadable from the NHANES website and thus, according to the Common Rule, was exempt from institutional review board review and the informed consent requirement. NHANS was conducted by the Centers for Disease Control and Prevention National Center for Health Statistics. NHANES survey procedures and protocol were approved by the National Center for Health Statistics ethics review board for each survey cycle [ 79 ].

Study Population Derived From NHANES

Our study population consisted of 15,149 youths aged 12-19 years who participated in the 1999-2018 NHANES cycles and met our selection criteria ( Figure 2 ). Approximately 13.3% (2010/15,149) of US youth were at risk of pre-DM/DM according to the clinically standard criteria for defining pre-DM/DM per ADA guidelines (FPG ≥100 mg/dL and HbA 1c ≥5.7%; Table 1 ).

a Unweighted statistics of some key variables describing the study population in the youth pre-DM/DM data set overall and by pre-DM/DM status. More detailed statistics for all the variables in our data set can be found in the Data Exploration section of POND.

b Pre-DM/DM: pre–diabetes mellitus and diabetes mellitus.

c CHIP: child health insurance program.

d Hypertensive was defined by blood pressure ≥90th percentile or ≥120/80 mm Hg for children 13 years of age and older [ 2 ].

e eq: equivalent.

We estimated that the survey-weighted prevalence of pre-DM/DM in our study population rose substantially from 4.1% (95% CI 2.8-5.4) in 1999 to 22% (95% CI 18.5-25.6) in 2018 (Figure S3 and section S6 in Multimedia Appendix 1 ). This increasing trend of pre-DM/DM prevalence was consistent with that reported in other NHANES-based studies, which had pre-DM/DM prevalence ranging from 17.7% to 18% [ 18 , 19 ]. We also applied the study population and pre-DM definition criteria reported in a recent study [ 13 ] to NHANES data and derived a similarly sized study population (n=6656 vs n=6598 in the current vs previous analysis [ 13 ]) and youth pre-DM prevalence, which ranged from 11.1% (95% CI 8.9-13.3) to 37.3% (95% CI 31.0-43.6) in our analysis compared with from 11.6% (95% CI 9.5-14.1) to 28.2% (95% CI 23.3-33.6) in the study by Liu et al [ 13 ] (Table S6 in Multimedia Appendix 1 ).

Youth Pre-DM/DM-Focused Data Set

We extracted 95 epidemiological variables from NHANES and organized them into 4 pre-DM/DM-related domains, namely, sociodemographic, health status, diet, and other lifestyle behaviors (Table S1 in Multimedia Appendix 1 ). Table 1 shows the unweighted statistics of some key study population characteristics. Among youth with pre-DM/DM (n=2010), the proportion of youth who were non-Hispanic Black, non-Hispanic White, Hispanic, and other race or ethnicity (including non-Hispanic persons who reported races other than Black or White and non-Hispanic Asian) were 33.6% (n=676), 21.4% (n=431), 35.4% (n=711), and 9.6% (n=192), respectively. Approximately, half (7719/15,149, 51%) of the population were male, and they represented 65.6% (1319/2010) of those with pre-DM/DM. Approximately 32.4% (4528/15,149) of the youth had a family income below poverty level, and 69.4% (7833/15,149) were from households receiving food stamps. The proportion of youth covered by private insurance was higher among those with than with no pre-DM/DM (5648/13,139, 43.8% vs 744/2010, 37.7%). Overall, 21.5% (3214/15,149) of the youth were obese as defined by having a BMI at or above the 95th percentile based on age and gender, and the proportion was 33.3% (663/2010) among youth with pre-DM/DM. Youth with pre-DM/DM tended to have less fruit and vegetable intake and ate lower amounts of protein and total grains than those with no pre-DM/DM. Youth with and with no pre-DM/DM showed similar amounts of physical activity with 209 and 210 minutes per week, respectively ( Table 1 ).

Pre-DM/DM in Youth Online Dashboard

To facilitate other researchers’ use of our youth pre-DM/DM data set and make our methodology transparent and reproducible, we developed POND, which is available on [ 47 ]. Users can navigate POND through its built-in functionalities. For example, users are able to explore the details of the 95 individual variables ( Figure 3 A) and their distributions by pre-DM/DM status ( Figure 3 B), examine the risk factors of youth pre-DM/DM identified from the case studies described below ( Figure 3 C), as well as download the data for customized analysis and the analytical code to replicate our findings ( Figure 3 D). In addition, we make available all the code used to develop the data set, our case studies, and POND itself.

Case Studies Using Our Data Set to Better Understand Youth Pre-DM/DM

We examined the validity and use of our processed multidomain data set for translational studies on youth pre-DM/DM by the following 2 complementary types of data analyses.

Identifying Individual Variables Associated With Pre-DM/DM Status

In our bivariate analyses, we found 27 variables to be significantly ( P <.001, Bonferroni adjusted) associated with pre-DM/DM status ( Figure 4 [ 63 ] and Table S7 in Multimedia Appendix 1 ). These variables spanned all 4 domains and included gender, race or ethnicity, use of food stamps, health insurance status, BMI, total protein intake, and screen time. Similar results were found when repeating these bivariate association tests after accounting for NHANES survey design elements (Table S7 in Multimedia Appendix 1 ).

Predicting Youth Pre-DM/DM Status With ML

We used an ML framework, EI [ 53 , 54 ], to leverage the multidomain nature of our data set and predict youth pre-DM/DM status. We also compared EI’s performance with alternative prediction approaches, most prominently the widely used XGBoost algorithm [ 71 ].

The best-performing multidomain EI methodology, stacking [ 75 ] using logistic regression, predicted youth pre-DM/DM status (AUROC=0.67; BA=0.62) more accurately than all the alternative approaches ( Figure 5 ), namely, XGBoost (AUROC=0.64; BA=0.60; Wilcoxon rank sum FDR=1.7×10 4 and 1.8×10 4 , respectively), the ADA pediatric screening guidelines (AUROC=0.57, BA=0.57; Wilcoxon rank sum FDR=1.7×10 4 and 1.8×10 4 , respectively), and 4 single-domain EI (AUROC=0.63-0.54; BA=0.60-0.53; FDR <1.7×10 4 and 1.8×10 4 , respectively).

The multidomain EI also identified 27 variables (the same as the number of significant variables from bivariate analyses) that contributed the most to predicting youth pre-DM/DM status. Among these variables, 16 overlapped with those identified from the bivariate statistical analyses ( Figure 6 ; Fisher P of overlap=7.06×10 6 ). These variables identified by both approaches included some established pre-DM/DM risk factors such as BMI and high total cholesterol, as well as some less-recognized ones such as screen time and taking prescription drugs [ 2 ].

Principal Findings

Leveraging the rich information in NHANES spanning nearly 20 years, we built the most comprehensive epidemiological data set for studying youth pre-DM/DM. We accomplished this by selecting and harmonizing variables relevant to youth pre-DM/DM from sociodemographic, health status, diet, and other lifestyle behaviors domains. This youth pre-DM/DM data set, as well as several functionalities to explore and analyze it, is publicly available in our user-friendly web portal, POND. We also conducted case studies using the data set with both traditional statistical methods and ML approaches to demonstrate the potential of using this data set to identify factors relevant to youth pre-DM/DM. The combination of the comprehensive public data set and POND provides avenues for more informed investigations of youth pre-DM/DM.

The future translational impact of pre-DM/DM research, facilitated by comprehensive data sets such as the one developed in this study, holds significant promise for advancing our understanding of the disease and its risk factors among youth. By enabling researchers to investigate multifactorial variables associated with pre-DM/DM, this data set contributes to several areas of research and has a broader impact on the scientific community. First, the data set’s comprehensive nature allows researchers to explore the collective impact of various risk factors across multiple health domains. By incorporating sociodemographic factors, health status indicators, diet, and lifestyle behaviors, researchers can gain a holistic understanding of the interplay between these factors and pre-DM/DM risk among youth. This knowledge can be used to generate hypotheses for further studies and inform the development of targeted interventions and prevention strategies that address the specific needs of at-risk populations. Furthermore, the data set provides an opportunity to delve into less-studied variables and their interactions in relation to pre-DM/DM risk. Variables such as screen time, acculturation, or frequency of eating out, which are often overlooked in traditional research, can be examined to uncover their potential influence on pre-DM/DM risk among youth. This expands the scope of translational research and enhances our understanding of the multifaceted nature of the disease.

One of the major contributions of our work was POND, our publicly available web portal, which provided access to all materials related to our data set and analyses, thus enabling transparency and reproducibility. Although several such portals are available in other biomedical areas, such as genomics [ 76 - 78 ], there is a general lack of such tools in epidemiology and public health. We hope that, in addition to facilitating studies into pre-DM/DM, POND illustrates the use of such portals for population and epidemiological studies as well.

The results of the case studies and validation exercises we conducted were also consistent with existing literature. The case studies identified known pre-DM/DM risk factors, such as gender [ 15 , 17 , 19 ], race and ethnicity [ 2 , 9 , 10 , 24 ], health measures (BMI, hypertension, and cholesterol) [ 2 , 55 ], income [ 9 , 11 ], insurance status [ 9 , 10 ], and health care availability [ 9 , 10 ], thus affirming the validity of the data set. In addition, our analyses revealed some less studied variables, such as screen time, home ownership status, self-reported health status, soy and nut consumption, and frequency of school meal intake, which may influence youth pre-DM/DM risk. Further study of these variables may reveal new knowledge about pre-DM/DM among youth. More generally, such novel findings further demonstrate the use of our data set and data-driven methods for further translational discoveries about this complex disorder.

Limitations

Although our work has several strengths and high potential use for youth pre-DM/DM studies, it is not without limitations. First, as our data set was derived from NHANES, we adopt limitations to the survey in our data set. Since NHANES is a cross-sectional survey, the pre-DM/DM status and its related variables provide only consecutive snapshots of youth in the United States over time across the available survey cycles. Thus, the associations identified are better suited for hypothesis generation purposes and require in-depth investigation using prospective longitudinal and randomized trial designs. In addition, we modified the ADA guideline for determining pre-DM/DM status according to variable availability. Due to the high missingness of 45% in family history (DIQ170) and the complete missingness of maternal history (DIQ175S) from 1999 to 2010 in the raw NHANES data, we were unable to include family history of diabetes in the data set. Similarly, NHANES does not provide data regarding every condition associated with insulin resistance. Therefore, we used hypertension and high cholesterol as proxies for insulin resistance. On the other hand, as our main purpose is to use POND as a conduit between this comprehensive youth pre-DM/DM database and interested researchers, our method can be adopted to longitudinal data sets should they become available in the future. Second, for the prediction of pre-DM/DM status, EI’s performance was found to be significantly better than the alternative approaches, including a modified form of the suggested guideline [ 45 ]. However, this performance assessment was based only on cross-validation, which is no substitute for validation on external data sets that is necessary for rigorous assessment. Finally, while our preliminary case study analyses identified a wide range of variables associated with youth prediabetes and diabetes, other known risk factors, such as current asthma status [ 80 - 82 ], added sugar consumption [ 83 - 85 ], sugary fruit and juice intake [ 83 - 86 ], and physical activity per week [ 6 - 8 , 50 ], were not identified. This limitation can be addressed by using other data analysis methods beyond our bivariate testing and ML approaches, highlighting more potential use cases of our data set.

Conclusions

Overall, the future impact of translational pre-DM/DM research facilitated by comprehensive data sets and web servers like ours extends beyond individual studies. It creates opportunities for interdisciplinary collaboration and reproducibility, strengthens evidence-based decision-making, and supports the development of targeted interventions for the prevention and management of pre-DM/DM among youth. By providing rich resources, our work can enable researchers to build upon existing knowledge and push the boundaries of translational pre-DM/DM research, ultimately leading to improved health outcomes for at-risk populations.

Acknowledgments

This study was enabled in part by computational resources provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai. The Ensemble Integration used in this work was implemented by Jamie JR Bennett. This work was funded by National Institutes of Health grants R21DK131555 and R01HG011407.

Data Availability

The data set and code used in this study are available at Zenodo [ 87 ] and our web portal POND [ 47 ].

Authors' Contributions

BL and GP contributed equally as cosenior and cosupervisory authors. NV, BL, and GP conceptualized the project. CM, YCL, NV, BL, and GP designed the methodology. CM and BL implemented the data curation and bivariate analyses. YCL implemented the ML case study and POND. CM and YCL conducted formal analysis and visualization. CM, YCL, NV, BL, and GP wrote the manuscript. NV, BL, and GP supervised the project.

Conflicts of Interest

None declared.

Supplemental materials.

• Temneanu OR, Trandafir LM, Purcarea MR. Type 2 diabetes mellitus in children and adolescents: a relatively new clinical problem within pediatric practice. J Med Life. 2016;9(3):235-239. [ FREE Full text ] [ Medline ]
• ElSayed NA, Aleppo G, Aroda VR, Bannuru RR, Brown FM, Bruemmer D, et al. 2. Classification and diagnosis of diabetes: standards of care in diabetes-2023. Diabetes Care. 2023;46(Suppl 1):S19-S40. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weiss R, Dufour S, Taksali SE, Tamborlane WV, Petersen KF, Bonadonna RC, et al. Prediabetes in obese youth: a syndrome of impaired glucose tolerance, severe insulin resistance, and altered myocellular and abdominal fat partitioning. Lancet. 2003;362(9388):951-957. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang Y, Luk AOY, Chow E, Ko GTC, Chan MHM, Ng M, et al. High risk of conversion to diabetes in first-degree relatives of individuals with young-onset type 2 diabetes: a 12-year follow-up analysis. Diabet Med. 2017;34(12):1701-1709. [ CrossRef ] [ Medline ]
• Zhuang P, Liu X, Li Y, Wan X, Wu Y, Wu F, et al. Effect of diet quality and genetic predisposition on hemoglobin A and type 2 diabetes risk: gene-diet interaction analysis of 357,419 individuals. Diabetes Care. 2021;44(11):2470-2479. [ CrossRef ] [ Medline ]
• Pivovarov JA, Taplin CE, Riddell MC. Current perspectives on physical activity and exercise for youth with diabetes. Pediatr Diabetes. 2015;16(4):242-255. [ CrossRef ] [ Medline ]
• Colberg SR, Sigal RJ, Yardley JE, Riddell MC, Dunstan DW, Dempsey PC, et al. Physical activity/exercise and diabetes: a position statement of the American Diabetes Association. Diabetes Care. 2016;39(11):2065-2079. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Thomson NM, Kraft N, Atkins RC. Cell-mediated immunity in glomerulonephritis. Aust N Z J Med. 1981;11(Suppl 1):104-108. [ Medline ]
• Hill-Briggs F, Adler NE, Berkowitz SA, Chin MH, Gary-Webb TL, Navas-Acien A, et al. Social determinants of health and diabetes: a scientific review. Diabetes Care. 2020;44(1):258-279. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Butler AM. Social determinants of health and racial/ethnic disparities in type 2 diabetes in youth. Curr Diab Rep. 2017;17(8):60. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Walker RJ, Smalls BL, Campbell JA, Strom Williams JL, Egede LE. Impact of social determinants of health on outcomes for type 2 diabetes: a systematic review. Endocrine. 2014;47(1):29-48. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bansal N. Prediabetes diagnosis and treatment: a review. World J Diabetes. 2015;6(2):296-303. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu J, Li Y, Zhang D, Yi SS, Liu J. Trends in prediabetes among youths in the US from 1999 through 2018. JAMA Pediatr. 2022;176(6):608-611. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tönnies T, Brinks R, Isom S, Dabelea D, Divers J, Mayer-Davis EJ, et al. Projections of type 1 and type 2 diabetes burden in the US population aged 20 years through 2060: the SEARCH for Diabetes in Youth Study. Diabetes Care. Feb 1, 2023;46(2):313-320. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lawrence JM, Divers J, Isom S, Saydah S, Imperatore G, Pihoker C, et al. SEARCH for Diabetes in Youth Study Group. Trends in prevalence of type 1 and type 2 diabetes in children and adolescents in the US, 2001-2017. JAMA. 2021;326(8):717-727. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jensen ET, Dabelea D. Type 2 diabetes in youth: new lessons from the SEARCH Study. Curr Diab Rep. 2018;18(6):36. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dabelea D, Mayer-Davis EJ, Saydah S, Imperatore G, Linder B, Divers J, et al. Prevalence of type 1 and type 2 diabetes among children and adolescents from 2001 to 2009. JAMA. 2014;311(17):1778-1786. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Andes LJ, Cheng YJ, Rolka DB, Gregg EW, Imperatore G. Prevalence of prediabetes among adolescents and young adults in the United States, 2005-2016. JAMA Pediatr. 2020;174(2):e194498. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Menke A, Casagrande S, Cowie CC. Prevalence of diabetes in adolescents aged 12 to 19 years in the United States, 2005-2014. JAMA. 2016;316(3):344-345. [ CrossRef ] [ Medline ]
• Khan MAB, Hashim MJ, King JK, Govender RD, Mustafa H, Al Kaabi J. Epidemiology of type 2 diabetes—global burden of disease and forecasted trends. J Epidemiol Glob Health. 2020;10(1):107-111. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lin X, Xu Y, Pan X, Xu J, Ding Y, Sun X, et al. Global, regional, and national burden and trend of diabetes in 195 countries and territories: an analysis from 1990 to 2025. Sci Rep. 2020;10(1):14790. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Imperatore G, Boyle JP, Thompson TJ, Case D, Dabelea D, Hamman RF, et al. Projections of type 1 and type 2 diabetes burden in the U.S. population aged 20 years through 2050: dynamic modeling of incidence, mortality, and population growth. Diabetes Care. Dec 2012;35(12):2515-2520. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Herman WH, Ma Y, Uwaifo G, Haffner S, Kahn SE, Horton ES, et al. Diabetes Prevention Program Research Group. Differences in A1C by race and ethnicity among patients with impaired glucose tolerance in the Diabetes Prevention Program. Diabetes Care. 2007;30(10):2453-2457. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kahkoska AR, Shay CM, Crandell J, Dabelea D, Imperatore G, Lawrence JM, et al. Association of race and ethnicity with glycemic control and hemoglobin A levels in youth with type 1 diabetes. JAMA Netw Open. 2018;1(5):e181851. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lascar N, Brown J, Pattison H, Barnett AH, Bailey CJ, Bellary S. Type 2 diabetes in adolescents and young adults. Lancet Diabetes Endocrinol. 2018;6(1):69-80. [ CrossRef ] [ Medline ]
• Lee AM, Fermin CR, Filipp SL, Gurka MJ, DeBoer MD. Examining trends in prediabetes and its relationship with the metabolic syndrome in US adolescents, 1999-2014. Acta Diabetol. 2017;54(4):373-381. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weiss R, Taksali SE, Tamborlane WV, Burgert TS, Savoye M, Caprio S. Predictors of changes in glucose tolerance status in obese youth. Diabetes Care. 2005;28(4):902-909. [ CrossRef ] [ Medline ]
• Nadeau K, Anderson B, Berg E, Chiang J, Chou H, Copeland K, et al. Youth-onset type 2 diabetes consensus report: current status, challenges, and priorities. Diabetes Care. 2016;39(9):1635-1642. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dart A, Martens P, Rigatto C, Brownell M, Dean H, Sellers E. Earlier onset of complications in youth with type 2 diabetes. Diabetes Care. 2014;37(2):436-443. [ CrossRef ] [ Medline ]
• American Diabetes Association. Economic costs of diabetes in the U.S. in 2017. Diabetes Care. 2018;41(5):917-928. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Al-Goblan AS, Al-Alfi MA, Khan MZ. Mechanism linking diabetes mellitus and obesity. Diabetes Metab Syndr Obes. 2014;7:587-591. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chan JCN, Lim L, Wareham NJ, Shaw JE, Orchard TJ, Zhang P, et al. The Lancet Commission on diabetes: using data to transform diabetes care and patient lives. Lancet. 2021;396(10267):2019-2082. [ CrossRef ] [ Medline ]
• IDF Diabetes Atlas, 10th Edition. International Diabetes Federation. URL: https://diabetesatlas.org/ [accessed 2024-05-16]
• U.S. Chronic Disease Indicators: Diabetes | Chronic Disease and Health Promotion Data & Indicators. URL: https:/​/chronicdata.​cdc.gov/​Chronic-Disease-Indicators/​U-S-Chronic-Disease-Indicators-Diabetes/​f8ti-h92k [accessed 2023-05-17]
• Homepage of NCD Risk Factor Collaboration. NCD Risk Factor Collaboration. URL: https://ncdrisc.org/index.html [accessed 2023-05-17]
• NCD Risk Factor Collaboration (NCD-RisC). Worldwide trends in diabetes since 1980: a pooled analysis of 751 population-based studies with 4.4 million participants. Lancet. 2016;387(10027):1513-1530. [ FREE Full text ] [ CrossRef ] [ Medline ]
• UCI Machine Learning Repository. Diabetes 130-US hospitals for years 1999-2008 Data Set. URL: https://archive.ics.uci.edu/ml/datasets/Diabetes+130-US+hospitals+for+years+1999-2008 [accessed 2023-05-20]
• Type 2 Diabetes Knowledge Portal. URL: https://t2d.hugeamp.org/ [accessed 2023-05-17]
• Rashid A. Diabetes Dataset. Mendeley Data. German. Elsevier; Jul 18, 2020. URL: https://data.mendeley.com/datasets/wj9rwkp9c2/1 [accessed 2024-05-16]
• Diabetes Dataset 2019. URL: https://www.kaggle.com/datasets/tigganeha4/diabetes-dataset-2019 [accessed 2023-05-20]
• Diabetes Health Indicators Dataset. URL: https://www.kaggle.com/datasets/alexteboul/diabetes-health-indicators-dataset [accessed 2023-05-17]
• Vangeepuram N, Liu B, Chiu P, Wang L, Pandey G. Predicting youth diabetes risk using NHANES data and machine learning. Sci Rep. 2021;11(1):11212. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nagarajan S, Khokhar A, Holmes DS, Chandwani S. Family consumer behaviors, adolescent prediabetes and diabetes in the national health and nutrition examination survey (2007-2010). J Am Coll Nutr. 2017;36(7):520-527. [ CrossRef ] [ Medline ]
• Wallace AS, Wang D, Shin J, Selvin E. Screening and diagnosis of prediabetes and diabetes in US children and adolescents. Pediatrics. 2020;146(3):e20200265. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chu P, Patel A, Helgeson V, Goldschmidt AB, Ray MK, Vajravelu ME. Perception and awareness of diabetes risk and reported risk-reducing behaviors in adolescents. JAMA Netw Open. 2023;6(5):e2311466. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Patel CJ, Pho N, McDuffie M, Easton-Marks J, Kothari C, Kohane IS, et al. A database of human exposomes and phenomes from the US National Health and Nutrition Examination Survey. Sci Data. 2016;3:160096. [ FREE Full text ] [ CrossRef ] [ Medline ]
• PreDM/DM in youth ONline Dashboard (POND). URL: https://rstudio-connect.hpc.mssm.edu/POND/ [accessed 2024-02-02]
• Freepik. URL: https://www.flaticon.com [accessed 2024-05-31]
• Zipf G, Chiappa M, Porter KS, Ostchega Y, Lewis BG, Dostal J. National health and nutrition examination survey: plan and operations, 1999-2010. Vital Health Stat 1. 2013;(56):1-37. [ FREE Full text ] [ Medline ]
• Sampath Kumar A, Maiya AG, Shastry BA, Vaishali K, Ravishankar N, Hazari A, et al. Exercise and insulin resistance in type 2 diabetes mellitus: a systematic review and meta-analysis. Ann Phys Rehabil Med. 2019;62(2):98-103. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Karstoft K, Winding K, Knudsen SH, Nielsen JS, Thomsen C, Pedersen BK, et al. The effects of free-living interval-walking training on glycemic control, body composition, and physical fitness in type 2 diabetic patients: a randomized, controlled trial. Diabetes Care. 2013;36(2):228-236. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Karstoft K, Christensen CS, Pedersen BK, Solomon TPJ. The acute effects of interval- vs continuous-walking exercise on glycemic control in subjects with type 2 diabetes: a crossover, controlled study. J Clin Endocrinol Metab. 2014;99(9):3334-3342. [ CrossRef ] [ Medline ]
• Li Y, Wang L, Law J, Murali T, Pandey G. Integrating multimodal data through interpretable heterogeneous ensembles. Bioinform Adv. 2022;2(1):vbac065. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bennett JJR, Li YC, Pandey G. An open-source Python package for multi-modal data integration using heterogeneous ensembles. arXiv. Preprint posted online on January 17, 2024. 2024. [ FREE Full text ] [ CrossRef ]
• Arslanian S, Bacha F, Grey M, Marcus M, White N, Zeitler P. Evaluation and management of youth-onset type 2 diabetes: a position statement by the American Diabetes Association. Diabetes Care. 2018;41(12):2648-2668. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Centers for Disease Control and Prevention. The SAS Program for CDC Growth Charts. SAS Program. URL: https://www.cdc.gov/nccdphp/dnpao/growthcharts/resources/sas.htm [accessed 2024-05-20]
• BernardRosner. Childhood blood pressure macro-batch mode. URL: https:/​/sites.​google.com/​a/​channing.harvard.edu/​bernardrosner/​pediatric-blood-press/​childhood-blood-pressure [accessed 2023-05-19]
• United States Department of Agriculture (USDA). Food consumption and nutrient intake. URL: https://www.ers.usda.gov/data-products/food-consumption-and-nutrient-intakes/ [accessed 2024-05-20]
• Caruana R, Niculescu-Mizil A, Crew G, Ksikes A. Ensemble selection from libraries of models. In: Machine Learning. ACM International Conference Proceeding Series; 2004. Presented at: Proceedings of the Twenty-first International Conference (ICML 2004); July 4-8 2004; Banff, Alberta, Canada.
• Caruana R, Munson A, Niculescu-Mizil A. Getting the most out of ensemble selection. In: Machine Learning. IEEE Computer Society; 2006. Presented at: Proceedings of the 6th {IEEE} International Conference on Data Mining; March 24 2023:828-833; Hong Kong, China. URL: https://www.researchgate.net/publication/220766367_Getting_the_Most_Out_of_Ensemble_Selection
• Brodersen KH, Ong CS, Stephan KE, Buhmann JM. The balanced accuracy and its posterior distribution. In: Machine Learning. IEEE; 2004. Presented at: 2010 20th International Conference on Pattern Recognition; August 23-26 2010; Istanbul, Turkey. URL: https://ieeexplore.ieee.org/document/5597285/authors#authors
• Benjamini Y, Hochberg Y. Controlling the false discovery rate. a practical and powerful approach to multiple testing. 1995;57(1):289-300. [ FREE Full text ]
• R Markdown Format for flexible dashboards. URL: https://pkgs.rstudio.com/flexdashboard/ [accessed 2023-05-18]
• Shiny. Welcome to shiny. URL: https://shiny.posit.co/r/getstarted/shiny-basics/lesson1/index.html [accessed 2023-05-18]
• Kwak SG, Kim JH. Central limit theorem: the cornerstone of modern statistics. Korean J Anesthesiol. 2017;70(2):144-156. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tomczak M, Tomczak E. The need to report effect size estimates revisited. an overview of some recommended measures of effect size. Trends Sport Sci. Feb 15, 2014;1(21):19-25.
• Herman WH, Smith PJ, Thompson TJ, Engelgau MM, Aubert RE. A new and simple questionnaire to identify people at increased risk for undiagnosed diabetes. Diabetes Care. 1995;18(3):382-387. [ CrossRef ] [ Medline ]
• Bang H, Edwards AM, Bomback AS, Ballantyne CM, Brillon D, Callahan MA, et al. Development and validation of a patient self-assessment score for diabetes risk. Ann Intern Med. 2009;151(11):775-783. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Poltavskiy E, Kim DJ, Bang H. Comparison of screening scores for diabetes and prediabetes. Diabetes Res Clin Pract. 2016;118:146-153. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Whalen S, Pandey OP, Pandey G. Predicting protein function and other biomedical characteristics with heterogeneous ensembles. Methods. 2016;93:92-102. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proc 22nd ACM SIGKDD Int Conf Knowl Discov Data Min New York. USA. Association for Computing Machinery; 2016. Presented at: Association for Computing Machinery; 2016; NY. URL: https://dl.acm.org/doi/10.1145/2939672.2939785 [ CrossRef ]
• Shwartz-Ziv R, Armon A. Tabular data: deep learning is not all you need. Information Fusion. May 2022;81:84-90. [ FREE Full text ] [ CrossRef ]
• Goyal K, Dumancic S, Blockeel H. Feature interactions in XGBoost. arXiv. Preprint posted online on July 11, 2020. 2020. [ FREE Full text ] [ CrossRef ]
• Feature Interaction Constraints. XGBoost 2.0.3 documentation. URL: https://xgboost.readthedocs.io/en/stable/tutorials/feature_interaction_constraint.html [accessed 2024-02-01]
• Sesmero MP, Ledezma AI, Sanchis A. Generating ensembles of heterogeneous classifiers using stacked generalization. WIREs Data Min & Knowl. 2015;5(1):21-34. [ CrossRef ]
• Whetzel PL, Noy NF, Shah NH, Alexander PR, Nyulas C, Tudorache T, et al. BioPortal: enhanced functionality via new web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Res. 2011;39(Web Server issue):W541-W545. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bhattacharya S, Andorf S, Gomes L, Dunn P, Schaefer H, Pontius J, et al. ImmPort: disseminating data to the public for the future of immunology. Immunol Res. 2014;58(2-3):234-239. [ CrossRef ] [ Medline ]
• Zhang J, Baran J, Cros A, Guberman JM, Haider S, Hsu J, et al. International Cancer Genome Consortium Data Portal--a one-stop shop for cancer genomics data. Database (Oxford). 2011;2011:bar026. [ FREE Full text ] [ CrossRef ] [ Medline ]
• NHANES - NCHS Research Ethics Review Board Approval. 2022. URL: https://www.cdc.gov/nchs/nhanes/irba98.htm [accessed 2024-01-19]
• Rayner L, McGovern A, Creagh-Brown B, Woodmansey C, de Lusignan S. Type 2 diabetes and asthma: systematic review of the bidirectional relationship. Curr Diabetes Rev. 2019;15(2):118-126. [ CrossRef ] [ Medline ]
• Black MH, Anderson A, Bell RA, Dabelea D, Pihoker C, Saydah S, et al. Prevalence of asthma and its association with glycemic control among youth with diabetes. Pediatrics. 2011;128(4):e839-e847. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wu TD. Diabetes, insulin resistance, and asthma: a review of potential links. Curr Opin Pulm Med. 2021;27(1):29-36. [ CrossRef ] [ Medline ]
• Vartanian LR, Schwartz MB, Brownell KD. Effects of soft drink consumption on nutrition and health: a systematic review and meta-analysis. Am J Public Health. 2007;97(4):667-675. [ CrossRef ] [ Medline ]
• Greenwood DC, Threapleton DE, Evans CEL, Cleghorn CL, Nykjaer C, Woodhead C, et al. Association between sugar-sweetened and artificially sweetened soft drinks and type 2 diabetes: systematic review and dose-response meta-analysis of prospective studies. Br J Nutr. 2014;112(5):725-834. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Malik VS, Popkin BM, Bray GA, Després JP, Willett WC, Hu FB. Sugar-sweetened beverages and risk of metabolic syndrome and type 2 diabetes: a meta-analysis. Diabetes Care. 2010;33(11):2477-2483. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Muraki I, Imamura F, Manson JE, Hu FB, Willett WC, van Dam RM, et al. Fruit consumption and risk of type 2 diabetes: results from three prospective longitudinal cohort studies. BMJ. 2013;347:f5001. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McDonough C, Li Y. Youth preDM/DM dataset and Case Studies. Switzerland. Zenodo; 2024. URL: https://zenodo.org/records/10531245 [accessed 2024-05-29]

Abbreviations

Edited by A Mavragani, T Sanchez; submitted 05.10.23; peer-reviewed by S El Khamlichi, C Zhao, Y Su; comments to author 09.01.24; revised version received 06.02.24; accepted 26.04.24; published 02.07.24.

©Catherine McDonough, Yan Chak Li, Nita Vangeepuram, Bian Liu, Gaurav Pandey. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 02.07.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.

This paper is in the following e-collection/theme issue:

Published on 3.7.2024 in Vol 26 (2024)

Artificial Intelligence–Based Electrocardiographic Biomarker for Outcome Prediction in Patients With Acute Heart Failure: Prospective Cohort Study

Authors of this article:

Original Paper

• Youngjin Cho 1, 2 * , MD, PhD   ; 
• Minjae Yoon 1 * , MD   ; 
• Joonghee Kim 2, 3 , MD, PhD   ; 
• Ji Hyun Lee 1 , MD, PhD   ; 
• Il-Young Oh 1 , MD, PhD   ; 
• Chan Joo Lee 4 , MD, PhD   ; 
• Seok-Min Kang 4 , MD, PhD   ; 
• Dong-Ju Choi 1 , MD, PhD  

1 Division of Cardiology, Department of Internal Medicine, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Gyeonggi-do, Republic of Korea

2 ARPI Inc, Seongnam, Gyeonggi-do, Republic of Korea

3 Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam, Gyeonggi-do, Republic of Korea

4 Division of Cardiology, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea

*these authors contributed equally

Corresponding Author:

Dong-Ju Choi, MD, PhD

Division of Cardiology, Department of Internal Medicine

Seoul National University Bundang Hospital

Seoul National University College of Medicine

82 Gumi-ro 173 Beon-gil

Seongnam, Gyeonggi-do, 13620

Republic of Korea

Phone: 82 317877007

Fax:82 317877041

Email: [email protected]

Background: Although several biomarkers exist for patients with heart failure (HF), their use in routine clinical practice is often constrained by high costs and limited availability.

Objective: We examined the utility of an artificial intelligence (AI) algorithm that analyzes printed electrocardiograms (ECGs) for outcome prediction in patients with acute HF.

Methods: We retrospectively analyzed prospectively collected data of patients with acute HF at two tertiary centers in Korea. Baseline ECGs were analyzed using a deep-learning system called Quantitative ECG (QCG), which was trained to detect several urgent clinical conditions, including shock, cardiac arrest, and reduced left ventricular ejection fraction (LVEF).

Results: Among the 1254 patients enrolled, in-hospital cardiac death occurred in 53 (4.2%) patients, and the QCG score for critical events (QCG-Critical) was significantly higher in these patients than in survivors (mean 0.57, SD 0.23 vs mean 0.29, SD 0.20; P <.001). The QCG-Critical score was an independent predictor of in-hospital cardiac death after adjustment for age, sex, comorbidities, HF etiology/type, atrial fibrillation, and QRS widening (adjusted odds ratio [OR] 1.68, 95% CI 1.47-1.92 per 0.1 increase; P <.001), and remained a significant predictor after additional adjustments for echocardiographic LVEF and N-terminal prohormone of brain natriuretic peptide level (adjusted OR 1.59, 95% CI 1.36-1.87 per 0.1 increase; P <.001). During long-term follow-up, patients with higher QCG-Critical scores (>0.5) had higher mortality rates than those with low QCG-Critical scores (<0.25) (adjusted hazard ratio 2.69, 95% CI 2.14-3.38; P <.001).

Conclusions: Predicting outcomes in patients with acute HF using the QCG-Critical score is feasible, indicating that this AI-based ECG score may be a novel biomarker for these patients.

Trial Registration: ClinicalTrials.gov NCT01389843; https://clinicaltrials.gov/study/NCT01389843

Introduction

Heart failure (HF) is a major global health problem affecting millions of people worldwide, leading to significant morbidity, mortality, and health care expenditure [ 1 - 3 ]. Although several valuable biomarkers such as N-terminal prohormone of brain natriuretic peptide (NT-proBNP) [ 4 , 5 ] and cardiac troponins [ 6 ] have been introduced for patients with HF, their use in routine clinical practice is often constrained by their cost and limited availability.

Electrocardiogram (ECG) is an essential and cost-effective tool for evaluating cardiovascular diseases. ECG is widely available, noninvasive, and provides real-time information about cardiac electrical activity, which is crucial for detecting arrhythmias, ischemia, and other cardiac abnormalities. With advances in artificial intelligence (AI) and deep learning, there has been growing interest in employing AI algorithms to analyze ECG data and predict outcomes in patients with various cardiovascular conditions [ 7 , 8 ].

In this study, we investigated the utility of an AI algorithm that analyzes printed ECG images for outcome prediction in patients with acute HF. These findings will demonstrate the potential of AI-assisted ECG analysis for predicting outcomes in these patients, potentially overcoming the cost and availability constraints of current biomarkers.

Study Population

This was a substudy of the prospective multicenter Korean Acute Heart Failure (KorAHF) registry, which enrolled 5625 consecutive patients upon initial hospital admission for acute HF at 10 tertiary university hospitals in Korea. Details on the KorAHF registry objectives, design, and population are available on the clinical trial registration site (ClinicalTrials.gov NCT01389843) and have been published previously [ 9 , 10 ]. Briefly, patients who had signs or symptoms of HF and met one of the following criteria were eligible for enrollment in the KorAHF registry: (1) lung congestion or (2) objective left ventricular systolic dysfunction or structural heart disease findings. There were no exclusion criteria.

In this study, we retrospectively analyzed the prospectively collected data from 1254 patients who were hospitalized for acute HF from March 2011 to February 2014 at 2 out of 10 participating tertiary centers (Seoul National University Bundang Hospital and Severance Hospital) using the KorAHF registry ( Figure 1 ). Additional ECG image data were collected for this study.

Ethical Considerations

This study conformed with the principles outlined in the Declaration of Helsinki. The study protocol was approved by the institutional review board at Seoul National University Bundang Hospital (No. B-1104-125-014) and Severance Hospital (No. 2022-2166-001). The need for written informed consent was waived by the institutional review board. Our research strictly adheres to the Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research [ 11 ].

Clinical Follow-Up and Endpoints

Data collection methods have been previously described [ 9 ]. Briefly, data on patients’ clinical manifestations, biochemical parameters, medication, and outcome were collected using a web-based case report form for up to 60 months by research nurses. Outcome data on patients lost to follow-up were additionally collected from national death records.

The primary endpoint of this study was all-cause mortality. Secondary outcomes included in-hospital outcomes, especially in-hospital mortality. All deaths were considered to be cardiac-related unless a definite noncardiac cause could be established. All outcome data reported from the participating centers were reviewed by an independent clinical event adjudicating committee.

AI Algorithm

Quantitative ECG (QCG) is an AI analyzer composed of an encoder part and multiple task-specific networks. The encoder part is a modified convolutional neural network with residual connections, squeeze excitation modules, and a nonlocal block. The task-specific networks are multilayer percetron models. The encoder part accepts 2D ECG images as input to produce a common numerical feature vector for downstream tasks. The encoder part was pretrained on 49,731 open ECGs using self-supervised learning schemes and then fine-tuned on 47,194 annotated ECG images of over 32,968 patients who visited the Emergency Department of Seoul National University Bundang Hospital between 2017 and 2019 using multitask learning schemes. The tasks include the classification of 12 rhythms (with 35 subtypes) and production of 10 digital biomarkers correlated with the risk of (1) being critically ill (shock, respiratory failure, or cardiac arrest), (2) cardiac ischemia (acute coronary syndrome, ST-elevation myocardial infarction, or myocardial injury as defined by an elevated troponin level), (3) cardiac dysfunction (pulmonary edema, left and right heart dysfunction, pulmonary hypertension, and clinically significant pericardial effusion), and (4) hyperkalemia. Several validation studies of the system have been published previously [ 12 - 14 ]. The collection of these AI algorithms has been developed into a mobile app (ECG Buddy, ARPI), which has been approved by the Korean Ministry of Food and Drug Safety.

In this study, two QCG features were evaluated: QCG-Critical for critical conditions such as shock or mortality and QCG-HF for a reduced echocardiographic left ventricular ejection fraction (LVEF) of <40%. The QCG scores, representing probability, ranged from 0 to 1.0, with 0 indicating low and 1.0 indicating high probability. With a 9:1 ratio split of the training and test data sets, the internal validation results for these two QCG features showed an area under the curve (AUC) of 0.877 for QCG-Critical and 0.956 for QCG-HF. The composition of the training and validation data sets is presented as a flowchart in Figure 1 .

Statistical Analysis

Categorical variables are reported as frequencies (percentages) and continuous variables are expressed as means (SD) or medians (IQR). The two key AI-driven scores (QCG-Critical and QCG-HF) were analyzed as continuous variables. The Student t test and χ 2 (or Fisher exact) test were used to compare the baseline clinical characteristics between the two groups. The discrimination performance of QCG scores for in-hospital outcomes was evaluated using receiver operating characteristic (ROC) curve analysis. The AUC values were compared using the DeLong test. The logistic regression model was used to estimate the odds ratios (ORs) and 95% CIs. Survival analysis was performed using the Kaplan-Meier method, and the Cox proportional hazard model was used to estimate the hazard ratios (HRs) and 95% CIs for the clinical outcomes. Multivariable analysis was performed with the inclusion of clinically relevant variables.

All tests were two-tailed and a P value <.05 was considered statistically significant. Statistical analyses were performed using R programming version 4.3.0 (The R Foundation for Statistical Computing).

Baseline Characteristics

Data of 1254 patients (716 from Seoul National University Bundang Hospital and 538 from Severance Hospital) were analyzed. Among the 1254 patients, 53 (4.2%) experienced in-hospital cardiac death. The baseline characteristics of the study population according to the in-hospital outcomes are shown in Table 1 . Compared with survivors, patients who died in the hospital were older, had a higher prevalence of ischemic heart disease, lower LVEF, and higher NT-proBNP levels. By contrast, atrial fibrillation (AF) was more frequent in survivors. The QCG-Critical and QCG-HF scores were significantly higher in patients who experienced in-hospital cardiac death than in survivors ( P <.001) ( Table 1 and Figure S1 in Multimedia Appendix 1 ).

a HF: heart failure.

b LVEF: left ventricular ejection fraction.

c NT-proBNP: N-terminal prohormone of brain natriuretic peptide.

d QCG: Quantitative electrocardiogram artificial intelligence system.

Predictors of In-Hospital Cardiac Death

In the univariable logistic regression analysis, the QCG-Critical and QCG-HF scores were significant predictors of in-hospital cardiac death ( Table 2 ). Other than QCG scores, echocardiographic LVEF, NT-proBNP level, age, ischemic heart disease, and AF were significantly correlated with in-hospital cardiac death ( Table 2 ).

a Adjusted for age, sex, hypertension, diabetes, chronic kidney disease, cerebrovascular disease, ischemic heart disease, valvular heart disease, heart failure type, atrial fibrillation, and QRS duration.

b When a variable was included as a covariate for adjustment, it was not adjusted for itself and QCG-Critical was added to the adjustment model (presented in italics).

c Adjusted for the same covariates as model 1 and further adjusted for left ventricular ejection fraction and N-terminal prohormone of brain natriuretic peptide.

d OR: odds ratio.

e QCG: Quantitative electrocardiogram.

f HF: heart failure.

g LVEF: left ventricular ejection fraction.

h NT-proBNP: N-terminal prohormone of brain natriuretic peptide.

i ADHF: acute decompensated heart failure.

After adjustment for age, sex, hypertension, diabetes, chronic kidney disease, cerebrovascular disease, ischemic heart disease, valvular heart disease, HF type, AF, and QRS duration, the two QCG scores remained significant predictors of in-hospital cardiac death. Moreover, the QCG-Critical score was an independent predictor of in-hospital cardiac death after further adjustment for echocardiographic LVEF and NT-proBNP level (OR 1.59, 95% CI 1.36-1.87; P <.001).

In a subgroup analysis, the QCG-Critical score was a significant predictor of in-hospital cardiac death regardless of the initial rhythm (AF or sinus rhythm), QRS width (wide or narrow), hypertension, diabetes, HF etiology (ischemic or nonischemic), HF type (de novo or acute decompensated HF), and LVEF (HF with reduced ejection fraction vs HF with preserved or mildly reduced ejection fraction), after adjustment for other clinical parameters ( Figure 2 ).

QCG-Critical Score and In-Hospital Cardiac Death

The QCG-Critical score was significantly higher in patients who experienced cardiac death within 1 day, 2 days, or during hospitalization than in survivors ( Figure 3 A). When the performance of the QCG-Critical score for predicting these events was analyzed using ROC curves, the AUC values for 1- and 2-day mortality and in-hospital cardiac death were 0.936, 0.917, and 0.821, respectively ( Figure 3 B).

Comparatively, the AUC values of echocardiographic LVEF and NT-proBNP level for predicting in-hospital cardiac death were 0.642 ( P <.001 vs QCG-Critical) and 0.720 ( P =.07 vs QCG-Critical) ( Figure 4 A). The AUC value of the QCG-Critical score (0.821) was significantly ( P =.02). higher than that of model 1 (0.705) established using traditional clinical variables, including age, sex, hypertension, diabetes, chronic kidney disease, cerebrovascular disease, ischemic heart disease, valvular heart disease, HF type, AF, and QRS duration. In addition, when the QCG-Critical score was added to model 1, it significantly enhanced the prediction for in-hospital cardiac death (AUC of model 1=0.705 vs AUC of model 1 with QCG-Critical=0.843; P <.001) ( Figure 4 B). When NT-proBNP and LVEF were further included in model 1 (model 2), the QCG-Critical score again demonstrated additional predictive value for in-hospital cardiac death compared to model 2 alone (AUC of model 2=0.787 vs AUC of model 2 with QCG-Critical=0.863; P =.01) ( Figure 4 C).

QCG-Critical Score and Long-Term Outcomes

During a median follow-up of 2.7 years, 508 deaths occurred in the study population. To further analyze the performance of the QCG-Critical score for outcome prediction, we divided patients into three QCG-Critical score groups based on arbitrary cut-off values of 0.25 and 0.50 and then conducted survival analysis ( Figure 5 ).

After adjustment for age, sex, comorbidities, HF etiology and type, AF, and QRS widening, patients with higher QCG-Critical scores had significantly higher all-cause mortality rates during follow-up than those with lower QCG-Critical scores (<0.25). The adjusted HRs for patients with QCG-Critical scores between 0.25 and 0.50 and for patients with QCG-Critical scores higher than 0.50 were 1.57 (95% CI 1.28-1.93) and 2.69 (95% CI 2.14-3.38), respectively (all P <.001). With additional adjustment for LVEF and NT-proBNP to the previous model, the adjusted HRs were 1.61 and 2.27, respectively, consistent with the main analysis (Figure S2 in the Multimedia Appendix 1 ).

In a subgroup analysis, a higher QCG-Critical score (>0.50 vs ≤0.50) was significantly correlated with all-cause mortality during follow-up, regardless of the initial rhythm (AF or sinus rhythm), QRS width (wide or narrow), hypertension, diabetes, HF etiology, HF type (de novo or acute decompensated HF), and LVEF (HF with reduced ejection fraction vs HF with preserved or mildly reduced ejection fraction), after adjustment for other clinical parameters (Figure S3 in Multimedia Appendix 1 ).

Predicting outcomes in patients with HF is important for guiding management and improving prognosis [ 15 ] but is often hindered by the complexity of HF pathophysiology and the presence of other comorbidities. Recently, AI algorithms based on big data from medical records have been found to be helpful in predicting the outcomes of patients with HF [ 16 , 17 ]; however, these algorithms are difficult to apply in daily practice and their performance requires further improvements. In this study, the QCG-Critical score, a newly developed AI-based ECG score, was well correlated with early mortality and in-hospital cardiac death during the index after adjusting for traditional clinical risk factors. Moreover, the QCG-Critical score was an independent predictor of long-term all-cause mortality in this population, suggesting that this AI-based ECG score may serve as a novel biomarker for these patients.

ECG is a cost-effective, widely available, and easy-to-perform test, and is therefore often used as a first-line evaluation for patients with cardiovascular diseases. ST-elevation myocardial infarction is a quintessential disease where ECG evaluation is critical for a timely diagnosis. Although ECG is not deterministic for an HF diagnosis, several studies have demonstrated that some ECG features are correlated with the characteristics of HF [ 18 ]. In addition, the presence of AF or QRS widening may represent ECG features reflecting unfavorable underlying hemodynamics, thus correlating with a poor prognosis [ 19 , 20 ]. More subtle ECG changes have also been suggested as predictors of a poor prognosis in patients with HF; however, these require high levels of experience and skill for interpretation, which may limit their applicability [ 21 ].

Theoretically, the ECG signal may contain information regarding the electric and mechanical activities of the diseased heart beyond a physician’s perception. With the assistance of AI, ECG may provide valuable information beyond its current usage. For example, Attia et al [ 22 ] reported that LVEF reduction may be detected by ECG using AI. This new application of AI-ECG was reproduced by other researchers [ 23 , 24 ]. In this study, the QCG-HF score also showed good performance in predicting reduced echocardiographic LVEF of less than 40%, with an AUC value of 0.884 (Figure S4 in Multimedia Appendix 1 ). Notably, in the above-mentioned studies, the AI-ECG–predicted LVEF was correlated with the prognosis of patients with chronic HF, whereas the AI-based ECG score had a predictive value in patients with acute HF in this study. Thus, to the best of our knowledge, this study represents an initial effort in terms of predicting the outcomes of acute HF using AI-based ECG interpretation.

The QCG-Critical score was originally trained to detect critical medical conditions that may result in shock or mortality within 1 day [ 12 ]. In this study, the QCG-Critical score predicted early cardiac mortality in patients with acute HF with high accuracy. The AUC value of the QCG-Critical score was higher than that of echocardiographic LVEF for the prediction of in-hospital cardiac death and was also higher than the AUC value of the serum NT-proBNP level, but without statistical significance. Notably, the QCG-Critical score was available for all 1254 patients enrolled in the KorAHF study, whereas LVEF and NT-proBNP results were not available in 68 (5.4%) and 168 (13.4%) patients, respectively. Considering that the KorAHF study enrolled patients from tertiary centers in Korea, a high proportion of patients with acute HF might not have the opportunity to benefit from these echocardiographic or serum biomarker tests in real-world practice. Because ECG is a widely available evaluation tool and QCG scores are derived from ECG images, the QCG-Critical score may serve as an adequate alternative biomarker for risk stratification of patients with acute HF in real-world settings with limited resources. This score may also be useful even in well-equipped centers because it would be available immediately after the ECG exam, without requiring additional waiting for echocardiography or laboratory tests. This may be beneficial for timely risk stratification in the emergency department. The QCG-Critical score was not only correlated with in-hospital cardiac death but also showed a strong association with long-term mortality. In addition, the subgroup analysis demonstrated a consistent correlation between the QCG-Critical score and clinical outcomes. These results emphasize the potential of AI-based ECG interpretation as a novel biomarker in this field.

This study has several limitations. First, the study population predominantly consisted of Asian patients; hence, further studies are needed to validate our results across different ethnicities. Second, the AI algorithm tested in this study was derived from one of the participating centers (Seoul National University Bundang Hospital). However, there was a temporal difference between patient enrollment for algorithm training (2017 to 2019) and the test population (KorAHF enrollment, 2011 to 2014), and another external center (Severance Hospital) was involved in this study. Nevertheless, this may limit the generalizability of our findings. Third, the ECG format may affect the algorithm’s performance. Although the manufacturers of the ECG devices used in the two participating hospitals differed (Philips PageWriter TC 30 and TC 70 at Seoul National University Bundang Hospital and GE Healthcare MAC 5500 and MAC VU360 at Severance Hospital), there was no significant difference in the AI algorithm performance between the hospitals. However, because the system uses printed ECG images as input, there may be problematic scenarios where the qualities of the images influence the predictive power of the biomarkers. Although some recent AI algorithm–based studies suggest further interpretation analysis, the QCG system does not support gradient-weighted class-activation mapping or similar visualization for model explainability due to the custom network architecture used. Therefore, we could not evaluate which part of the ECG images the system uses for each prediction.

In conclusion, predicting outcomes in patients with acute HF using the newly developed AI-based ECG score appears to be feasible. Thus, this score may serve as a novel biomarker for patients with HF, potentially overcoming the cost and availability constraints of current biomarkers.

Acknowledgments

YC and MY are co-first authors, and SMK and DJC are co-corresponding authors. Author SMK can be reached at [email protected] for correspondence.

This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by the Ministry of Health & Welfare, Republic of Korea (grant RS-2023-00265933). This work was also supported by the Seoul National University Bundang Hospital Research Fund (grant 13-2024-0009).

Conflicts of Interest

JK and YC are employed by ARPI Inc and hold dual roles as CEO and Head of Research Collaboration Center, respectively. The other authors have no conflicts of interest to declare.

Distribution of QCG-Critical scores (Figure S1); Kaplan-Meier curves for long-term mortality according to the QCG-Critical scores and adjusted HRs with additional adjustment for LVEF and NT-proBNP (Figure S2); subgroup analysis results for predicting long-term mortality (Figure S3); performance of the QCG-HF score for diagnosing left ventricular dysfunction (Figure S4).

• Heidenreich PA, Bozkurt B, Aguilar D, Allen LA, Byun JJ, Colvin MM, et al. 2022 AHA/ACC/HFSA Guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. J Am Coll Cardiol. May 03, 2022;79(17):e263-e421. [ CrossRef ] [ Medline ]
• McDonagh T, Metra M, Adamo M, Gardner RS, Baumbach A, Böhm M, et al. ESC Scientific Document Group. 2021 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur Heart J. Sep 21, 2021;42(36):3599-3726. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Park JJ, Lee CJ, Park S, Choi J, Choi S, Park S, et al. Heart failure statistics in Korea, 2020: a report from the Korean Society of Heart Failure. Int J Heart Fail. Oct 2021;3(4):224-236. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Januzzi JL, Rehman SU, Mohammed AA, Bhardwaj A, Barajas L, Barajas J, et al. Use of amino-terminal pro-B-type natriuretic peptide to guide outpatient therapy of patients with chronic left ventricular systolic dysfunction. J Am Coll Cardiol. Oct 25, 2011;58(18):1881-1889. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Masson S, Latini R, Anand IS, Barlera S, Angelici L, Vago T, et al. Val-HeFT Investigators. Prognostic value of changes in N-terminal pro-brain natriuretic peptide in Val-HeFT (Valsartan Heart Failure Trial). J Am Coll Cardiol. Sep 16, 2008;52(12):997-1003. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peacock WF, De Marco T, Fonarow GC, Diercks D, Wynne J, Apple FS, et al. ADHERE Investigators. Cardiac troponin and outcome in acute heart failure. N Engl J Med. May 15, 2008;358(20):2117-2126. [ CrossRef ] [ Medline ]
• Verbrugge FH, Reddy YNV, Attia ZI, Friedman PA, Noseworthy PA, Lopez-Jimenez F, et al. Detection of left atrial myopathy using artificial intelligence-enabled electrocardiography. Circ Heart Fail. Jan 2022;15(1):e008176. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Attia ZI, Noseworthy PA, Lopez-Jimenez F, Asirvatham SJ, Deshmukh AJ, Gersh BJ, et al. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. Lancet. Sep 07, 2019;394(10201):861-867. [ CrossRef ] [ Medline ]
• Lee SE, Cho H, Lee H, Yang H, Choi J, Jeon E, et al. A multicentre cohort study of acute heart failure syndromes in Korea: rationale, design, and interim observations of the Korean Acute Heart Failure (KorAHF) registry. Eur J Heart Fail. Jun 02, 2014;16(6):700-708. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cho Y, Cho S, Oh I, Lee JH, Park JJ, Lee H, et al. Implantable cardioverter-defibrillator utilization and its outcomes in Korea: data from Korean Acute Heart Failure Registry. J Korean Med Sci. Nov 30, 2020;35(46):e397. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. Dec 16, 2016;18(12):e323. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kim D, Hwang JE, Cho Y, Cho H, Lee W, Lee JH, et al. A retrospective clinical evaluation of an artificial intelligence screening method for early detection of STEMI in the emergency department. J Korean Med Sci. Mar 14, 2022;37(10):e81. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Choi YJ, Park MJ, Ko Y, Soh M, Kim HM, Kim CH, et al. Artificial intelligence versus physicians on interpretation of printed ECG images: diagnostic performance of ST-elevation myocardial infarction on electrocardiography. Int J Cardiol. Sep 15, 2022;363:6-10. [ CrossRef ] [ Medline ]
• Park J, Yoon Y, Cho Y, Kim J. Feasibility of artificial intelligence-based electrocardiography analysis for the prediction of obstructive coronary artery disease in patients with stable angina: validation study. JMIR Cardio. May 02, 2023;7:e44791. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fonarow GC, Albert NM, Curtis AB, Stough WG, Gheorghiade M, Heywood JT, et al. Improving evidence-based care for heart failure in outpatient cardiology practices: primary results of the Registry to Improve the Use of Evidence-Based Heart Failure Therapies in the Outpatient Setting (IMPROVE HF). Circulation. Aug 10, 2010;122(6):585-596. [ CrossRef ] [ Medline ]
• Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, et al. A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak. Jun 22, 2018;18(1):44. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ahmad T, Lund LH, Rao P, Ghosh R, Warier P, Vaccaro B, et al. Machine learning methods improve prognostication, identify clinically distinct phenotypes, and detect heterogeneity in response to therapy in a large cohort of heart failure patients. J Am Heart Assoc. Apr 12, 2018;7(8):e008081. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hendry PB, Krisdinarti L, Erika M. Scoring system based on electrocardiogram features to predict the type of heart failure in patients with chronic heart failure. Cardiol Res. Jun 2016;7(3):110-116. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lee JH, Park JJ, Cho Y, Oh IY, Yoo BS, Kim JJ, et al. Prognostic implication of ventricular conduction disturbance pattern in hospitalized patients with acute heart failure syndrome. Korean Circ J. Jul 2019;49(7):602-611. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang NC, Maggioni AP, Konstam MA, Zannad F, Krasa HB, Burnett JC, et al. Efficacy of Vasopressin Antagonism in Heart Failure Outcome Study With Tolvaptan (EVEREST) Investigators. Clinical implications of QRS duration in patients hospitalized with worsening heart failure and reduced left ventricular ejection fraction. JAMA. Jun 11, 2008;299(22):2656-2666. [ CrossRef ] [ Medline ]
• Das MK, Maskoun W, Shen C, Michael MA, Suradi H, Desai M, et al. Fragmented QRS on twelve-lead electrocardiogram predicts arrhythmic events in patients with ischemic and nonischemic cardiomyopathy. Heart Rhythm. Jan 2010;7(1):74-80. [ CrossRef ] [ Medline ]
• Attia ZI, Kapa S, Lopez-Jimenez F, McKie PM, Ladewig DJ, Satam G, et al. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat Med. Jan 7, 2019;25(1):70-74. [ CrossRef ] [ Medline ]
• Choi J, Lee S, Chang M, Lee Y, Oh GC, Lee H. Deep learning of ECG waveforms for diagnosis of heart failure with a reduced left ventricular ejection fraction. Sci Rep. Aug 20, 2022;12(1):14235. [ CrossRef ] [ Medline ]
• Kwon J, Kim K, Jeon K, Kim HM, Kim MJ, Lim S, et al. Development and validation of deep-learning algorithm for electrocardiography-based heart failure identification. Korean Circ J. Jul 2019;49(7):629-639. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani; submitted 27.08.23; peer-reviewed by S Khan, J Zeng; comments to author 08.02.24; revised version received 22.02.24; accepted 29.05.24; published 03.07.24.

©Youngjin Cho, Minjae Yoon, Joonghee Kim, Ji Hyun Lee, Il-Young Oh, Chan Joo Lee, Seok-Min Kang, Dong-Ju Choi. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 03.07.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 27 June 2024

Suppression or promotion: research on the impact of industrial structure upgrading on urban economic resilience

• Lu Zhang 1 ,
• Guodong Lin 1 ,
• Xiao Lyu 2 &
• Wenjie Su 3  

Humanities and Social Sciences Communications volume  11 , Article number:  843 ( 2024 ) Cite this article

96 Accesses

Metrics details

• Social policy

Industrial The upgrading of industrial structure, as the main means of urban economic transformation, plays a crucial role in the process of achieving urban economic resilience construction. We conducted a study on the nonlinear impact mechanism of industrial structure upgrading on urban economic resilience based on panel data from 267 prefecture-level and above-level cities and above in China from 2008 to 2021, using globalization as a threshold variable. The obtained results demonstrated the following: (1) there existed a significant nonlinear relationship between industrial structure upgrading and rationalization and urban economic resilience, with a significant double threshold effect. (2) A robustness test was performed by removing extreme values from the sample, controlling for the time series and individual interaction terms while considering control variables, which did not change the basic conclusions based on the model. This demonstrated that the threshold regression model constructed in this study is robust and reliable. (3) From a regional heterogeneity perspective, the impact of industrial structure upgrading on urban economic resilience varied among different regions. Notably, industrial structure upgrading imposed a significant double threshold effect on urban economic resilience in the eastern and central regions, manifested as an inverted U-shaped trend. In the northeastern region, there was only a single threshold effect with globalization as the threshold variable, which still occurred on the left side of the inverted U-shaped curve, while no threshold effect was observed in the western region.

Similar content being viewed by others

Digital economy development boosts urban resilience—evidence from China

Smart city construction and new-type urbanization quality improvement

Industrial upgrading and its influence on green land use efficiency

Introduction.

As the main focus area of economic activity and the core subject of mitigating various risks, enhancing the economic resilience of cities is a necessary means to effectively enhance the ability of their economic systems to resist risks and shocks (Fingleton and Palombi, 2013 ; Martin and Sunley, 2014 ; Cheng et al. 2022 ; Wang and Wang, 2021 ; Papaioannou, 2023 ; Fan et al. 2023 ). With the increasingly close global socioeconomic development, economic cooperation among countries is gradually moving toward a path of diversification, openness, and sharing (Zhou and Qi, 2023 ; Hynes et al. 2022 ; Jayasinghe et al. 2022 ; Gajewski, 2022 ). While countries worldwide share the fruits of economic development due to globalization, some countries have become “shock absorbers” for the cyclic regulation of the international economic system (Andrew et al. 2020 ; Ye and Qian, 2021 ; Ben and Ifergane, 2022 ). In particular, problems related to notable fluctuations in economic development, limited defense capabilities, and low competitiveness are particularly prominent in developing countries. These problems further exacerbate the vulnerability of cities in various countries in response to internal and external changes (Ženka et al. 2019 ; Wang et al. 2021 ; Mai et al. 2021 ). Therefore, how to build a strong and resilient economic system and employ resilience thinking to enhance the driving force and capacity of regional economic development has become an important research topic for countries globally, with the aim of promoting high-quality urban economic development.

The term resilience first evolved from the Latin word “resilio” (Yang et al. 2023 ). With the increase in the occurrence of uncertain events and external shocks, scholars have applied resilience in fields such as urban engineering resilience, ecological resilience, and economic resilience (Dario and Weterings, 2015 ; Paolo, 2017 ; Lemke et al. 2023 ; Du, 2023 ; Yu et al. 2023 ). The connotation of urban economic resilience has been widely investigated by the government, society, and academia, as it better conforms with the current stage of urban economic development in certain countries and the interpretation of the practical problems faced (Du et al. 2023 ; Gai and Yang, 2023 ; Hui and Tan, 2023 ). Urban economic resilience refers to the ability of a city to prevent and resist risks, as well as maintain efficient and sustainable economic development during a specific period (Drobniak, 2017 ; Tan et al. 2017 ; Pashapour et al. 2019 ; Erika and Mangirdas, 2020 ). Today’s world is vulnerable to severe impacts such as economic crises, epidemics, and natural disasters. Some regions may continue to maintain stable economic growth after impact, while others may suffer heavy losses and fail to recover (Deng et al. 2023 ; Wang et al. 2023 ; Lee and Wang, 2023 ). The reason for this difference is that countries with greater urban economic resilience often exhibit characteristics such as dynamic balance, redundant buffering, and self-healing, which can enable these countries to quickly eliminate risks and automatically adjust and recover, thus effectively resisting external shocks and mitigating internal disasters (Cheng et al. 2023 ; Zhou and Qi, 2023 ). Notably, 2008 and 2019 were critical time points in terms of global fluctuations and changes. Under the impacts of the global crisis and the COVID-19 pandemic, respectively, Compared with developed countries such as the United States and Japan, although China’s economic development has also encountered obstacles, its economic growth rate has decreased by 1.8% and 0.1% respectively compared to the previous year, But China’s long-term economic fundamentals have not changed, and the characteristics of sustained economic recovery, notable development potential, high resilience and broad space have not changed (Hao, 2023 ; Wang et al. 2024 ; Yang, 2023 ). For example, under the influence of the COVID-19 pandemic, the economies of the United States and Japan shrank by 3.5 and 5.3%, respectively, in 2021. On the one hand, the economy of China benefited from a series of policy documents issued by the Chinese government on “optimizing the environment, expanding the domestic demand, stabilizing growth, and promoting development”. On the other hand, it benefited from a strong labor force and high consumer market supply capacity.

Industrial structure upgrading is important for exploring the economic resilience of Chinese cities (Drobniak, 2017 ; Zhou et al. 2019 ; Betts and Buzzanell, 2022 ). Especially at this stage, China’s economy is transitioning from high-speed growth to high-quality development, and industrial structure upgrading plays an important role in the urban economy resilience process in different regions and stages (Zhang, 2022 ; Yin et al. 2023 ). On the one hand, industrial structure upgrading can cause an acceleration in regional industrial structure transformation from traditional high energy-consuming industries to high-tech industries. In this process, with the emergence of an advanced industrial structure and a specialized division of labor, new economic growth paths can be created, which can enhance the ability of cities to withstand market risks and can facilitate resilient growth of the urban economy (Tan et al. 2017 ; Cheng et al. 2023 ; Li et al. 2024 ). On the other hand, industrial structure upgrading can lead to enhancement in the service industry, but a high proportion of the service industry can easily cause the problem of hollowing out industries, which is not conducive to improving urban economic resilience (Gai and Yang, 2023 ; Hui and Tan, 2023 ). It should be noted that industrial structure upgrading entails a long and tortuous process, and its impact on urban economic resilience cannot be achieved overnight. Therefore, the causal relationship between industrial structure upgrading and urban economic resilience is not direct nor obvious. However, the impact of the former on the latter still objectively occurs through various transmission mechanisms. Moreover, due to the poor coherence and sustainability of relevant systems, this impact relationship may be repetitive and not simply linear.

From 1492 to 2023, globalization increased from the 1.0 era to the 4.0 era (Roberta et al. 2015 ; Aida et al. 2016 ; Gereffi et al. 2022 ). Globalization has firmly woven all countries into the network of the world system, and the commerce, economy, and system of each country have undergone tremendous transformation (Dunn, 2020 ; Carlos et al. 2022 ). In the globalization process, achieving high-quality and sustainable economic development has become a major challenge for the international community (Ngo, 2023 ). Since its accession to the World Trade Organization (WTO), China has received a large amount of foreign direct investment due to its own resource endowment. Foreign direct investment has, to a certain extent, accelerated the process of high-quality development of China’s urban economy, which is mainly reflected in two aspects: on the one hand, foreign direct investment can effectively promote industrial structure upgrading and optimization (Tao et al. 2023 ). Foreign direct investment usually leads to the introduction of technology, management, and market factors, accelerating the transformation of traditional Chinese industries to high-tech and high value-added industries and promoting sustainable development of the urban economy (Anis and Andreea, 2023 ). On the other hand, foreign direct investment not only provides financial support but also introduces advanced technology and management experience, providing Chinese industries with a larger market and more abundant resources (Knoke et al. 2022 ; Gereffi et al. 2022 ). Production scale increase, product quality improvement, and cost reduction further enhance the competitiveness of China’s industries (Carlos et al. 2022 ; Johnson and Mundell, 2023 ). Therefore, we must consider several questions: under the acceleration of globalization, what is the impact of industrial structure upgrading (with a focus on industrial structure upgrading and rationalization) on urban economic resilience? Are there certain stage characteristics? These questions should be explored in depth. To this end, this study links the above three aspects and focuses on determining whether there exists a threshold effect in terms of the impact of industrial structure upgrading on urban economic resilience with globalization as the threshold variable, examining its mechanism and effect, and investigating whether there is regional heterogeneity to provide an empirical basis for relevant departments to formulate targeted policies.

The remainder of this study is organized as follows: in the second section, a literature review is provided, in which the existing research on industrial structure upgrading, globalization, and urban economic resilience is summarized, providing a sound basis for this analysis. The third section constructs a theoretical framework and research hypotheses, elaborates on the theoretical basis of industrial structure upgrading, explores the impact of industrial structure upgrading and rationalization on urban economic resilience in the context of globalization, and proposes research hypotheses. In the fourth section, the research design is described in detail, including model settings, variable selection, data sources, and descriptive statistical analysis. In the fifth section, the empirical tests are introduced, including threshold effect assessment, threshold panel model-based estimation, and regional heterogeneity analysis. Finally, the sixth and seventh selections, respectively outline the conclusions, policy recommendations, and future prospects of this article.

Literature review

The study of urban economic resilience has become an important topic for countries to explore high-quality and sustainable economic development (Guo et al. 2023 ). Boschma ( 2015 ) first proposed the concept of economic resilience, stating that economic resilience is the ability of an economic system to absorb shocks without catastrophic changes in its basic functional organization. Thereafter, the academic community investigated urban economic resilience from the perspectives of regional and geographic economics (Wang and Ge, 2023 ; Wang et al. 2024 ). Regional economics mainly explores how to cope with the decline in the regional economy, while geographic economics largely focuses on the study of spatial differences, correlations, and influencing factors of urban economic resilience (Zhang et al. 2023 ). In recent years, academic research on urban economic resilience has focused on three main aspects (Jesse, 2023 ; Cheng et al. 2022 ): measurement methods, influencing factors, and the impact of industries on urban economic resilience. First, from the perspective of measurement methods for urban economic resilience, scholars have measured the urban economic resilience index based on methods such as the core variable method and comprehensive indicator method from different disciplinary backgrounds (Qiang et al. 2020 ; Wang and Wang, 2021 ). However, due to the varying focuses, there is no consensus at present. Second, from the perspective of the influencing factors of urban economic resilience, scholars have considered that fiscal gaps, geographical location conditions, and resource endowments are key factors that constrain urban economic resilience (Gan and Chen, 2021 ; Cheng et al. 2022 ; Wang and Wang, 2021 ). The processes of urbanization, technological innovation, economic development, and industrial agglomeration are major factors driving urban economic resilience (Du et al. 2023 ; Gai and Yang, 2023 ). Third, from the perspective of the impact of industries on urban economic resilience, existing research has focused on two main aspects: (1) from a microscopic perspective, the impact of a single industry, such as the digital industry, financial industry, or manufacturing industry, on urban economic resilience has been examined; and (2) from a macroscopic perspective, the impacts of industrial structure upgrading, adjustment, transformation, and agglomeration on urban economic resilience have been explored. For example, Feng et al. ( 2023 ) empirically determined that industrial structure rationalization and upgrading are important ways for regional integration to affect urban economic resilience. However, the policy effects of regional integration on economic resilience vary over time, by region, and by urban structure. Zhang et al. ( 2023 ) noted that regional economic resilience is closely related to the state of the industrial structure, and there exists a spatiotemporal correlation in the evolution of the two systems. In the literature review process, we could conclude that the existing research on the relationship between industry and urban economic resilience has not yet reached a consensus, both at the micro- and macroscopic levels, thus providing a theoretical basis and new ideas for this study.

Regarding the relationship between globalization, industrial structure upgrading, and urban economic resilience, studies have mostly focused on the relationship between globalization and industrial structure upgrading, as well as the relationship between industrial structure upgrading and economic resilience (Carlos et al. 2022 ). In terms of the relationship between globalization and industrial structure upgrading, studies have suggested that globalization can promote industrial structure upgrading through the division of labor and cooperation in the industrial structure (Dunn, 2020 ) and that globalization can promote industrial structure upgrading through technological innovation and industrial transformation (Ngo, 2023 ). Globalization can improve factor allocation efficiency and drive industrial structure upgrading by influencing the direction and quantity of factor flow. In addition, scholars have noted that the development of globalization encompasses various stages, while its impact on the industrial structure is also cyclical, which can lead to instability in the impact of globalization on industrial structure rationalization and upgrading. In terms of the relationship between globalization and urban economic resilience, there are three specific viewpoints: globalization imposes a reducing effect on urban economic resilience (Tao et al. 2023 ), globalization exerts an expanding effect on urban economic resilience (Anis and Andreea, 2023 ), and the impact of globalization on urban economic resilience is dynamic (Martin et al. 2016 ). In terms of the relationship between industrial structure upgrading and urban economic resilience, industrial structure rationalization and upgrading can help to reduce the impact of international markets by improving the industrial configuration and quality level, providing greater development space for adaptive structural adjustment after impact occurrence and thus continuously enhancing urban economic resilience.

In the literature, scholars have empirically evaluated industrial structure upgrading and globalization as important factors affecting urban economic resilience based on econometric models, geographic models, and spatial econometric models (Maria, 2023 ; Cheng et al. 2023 ); however, few scholars have explored the relationships among these three factors. Within the context of globalization, the transformation of the new international division of labor model profoundly affects the process of industrial structure adjustment in various countries worldwide. However, existing research has overlooked the moderating effect of globalization on industrial structure upgrading and urban economic resilience. Moreover, the literature has mostly focused on analyzing the linear effect of industrial structure upgrading on urban economic resilience, and various conclusions have been obtained. This also reflects the complexity of the relationship between the two aspects, which suggests that they may not be characterized by a simple linear relationship, namely, there may be a nonlinear relationship. Compared with the literature, the marginal contribution of this study lies in coupling industrial structure upgrading, globalization, and urban economic resilience. It was preliminarily determined that the impact of industrial structure upgrading on urban economic resilience is nonlinear, and globalization was used as a threshold variable. Panel data from 267 prefecture-level cities and above in China from 2008 to 2021 were selected, and we empirically examined the nonlinear relationship between industrial structure upgrading and urban economic resilience and studied its stage characteristics. Moreover, we conducted robustness tests.

Theoretical foundations and research hypotheses

Industrial structure upgrading is important for exploring the economic resilience of Chinese cities. Especially at the current stage, China’s economy is shifting from high-speed growth to high-quality development, and industrial structure upgrading fulfills an important role in the resilience process of the urban economy in different regions and stages (Feng et al. 2023 ). The theory of industrial structure was first proposed by Fisher, who divided the overall industrial structure into the primary industry, secondary industry, and tertiary industry (Zheng et al. 2023 ; Li, 2024 ). William Petty established the theory of industrial structure upgrading, which refers to the process of industrial structure transformation from lower to higher stages, and constructed a theoretical analysis framework for industrial structure upgrading, including two aspects: the rationalization and advancement in the industrial structure (Feng et al. 2023 ). This theoretical framework laid the foundation for subsequent related research. With the deepening and development of theoretical research, the theory of industrial structure upgrading is the result of the joint action of the two forces of industrial structure upgrading and rationalization. There is widespread consensus among scholars at home and abroad, and this approach has been widely applied in fields such as social crises, regional environments, grassroots governance, and high-quality urban development. This has provided a new research approach for exploring the impact of industrial structure upgrading on urban economic resilience.

Since China joined the WTO, the Chinese economy has been further integrated into the world economy and become an important destination for FDI worldwide. Since then, FDI has become an important manifestation of globalization (Zhang et al. 2020 ). Globalization not only enhances the frequent exchange of resources such as technology, capital, and labor among countries but also promotes increasingly close economic development relationships among countries. Within the context of globalization, while countries achieve effective resource allocation in their industrial structures, they also move toward advanced and rational industrial structures, thereby promoting high-quality development of urban economic resilience through the release of structural dividends. In other words, industrial structure upgrading has become an important driving force for urban economic resilience within the context of globalization (Fig. 1 ).

In the figure, S denotes suppression, P denotes promotion, C denotes connection, and R denotes regular.

Industrial structure upgrading involves a process of rationalization. The singularity of the industrial structure is not conducive to urban entities overcoming internal and external market risks within an uncertain environmental context. At the primary stage of globalization, foreign direct investment can not only solve the problem of imbalanced and insufficient development of the regional industrial structure via the utilization of local resource endowments but also reduce production costs, provide stable and long-term global competitive advantages, and promote the intensive and large-scale development of local industries by achieving a reasonable division of labor and allocation of various industries in the regional industrial chain. The entry of labor- and capital-intensive industries into developing countries creates a large number of employment opportunities, promotes the flow of surplus labor from the agricultural sector to the nonagricultural sector, and enhances regional resistance to internal and external market risks. With the improvement in the globalization level and the increase in foreign investment, the formation of a diversified industrial system in cities is accelerating. Cities rely on industrial diversification, product richness, high-added value, and asynchronous industrial cycles to avoid drastic fluctuations in output and employment, thereby enhancing the ability of the urban economic system to overcome risks and adapt to shocks. Therefore, Hypothesis 1 was proposed.

Hypothesis 1: Industrial structure rationalization exerts a significant positive impact on urban economic resilience at different stages of globalization.

Another main feature of industrial structure upgrading is the upgrade process. At the early stages of globalization, foreign direct investment in factory construction, to some extent, dealt a heavy blow to traditional industries in developing countries, reducing the ability of urban economic entities to eliminate market risks. With the increase in globalization, on the one hand, FDI can promote the return of technology, labor, and capital and accelerate the replacement of high value-added industries and low value-added industries. On the other hand, foreign direct investment can not only promote the transition from traditional resource-intensive industries to resource-intensive industries but also attract more high-quality labor and technological resources, thereby increasing the resistance of urban economic systems to risks and the ability to adapt to impact factors. Therefore, Hypothesis 2 was proposed.

Hypothesis 2: There may be significant stage differences in the impact of industrial structure upgrading on urban economic resilience between different levels of globalization.

Research design and data sources

Model settings.

As mentioned earlier, industrial structure upgrading is accompanied by the reasonable flow and optimized allocation of production factors between industries and countries, which exerts a certain impact on urban economic resilience. Globalization also plays an important role in this process, but confirmation of the existence of threshold effects still requires further empirical testing. Therefore, in this study, globalization was adopted as a threshold variable, urban economic resilience was used as the dependent variable, and industrial structure upgrading and rationalization were employed as the core explanatory variables. Then, the following threshold panel model was constructed (Dou and Gao, 2023 ; Zheng et al. 2023 ):

where UER it denotes the urban economic resilience of city i during the t-th period; GL it denotes the globalization of city i during the t-th period as a threshold variable, with r 1 , r 2 …, r n representing n threshold values; RIS it and AIS it denote industrial structure rationalization and advancement, respectively; α 1 , α 2 , and α n, and β 1 , β 2 , and β n denote the regression coefficients for different threshold intervals; X it is a series of control variables; and θ and k are the regression coefficients of the control variables. Note that the only difference between Models (1) and (2) is the core explanatory variables (without considering differences in the parameter values), which are industrial structure upgrading and industrial structure rationalization, respectively. For the sake of brevity, Models (1) and (2) are distinguished by their core explanatory variables and are thus referred to as the RIS and AIS models, respectively.

Variable selection

Explained variable, calculation model for urban economic resilience.

The regional economic foundation often determines the lower limit of a region’s ability to withstand shocks, which, to a certain extent, affects its resilience and recovery level (Wang et al. 2021 ). Notably, urban economic resilience is not only related to the total economic output but also closely related to the economic structure. The higher the total economic output is, the higher the urban economic resilience when facing risks. The GDP is an appropriate indicator of the total economic output of a city, while a reasonable economic structure is also a key factor in ensuring healthy and dynamic growth of the urban economy. In contrast to the former, the economic structure more strongly reflects changes in economic growth rates. The main research methods for measuring urban economic resilience are the core variable method and the comprehensive indicator method, but a consensus has not yet been reached. Considering the representativeness and continuity of the core variable method, this study refers to existing research, and the output method was used to measure urban economic resilience. This indicator can directly reflect the degree of change in the urban economy in the face of pressure and shocks (Feng et al. 2023 ). Compared with previous studies, this approach avoids the subjectivity of using a comprehensive indicator system, but it neglects the relationships and dependencies between factors and does not fully reflect the actual situation. The specific calculation method is as follows:

where GDP denotes the standardized value of the GDP and ∆GDPV is the standardized value of the absolute change in the GDP growth rate in adjacent years. As the product of multiple standardized values is very small, to better visualize the differences in urban economic resilience, the regression coefficient is multiplied by 100.

Core explanatory variables

According to existing research, industrial structure rationalization and advancement were used as alternative indicators to measure urban industrial structure upgrading (Gan and Chen, 2021 ; Yin et al. 2023 ). As expressed in the equation below, rationalization of the industrial structure (RIS) mainly reflects the coupling relationship between the input and output, where i denotes the primary industry, secondary industry, or tertiary industry, Y represents industrial economic output, L represents labor input, and I represents industrial sector (i = 1,2,3). Upgrading of the industrial structure (AIS) largely reflects the proportion of the service industry, measured as the ratio of the added value of the tertiary industry to that of the secondary industry.

Threshold variables

The threshold variable selected in this study is globalization (GL). After China’s accession to the WTO, China gained a large amount of foreign direct investment due to its resource endowment. With the increasingly close relationship between China’s economic development and the global economy, FDI transformed globalization into a localization force through location selection, which, to a certain extent, promoted the development of the regional economy. However, the greater the degree of closeness to the global economy is, the greater the occurrence probability of unpredictable risks, thereby exacerbating the vulnerability of regional economic resilience. Therefore, the per capita foreign direct investment amount was selected as a characterization indicator to measure globalization.

Control variables

(1) Industrial agglomeration level of industrial enterprises. We selected the total number of industrial enterprises/urban construction land area as an indicator to measure the level of industrial enterprise agglomeration (Wang et al. 2021 ). The agglomeration of industrial enterprises can accelerate the convergence of regional enterprises, goods, services, and highly skilled labor. The agglomeration of a large number of intermediate inputs, high-level services, and human capital imposes a significant positive effect on urban economic resilience. However, the excessive agglomeration of industrial enterprises could also lead to significant negative externalities with respect to factors such as infrastructure, residents’ health, and the ecological environment, reducing the responsiveness of urban economic systems to external shocks. (2) Local financial gap. We selected the indicator of (expenditure within the local fiscal budget—revenue within the local fiscal budget)/revenue within the local fiscal budget to measure fiscal gaps (Xiong et al. 2023 ). The level of local finance fulfills an important role in achieving the optimal allocation of regional resources. Research has shown that the smaller the local fiscal gap, the more capable local governments are of achieving a balance and structural optimization between the total social demand and supply when facing external shocks. In contrast, the greater the local fiscal gap is, the lower the ability of local governments to self-adjust and repair in the face of external shocks. (3) Technology investment level Footnote 1 . We selected scientific and technological investment/local general public budget expenditure as a measurement indicator (Zhang et al. 2020 ). Improving the technology investment level can, to a certain extent, accelerate the elimination of traditional industries and the growth of emerging industries in a given region, enhance the competitive advantage of the entire industry chain in key fields, and overcome the monopoly of core technologies in certain foreign fields, thus increasing urban economic resilience. (4) Population density. We selected the total number of permanent residents in cities within the province/the area of provincial jurisdiction as a measurement indicator (Tan et al. 2020 ). The population density is a key factor affecting urban economic resilience and varies across different regions. The regional agglomeration effect generated by the population density can cause various resources to gather in cities, while population aggregation can cause various high-quality resources to accumulate in cities, generating a positive agglomeration effect, which facilitates the construction of regional spatial governance systems, high-quality and intelligent public services, and infrastructure systems and has a certain significance in building urban economic resilience. When there is a turning point in the population density, the phenomenon of excessive development and utilization of regional resources may occur, requiring local governments to consume more financial and material resources to solve problems such as environmental damage, severe resource depletion, and high carrying pressure caused by population agglomeration. This exerts a significant inhibitory effect on urban economic resilience, so the total number of permanent residents in cities within the province/the area of provincial jurisdiction was chosen as a measurement indicator. (5) Infrastructure level. We selected the coverage of public transportation routes/total population as a measurement indicator (Zhang, 2022 ). Infrastructure, including transportation and communication facilities, provides the basic guarantee for urban economic activities. The convenience of infrastructure is directly related to the development of local economic resilience. Research has shown that improving infrastructure is not only conducive to high-quality development of the local economy but also enhances the ability of urban economic systems to overcome risks and adapt to shocks. (6) The economic development level was measured by the total GDP/total population ratio in this study (Deng et al. 2023 ). The economic strength and development level of a city significantly impact its resilience: the stronger the economy is, the more reasonable the structure, and the higher the innovation ability is, the greater the resilience level of a city. Research has indicated that the urban economic development level is related to the ability to withstand macroeconomic and financial risks.

Data sources and descriptive statistics

We selected panel data for 267 prefecture-level cities and above in China (Excluding Hong Kong, Macao, Taiwan, Xinjiang, and Xizang) from 2008 to 2021 for empirical analysis. The panel data were obtained from the Statistical Yearbook of Urban Construction in China and the Statistical Yearbook of Chinese Cities from 2008 to 2021. Descriptive statistics of all variables are listed in Table 1 . The differences between the minimum and maximum values for measuring the urban economic resilience index were significant, indicating a significant difference in urban economic resilience among the 267 prefecture-level cities and above in China between 2008 and 2022. The maximum value of urban economic resilience was observed for Shanghai in 2019, with an urban economic resilience of 20.033, while the minimum value was obtained for Dingxi city in 2018, with an urban economic resilience of 0.000. The minimum and maximum values of the industrial structure rationalization indicator were 0.581 and 29.114, respectively, while the minimum and maximum values of the industrial structure upgrading indicator were 0.139 and 5.350, respectively. This reflects the considerable differences in industrial structure rationalization and upgrading among the 267 prefecture-level cities and above between 2008 and 2022. Similarly, there was a significant difference between the minimum value (−2.028) and the maximum value (9.970) of the globalization indicator, indicating a significant difference in globalization among the 267 prefecture-level cities and above between 2008 and 2022.

Empirical testing

Hausman test.

Before conducting the Hausman test, we first applied the Durbin–Wu–Hausman (DWH) test to assess for endogeneity issues in the benchmark panel model (Zheng et al. 2023 ). According to the test results, the P value of the DWH test was 0.000, indicating that the null hypothesis of “all explanatory variables are exogenous” could not be rejected at a significance level of 1%. Therefore, the model did not exhibit endogeneity issues. Finally, we conducted a Hausman test of the relationship between industrial structure rationalization and upgrading and urban economic resilience. The results indicated that the P value of both models (1 and 2) in the Hausman test was 0.0000, indicating that the original hypothesis of random effects could be rejected at the 1% significance level and that the alternative hypothesis of fixed effects could be accepted. Therefore, we utilized the panel fixed effects model to estimate Models 1 and 2.

Assessing the threshold effect of industrial structure upgrading

Before establishing a specific threshold effect model, two important tests were conducted: one involved testing for the existence of threshold effects, with the aim of exploring whether the parameter spaces within different threshold intervals divided by the threshold values significantly differ; the other was using the bootstrap method for consistency testing, with the aim of determining whether the estimated threshold value is consistent with the actual value. The former is generally evaluated by the F statistic, while the latter is assessed by the likelihood ratio (LR) statistic.

The test results for the industrial structure rationalization and upgrading models with globalization as the threshold variable are provided in Table 2 . First, a single threshold test was conducted, and the corresponding F values were 193.71 and 514.98, respectively, while the P values were 0.0033 and 0.0000, indicating that there exists a threshold effect in both the rationalized and advanced industrial structure models. Second, the double threshold effect existence test was performed, with corresponding F values of 55.85 and 90.97, respectively, and P values of 0.0700 and 0.0133, respectively, which were significant at the 10 and 5% levels. The original hypothesis of the existence of a single threshold could be rejected, and it could be considered that both models contain a double threshold effect. Finally, a triple threshold test was conducted, and the P values at this time indicated that the original hypothesis of the existence of double thresholds could not be rejected. Therefore, in this study, a dual threshold effect model was chosen for estimation. The model can be formulated as follows:

We used the estimation method of minimizing the sum of squares of residuals to determine the specific threshold values, and the results are provided in Table 3 . The first and second threshold values for the model variables of industrial structure rationalization and upgrading (GL) were 6.8724 and 6.5514, respectively, and 7.5034 and 6.8724, respectively. Table 3 also provides the confidence intervals for each threshold value at a 95% confidence level.

Next, we performed a consistency test between the estimated threshold values and the actual values. Based on the estimation results listed in Table 3 , a likelihood ratio function graph was created. The horizontal axis in Fig. 2 represents the threshold value for globalization, the vertical axis represents the likelihood ratio function value, and the dashed line represents the critical value at the 95% confidence level. Here, the upper half represents the confidence interval for the first threshold of globalization, while the lower half represents the confidence interval for the second threshold of globalization. According to Hansen’s likelihood ratio test model, for LR(γ) > C(θ), the original hypothesis can be rejected. For θ = 5%, the critical value of the LR statistic is 7.35. According to Fig. 1 , the threshold values of the LR statistic corresponding to industrial structure rationalization and globalization of the advanced models were significantly lower than the critical values, so the above threshold estimates could be considered true and effective.

Dual threshold values and likelihood ratio function of GL in the RIS and AIS model.

Estimation results of the threshold panel model

Regression estimates of the impact threshold of industrial structure upgrading on urban economic resilience are provided in Table 4 . Among them, the regression results for basic Models (I) and (V) indicated that the relationship between industrial structure upgrading and globalization could be divided into three intervals by the threshold variable of globalization, and there were significant differences between the different intervals. At a lower globalization level, the impact of industrial structure rationalization on globalization was significantly positive at the 1% level, while the impact of industrial structure upgrading on globalization was not significantly negative. On the one hand, this suggests that at a lower globalization level, industrial structure rationalization could significantly improve urban economic resilience. When faced with external shocks, the economic resilience of cities significantly differs. Cities with a reasonable industrial layout exhibit higher economic resilience, namely, cities with a more reasonable industrial structure can quickly adjust their industrial structure, thereby obtaining more persistent and robust economic resilience. On the other hand, this result indicates that the level of industrial upgrading in China is still low, and products are at a disadvantage in the global market competition process. At this time, globalization is still dominated by the negative siphon effect.

Notably, at the early stages of globalization, industrial structure rationalization is the foundation for the economy to overcome external shocks, while later, the same process is the source of sustained economic resilience. When globalization crosses the first threshold and reaches the intermediate stage, the regression coefficient between industrial structure rationalization and urban economic resilience remains significantly positive, The above research confirms the validity of Hypothesis 1. This may occur because, with the acceleration of globalization, industrial structure rationalization eliminates market barriers between regions and internationally through the rational division of labor and the allocation of various industries within the industrial chain, achieving effective resource allocation, releasing structural dividends, and providing stable and long-term global competitive advantages, thus enhancing the ability of urban economic systems to mitigate risks and adapt to shocks. The impact of an advanced industrial structure on urban economic resilience within this interval shifted from negative to positive, indicating that with the improvement in the level of the advanced industrial structure, the return of technology, talent, and capital is promoted, accelerating the replacement of high value-added industries and low value-added industries, improving the demand income elasticity of high value-added products, reducing the operational risks of urban entities, and increasing the ability of urban economic entities to withstand market risks. When globalization crosses the second threshold and reaches a high level, the impact coefficients of industrial structure rationalization and upgrading on globalization are significantly positive at the 1% significance level, with regression coefficients of 0.037 and 0.780, respectively. The main reason is that with the rapid development of globalization, the industrial structure layout becomes more reasonable, which is conducive to the formation of a diversified industrial system in cities. Cities rely on industrial diversification, product richness, high-added value, and asynchronous industrial cycles to prevent sharp fluctuations in output and employment, thereby improving the ability of the urban economic system to overcome risks and adapt to shocks. In contrast, the deepening of globalization promotes the transition from traditional resource-intensive industries to resource-intensive industries, thus vigorously enhancing the level of industrial intensification and, in return, the ability of the urban economic system to withstand risks and adapt to shocks. The above research confirms the validity of Hypothesis 2.

After benchmark regression, to ensure the reliability of the conclusions obtained, we conducted two robustness tests. First, considering that extreme values in the sample may impact the results, Beijing (which attained the highest urban economic resilience value in 2019) and Dingxi (which attained the lowest urban economic resilience value in 2018) were excluded, and the model was again regressed to obtain robustness test results (II and VI in Table 4 ). Second, robustness tests were performed by controlling for time and individual interaction terms, and the results of robustness tests III and VII are listed in Table 4 . Finally, by including control variables for robustness testing, additional robustness test results (IV) and (VIII) were obtained, as provided in Table 4 . The three additional control variables were the market size, financial development level, and cultural soft power (Guo et al. 2023 ; Zhang and Yao., 2023 ), where the market size can be measured by the proportion of the total retail sales of consumer goods to the GDP. The financial development level can be measured by the ratio of the bank loan scale to the GDP. Cultural soft power can be measured by the logarithmic value of book collection. Compared with the estimation results of the basic model, the core explanatory variables in the robustness test results—namely, industrial structure rationalization and industrial structure upgrading—basically exhibited regression coefficients of the same sign and significance within the various threshold ranges of globalization (represented by GL L , GL M , and GL H ). For example, robustness test result II was relatively close to that of basic Model I, while robustness test result VI was relatively close to that of basic Model V. The sign and significance of the regression coefficients for the core explanatory variable, i.e., industrial structure rationalization, within the various threshold intervals of globalization were highly similar. However, after removing the extreme values in the sample data, the regression coefficient and significance of industrial structure upgrading exhibited significant changes within the various threshold ranges of globalization, with significance decreasing from the previous 1% level to the 5% level. However, it should be noted that although the removal of extreme values generated a certain impact in this study, the existence of individual extreme values did not affect the basic conclusions of the model. For example, robustness test result III was related to basic Model I, while robustness test result VII was related to basic Model V. The regression coefficients for the core explanatory variables of industrial structure rationalization and industrial structure upgrading did not show significant changes within the various threshold ranges of globalization, as Model VII incorporated time and individual interaction terms, and the main difference between the two models was only the fact that within the third threshold range (GL H ) of globalization, the significance of the regression coefficient of Model VI slightly decreased. However, it was still significantly positive at the 5% level, and the conclusion remained significant. This indicates that, regardless of whether time and individual interaction terms were controlled, the conclusions of model analysis remained consistent. The results of robustness tests (IV) and (VIII) showed that the three additional control variables slightly but significantly affected the basic regression results. Therefore, the threshold regression model constructed based on panel data from 267 prefecture-level and above cities in China for assessing the impact of industrial structure upgrading on urban economic resilience was robust and reliable.

Analysis of regional heterogeneity

To eliminate the interference of regional heterogeneity factors and verify the moderating effect of globalization on industrial structure upgrading and urban economic resilience, we divided the 267 prefecture-level and above cities into four major regions—eastern, central, northeastern, and western regions—and conducted threshold effect testing and threshold model estimation. The specific results are listed in Table 5 . The industrial structure rationalization and upgrading models in both the eastern and central regions passed the double threshold test. The threshold values for globalization of the two models in the eastern region were 6.54 and 6.88, respectively, and 6.44 and 6.88, respectively, while the threshold values for globalization of the two models in the central region were 6.63 and 6.97, respectively, and 6.97 and 7.23, respectively. The industrial structure rationalization and upgrading models in the western region and the industrial structure upgrading model in the northeastern region did not pass the double threshold test, while the industrial structure rationalization model in the northeastern region exhibited only a single threshold effect.

The threshold panel regression results for the eastern, central, western, and northeastern regions are listed in Table 6 , which indicates certain differences between the regression results for each region and the national level.

In the eastern region, when globalization fell below the first threshold, the regression coefficients of industrial structure rationalization and upgrading on globalization were 0.056 and 0.275, respectively, and both passed the 1% significance test. When globalization crossed the first threshold but remained lower than the second threshold, the regression coefficients of both were significant. After globalization crossed the second threshold, the regression coefficients of the two variables were 0.109 and 1.546, respectively, and both passed the 1% significance test. These results indicated that with the continuous improvement in globalization, the relationship between industrial structure rationalization and upgrading and urban economic resilience in eastern China exhibits an inverted U-shaped trend. The reason for this may be that the eastern region has become the preferred region for accepting foreign investment due to its inherent resource endowment and location advantages. On the one hand, with increasing foreign investment, cities in the eastern region have launched a prelude to traditional industrial transformation. The optimization and improvement in industrial and product structures have, to a certain extent, increased the overall development level of regional industries and international competitiveness while also driving the improvement in urban economic resilience. On the other hand, with the limited urban carrying capacity in the eastern region, foreign direct investment is expected to be gradually transferred to the central and western regions, resulting in the phenomenon of urban economic resilience first increasing and then decreasing.

The threshold effect of industrial structure upgrading and rationalization on urban economic resilience in the central region was similar to that in the eastern region, also exhibiting an inverted U-shaped trend. On the one hand, in recent years, the central region has become an important position for China’s development of strategic emerging industries, committed to solving problems such as industrial structure convergence and chain fragmentation, and relies on diversified industrial structures (strategic emerging industries) to accelerate the rational allocation of industrial structures and regions to enhance the synergy between the secondary industry (manufacturing) and tertiary industry (productive services), improve the correlation and cohesion between industries within the region, promote the spillover of knowledge and innovation between departments, accelerate the formation of regional industrial economies of scale, and achieve long-term and robust urban economic resilience. On the other hand, the central region has adjusted its industrial structure through specific plans such as urban renewal, accelerating the circulation of key technological elements between different industries, improving the efficiency of regional resource allocation, increasing the size of the regional economy, and taking the lead in entering the post-industrialization stage, which, to a certain extent, serves to improve urban economic resilience.

Industrial structure rationalization exerted a single threshold effect on urban economic resilience in the northeastern region. When globalization occurred below the threshold value of 6.63, the regression coefficient of industrial structure rationalization was −0.003, which is not significant. When globalization exceeded the threshold value of 6.63, the impact of industrial structure rationalization on urban economic resilience remained inhibitory, with a regression coefficient of −0.021. The reason for this may be that, as an old industrial base in China, the northeastern region exhibits a heavy industrial structure and relatively slow development, which is less favored by foreign direct investment. During the sample period, with the promotion of the Northeast Revitalization strategy and the implementation of a series of supporting measures, the evolution of the industrial structure also showed a continuous trend of improvement. However, due to the failure to achieve the ideal goal of upgrading traditional industries, they often do not exhibit market competitiveness, so the promotion effect on urban economic resilience is relatively limited.

The western region differs from other regions and did not show a threshold effect. We believe that the reason for this may be that the Western region implemented only the Western Development strategy during the early 21st century. Transportation, communication, and other infrastructure in the western region are relatively outdated, and the opportunities for foreign direct investment are lower than those in the eastern, central, and northeastern regions. As a result, the industrial spillover effect and positive externality effect in the western region are not significant. Therefore, even at the advanced stage of globalization, industrial structure rationalization and upgrading in the Western region do not significantly impact urban economic resilience.

Conclusions and policy recommendations

In this study, 267 prefecture-level cities and above in China were adopted as the research object, and globalization was considered a threshold variable to couple industrial structure upgrading globalization and urban economic resilience. We empirically assessed the nonlinear impact of industrial structure upgrading on urban economic resilience. The results indicated that, from a national perspective, the impact of industrial structure rationalization and upgrading on urban economic resilience exhibited a nonlinear relationship, and both showed a double threshold effect. At the initial stage of globalization, industrial structure rationalization promotes urban economic resilience, while industrial structure upgrading negatively impacts urban economic resilience. After globalization crosses the first threshold and enters the intermediate stage, industrial structure rationalization and upgrading exert a positive promoting effect on urban economic resilience, and industrial structure upgrading shifts from a negative to a positive promoting effect, which is significant. When urban economic resilience crosses the second threshold and enters the advanced stage, industrial structure rationalization and upgrading exert a significant positive effect on urban economic resilience. By removing extreme values from the sample, controlling for time and individual interaction terms, including control variables, and then modeling again, it was found that the above conclusion still holds and passes the robustness test. To empirically analyze regional heterogeneity, we divided the sample into eastern, central, northeastern, and western regions. We found that industrial structure upgrading still imposes a dual threshold effect on urban economic resilience in the eastern and central regions with globalization as the threshold variable, but the performance varied among the different regions. The threshold effect of industrial structure upgrading (including rationalization and upgrading) in the eastern and northeastern regions on urban economic resilience was characterized by an inverted U-shaped trend. In particular, the northeastern region still occurred on the left side of the inverted U-shaped curve, with only a single threshold effect. No threshold effect was observed for the western region.

Based on the research findings presented here, the following policy recommendations are proposed: firstly, we should continue to accelerate industrial transformation and upgrading, thus promoting high-quality development of urban economic resilience. We should actively promote regional industrial structure upgrading and, through the development of high value-added new economy sectors, we should embark on accelerated internal economic structure optimization, thereby enhancing urban economic resilience. Consequently, to improve the rationalization of the regional industrial structure, reasonable leading industries and strategic industries should be selected for driving economic development to support the development of leading industries, extend the industrial chain of products and enhance their added value, achieve differentiated competition and gradient transfer of industries, construct a circular economic development model, optimize the layout and management of the entire industrial chain, and establish a reasonable distribution of upstream, midstream, and downstream resources in the industry. Secondly, we should increase openness and innovation efforts to promote high-quality development through high-level openness. We should make good use of the national independent innovation demonstration zone platform, highlight open innovation, enhance innovation capabilities, promote independent innovation through open innovation, and continuously improve our competitiveness in opening up to the outside world. This would allow for increased efforts to “bring in and go out,” i.e., cultivating more market entities that are open to the outside world. We should continuously optimize the export structure, encourage and support local enterprises to “go global”, attract high-quality foreign investment, and promote the “optimal entry and exit” of opening up to the outside world. Thirdly, regional governments should implement differentiated measures based on the actual situation in their respective regions to avoid one-size-fits-all measures. The level of globalization in the eastern and central regions is already high, and it is necessary to focus on accelerating the development of high-tech industries, especially the updating and development of industries such as those in information, biology, new materials, aerospace, and ocean fields. The industrial structure and globalization levels in the northeastern and western regions are relatively low. As such, it is necessary to accelerate the replacement of new and old industries, vigorously promote the integration of informatization and industrialization, and thereby improve urban economic resilience. In the western region, it is necessary to continue industrial transfer from the eastern and central regions, optimize the regional industrial structure layout, create high-quality service platforms, focus on cultivating leading enterprises, fully leverage the siphon effect to drive industrial agglomeration, and create a bridgehead for opening up to the west, leveraging the advantages of large ports, channels, logistics, hubs, and trade. In Northeast China, efforts should be made to consolidate inventories, achieve incremental expansion, extend the industrial chain, and increase the added value. The digital, networked, and intelligent transformation of traditional manufacturing industries should be accelerated, promoting the extension of the industrial chain both upstream and downstream and finally creating a relatively complete industrial chain and cluster.

Further discussion

Industrial structure upgrading is an effective way to improve the urban economy, while globalization is an important driving force for regulating the correlation between urban industrial structure upgrading and urban economic resilience (Cheng et al. 2022 ). Although scholars have explored the coupling relationship between industrial structure upgrading and economic resilience, there is still insufficient research on the relationship between industrial structure upgrading, globalization, and urban economic resilience using globalization as a threshold. Compared with existing research, the similarity with previous studies lies in the fact that both have validated the nonlinear relationship between industrial structure upgrading and urban economic resilience. The difference from previous research lies in that, from a research perspective, we have taken the lead in placing industrial structure upgrading, globalization, and urban economic resilience within the same framework, and systematically elaborated on the relationship between the three, Fully considering the impact of industrial structure upgrading on urban economic resilience under the regulation of globalization is a beneficial supplement to existing theoretical research. From the perspective of research content, further subdividing the upgrading of industrial structure, using globalization as a threshold variable, exploring the threshold effect of industrial structure upgrading and industrial structure rationalization on urban economic elasticity, revealing the nonlinear relationship and stage characteristics between industrial structure upgrading and urban economic elasticity, is a strong challenge to the traditional research conclusion that there is a coupling relationship between the two. From the empirical results, although it cannot be said that the framework constructed in this article solves the “black box” problem of urban economic elasticity, it can indeed indicate that there is a linear relationship between the advancement and rationalization of industrial structure and urban economic elasticity, and it has a significant dual threshold effect and regional heterogeneity.

For developing countries, globalization is a double-edged sword. On the one hand, globalization can accelerate the development of international trade and close the global wealth gap (Xuan et al. 2023 ). On the other hand, globalization can create a more stimulating environment of international market competition (Shang, 2022 ). Research has also shown that some developing countries have deviated from their actual problems and needs, are blindly integrated into the global development system, and are constrained by factors such as industrial technology, product quality, and talent, becoming the weakest link in the global industrial chain (Carlos et al. 2022 ). The invasion of Western cross-border capital and industries has undoubtedly enormously impacted the economic and social development of developing countries, leading to issues such as the fragility of urban economic resilience and limited defense capabilities. Since China’s integration into the global economic system, it has adhered to a mutually beneficial and win-win strategy of opening up to the outside world. While fully leveraging its own endowments, it is adept at innovative integration of foreign and local technologies. Based on the advanced and rational industrial structure, it has increased investment in intelligent manufacturing and digital transformation, achieving the transformation from traditional manufacturing to intelligent manufacturing in China. At the same time, it enriches the content of the urban industrial system, improves industrial production efficiency and quality, expands international market competitiveness advantages, and promotes the formation of urban economic resilience. In addition, in recent years, China has also actively promoted the the Belt and Road Initiative, aiming to optimize the industrial structure system of countries along the Belt and Road through infrastructure construction, trade and investment, financial cooperation, and other means, while increasing the economic closeness of countries along the Belt and Road, so as to promote high-quality economic development of cities along the Belt and Road. This is also the main reason why China’s economy is still able to recover and maintain rapid growth under the impact of the financial crisis in 2008 and the COVID-19 epidemic. Although China still has a long way to go in enhancing urban economic resilience, it can undoubtedly provide important experiences for other developing countries to achieve the same goals.

It should be noted that this study has certain limitations. Firstly, regarding the measurement of urban economic resilience indicators, in existing research, the core variable method and the comprehensive indicator method have mostly been used to measure urban economic resilience, but no consensus has been reached thus far. The measurement indicator method selected in this study focused more notably on the representativeness and continuity of the core variable method, while the output method was used to measure urban economic resilience. Should periodicity be considered? This question may yield a direction for future research. Secondly, there may be certain shortcomings in the measurement of indicators for industrial structure advancement and rationalization, which cannot restore the comprehensiveness of industrial structure upgrading. Can industrial structure upgrading be further classified? This issue must be improved upon in the future. Finally, there were certain missing values for the research area considered in this article (especially in the western region of China). If further consideration is given to all prefecture-level cities and above, it cannot be ruled out that this may impact the conclusions of this article, which may also provide a direction for future research.

Data availability

The original data for this article is included in the supplementary material of the article, and further inquiries can be directed to the corresponding author.

It should be noted that our measurement of the technology investment level is based on the government’s perspective, confirming whether the technology investment level of the government impacts urban economic resilience. Therefore, private investment and foreign direct investment are not included in the measurement of the technology investment level.

Aida CS, Morten R, Oliver R (2016) Economic resilience: what role for policies? J Int Commer Econ Policy 7(2):1650009–1650009

Article   Google Scholar  

Andrew PB, Steven GT, Joana, F (2020) Disadvantage and economic viability: characterising vulnerabilities and resilience in upland farming systems. Land Use Policy, 96(C). https://doi.org/10.1016/j.landusepol.2020.104698

Anis B, Andreea CB (2023) Globalization beyond business sustainability, energy and the economy of the future. Proc Int Conf Bus Excell 17(1):1569–1583

Ben ZN, Ifergane T (2022) Firing restrictions and economic resilience: protect and survive? Rev Economic Dyn 43:93–124

Betts T, Buzzanell PM (2022) Enacting economic resilience: a synthesis of economic and communication frameworks. J Risk Financ Manag. 15(4):178–178

Boschma R (2015) The evolution of path dependence. Edited by Lars Magnusson and Jan Ottosson. Economic Geogr. 87(3):363–364

Carlos M, Antonios K, Carsten, H (2022) International trade resilience and the Covid-19 pandemic. J Bus Res. https://doi.org/10.1177/00081256211069420

Cheng T, Zhao, YH, Zhao CJ (2022) Exploring the spatio-temporal evolution of economic resilience in Chinese cities during the COVID-19 crisis. Sustain Cities Soc. https://doi.org/10.1016/J.SCS.2022.103997

Cheng, T, Zhao, YH, Song, YZ, Ma, L, Zhang, ZH, Luo, P, Gao, P, Zhang, MN, Zhao, CJ (2023) Towards resilience effectiveness: assessing its patterns and determinants to identify optimal geographic zones. J Clean Prod. https://doi.org/10.1016/J.JCLEPRO.2023.139596

Dario D, Weterings ABR (2015) The resilience of regional labour markets to economic shocks: exploring the role of interactions among firms and workers. J. Economic Geogr. 15(4):723–742

Deng Y, Jiang WY, Wang ZY (2023) Economic resilience assessment and policy interaction of coal resource oriented cities for the low carbon economy based on AI. Resour Policy. https://doi.org/10.1016/J.RESOURPOL.2023.103522

Dou QQ, Gao, XW (2023) How does the digital transformation of corporates affect green technology innovation? An empirical study from the perspective of asymmetric effects and structural breakpoints. J Clean Prod. https://doi.org/10.1016/J.JCLEPRO.2023.139245

Drobniak A (2017) Economic resilience and hybridization of development – A case of the Central European Regions. Region Stat. 7(1):43–62. https://doi.org/10.15196/rs07103

Du CY (2023) Research on evaluation of Thailand’s economic resilience and coupling coordination. Financ Eng Risk Manag. https://doi.org/10.23977/FERM.2023.060802

Du YN, Wang QX, Zhou JP (2023) How does digital inclusive finance affect economic resilience: Evidence from 285 cities in China. Int Rev Financ Anal. https://doi.org/10.1016/J.IRFA.2023.102709

Dunn HS (2020) Creative resilience and globalization from within: evolving constructs for analysing culture, innovation, and enterprise in the global south. Ann Int Commun Assoc. https://doi.org/10.1177/00081256211069420

Erika Q, Mangirdas M (2020) The economic resilience of the Austrian agriculture since the EU accession. J Risk Financ Manag 13(10):236–236

Fan WR, Wang ZX, Lv WQ (2023) Effects of the resilience of energy systems on economic sustainability. Sustain Prod Consum 41:379–390

Feng Y, Li CC, Peng DY (2023) Does regional integration improve economic resilience? Evidence from urban agglomerations in China. Sustain Cities Soc. https://doi.org/10.1016/J.SCS.2022.104273

Fingleton B, Palombi S (2013) Spatial panel data estimation, counterfactual predictions, and local economic resilience among British towns in the Victorian era. Region Sci Urban Econ 43(4):649–660

Gai M, Yang QF (2023) Synergistic study of the green efficiency and economic resilience of agriculture from a sustainable development perspective: evidence from Northeast China. Environ Sci Pollut Res Int 30(31):77568–77592

Article   PubMed   Google Scholar  

Gajewski P (2022) Regional resilience to the Covid-19 shock in Polish regions: how is it different from resilience to the 2008 Global Financial Crisis? Region Stud Region Sci 9(1):672–684

Gan QH, Chen SM (2021) The impact of market integration on regional economic growth from the perspective of upgrading industrial structure. Ind Econ Res. https://doi.org/10.13269/j.cnki.ier.2021.05.004

Gereffi G, Pananond P, Pedersen T (2022) Resilience decoded: the role of firms, global value chains, and the state in COVID-19 medical supplies. Calif Manag Rev 64(2):46–70

Guo AJ, Zhang XY, Li Q (2023) Network infrastructure construction and urban economic resilience: a quasi natural experiment based on the demonstration city construction of “Broadband China”. Financ Trad Res. https://doi.org/10.19337/j.cnki.34-1093/f.2023.05.003

Hao YY (2023) The dynamic relationship between trade openness, foreign direct investment, capital formation, and industrial economic growth in China: new evidence from ARDL bounds testing approach. Humanit Soc Sci Commun. https://doi.org/10.1057/S41599-023-01660-8

Hui J, Tan QM (2023) Dynamic evaluation of regional economic resilience under major public emergencies: based on an improved dynamic evaluation model of grey incidence projection-fuzzy matter element. Wirel Netw 29(7):3223–3238

Hynes W, Trump BD, Kirman A, Haldane A, Linkov I (2022) Systemic resilience in economics. Nat Phys 18(4):381–384

Article   CAS   Google Scholar  

Jayasinghe N, Fernando S, Haigh R, Amaratunga D, Fernando N, Vithanage C, Ratnayake, RC (2022) Economic resilience in an era of ‘systemic risk’: Insights from four key economic sectors in Sri Lanka. Prog. Disaster Sci. https://doi.org/10.1016/J.PDISAS.2022.100231

Jesse S (2023) Arcidiacono Alessia,Torrisi Gianpiero,Arku Robert Nutifafa. Regional economic resilience: a scoping review. Prog Hum Geogr 47(4):500–532

Johnson LC, Mundell M (2023) Regional resilience and an interventionist state: the case of Geelong, Victoria, 1990–2020. Urban Policy Res 41(3):247–261

Knoke T, Paul C, Gosling E, Jarisch I, Mohr J, Seidl R (2022) Assessing the economic resilience of different management systems to severe forest disturbance. Environ Resour Econ 84(2):343–381

Lee SM, Wang S (2023) Impacts of political fragmentation on inclusive economic resilience: examining American metropolitan areas after the Great Recession. Urban Stud 60(1):26–45

Lemke LK, whang G, Sakdapolrak P, Trippl M (2023) Unresolved issues in regional economic resilience: Conceptual ways forward. Prog Hum Geogr 47(5):699–717

Li XM, Zhou SW, Zhao YF, Wan GX (2024) A CRITIC-TOPSIS and optimized nonlinear grey prediction model: a comparative convergence analysis of marine economic resilience. Expert Syst Appl. https://doi.org/10.1016/J.ESWA.2023.121356

Li J (2024) Impact of green finance on industrial structure upgrading: implications for environmental sustainability in Chinese regions. Environ Sci Pollut Res Int 31(9):13063–13074

Maria T (2023) Social capital and characteristics of economic dualism as determinants of regional resilience. Eur Plan Stud 31(12):2568–2589

Mai X, Zhan CQ, Chan Roger CK (2021) The nexus between (re)production of space and economic resilience: an analysis of Chinese cities. Habitat Int. https://doi.org/10.1016/J.HABITATINT.2021.102326

Martin R, Sunley P (2014) On the notion of regional economic resilience: conceptualization and explanation. J Econ Geogr 15(1):1–42

Martin R, Sunley P, Gardiner B (2016) How regions react to recessions:Resilience and the role of economic structure. Region Stud 50(4):561–585

Ngo HT (2023) Causal relationship between globalization, economic growth and CO2 emissions in Vietnam using wavelet analysis. Energy Environ 34(7):2386–2412

Paolo DC (2017) Testing and explaining economic resilience with an application to. Pap Region Sci 96(1):93–113

Papaioannou SK (2023) ICT and economic resilience: evidence from the COVID-19 pandemic. Econ Model. https://doi.org/10.1016/J.ECONMOD.2023.106500

Pashapour SA, Azadeh A, Ghaderi SF, Keramati A (2019) Performance optimization of organizations considering economic resilience factors under uncertainty: a case study of a petrochemical plant. J Clean Prod. https://doi.org/10.1016/j.jclepro.2019.05.171

Qiang Y, Huang QX, Xu JW (2020) Observing community resilience from space: Using nighttime lights to model economic disturbance and recovery pattern in natural disaster. Sustain Cities Soc. https://doi.org/10.1016/j.scs.2020.102115

Roberta C, Andrea C, Ugo F (2015) Spatial heterogeneity in the costs of the economic crisis in Europe: are cities sources of regional resilience? J Econ Geogr 15(5):951–972

Shang ZC (2022) China’s ecological civilization and economic development model under globalization. Financ Eng Risk Manag. https://doi.org/10.23977/FERM.2022.050606

Tan JT, Zhang PY, Kevin L, Li J, Liu SW (2017) Conceptualizing and measuring economic resilience of resource-based cities: case study of Northeast China. Chin Geogr Sci 27(3):471–481

Tan JT, Zhao HB, Liu WX, Zhang PY, Qiu FD (2020) Regional economic resilience and influential mechanism during economic crises in China. Sci Geogr Sin 40(2):173–181

Google Scholar  

Tao MM, Stephen P,Wen L, Sheng MY, Wang JD, Wang GH, Zheng YH (2023) Appraising the role of the digital economy in global decarbonization: a spatial non-linear perspective on globalization. J Environ Manag 347:119170–119170

Wang JJ, Ma CC, Jiang T, Phillips A, Wu X, Wang Y, Wang ZM, Cao Y (2024) A spatial econometric investigation into road traffic accessibility and economic growth: insights from the Chengdu-Chongqing twin-city economic circle. Humanit Soc Sci Commun. https://doi.org/10.1057/S41599-024-02695-1

Wang H, Li Y, Liu YZ, Qing FL, Zhou YD, Chen YM, Fang DD (2021) Study on the Influencing factors of urban economic resilience in post epidemic Era—A case study of Kunming City. J Urban Manag. https://doi.org/10.1016/J.JUM.2021.06.006

Wang HP, Ge Q (2023) Spatial association network of economic resilience and its influencing factors: evidence from 31 Chinese provinces. Humanit Soc Sci Commun 10(1):290–290

Article   MathSciNet   PubMed   PubMed Central   Google Scholar  

Wang KW, Ma HT, Fang CL (2023) The relationship evolution between urbanization and urban ecological resilience in the Northern Slope Economic Belt of Tianshan Mountains, China. Sustain Cities Soc. https://doi.org/10.1016/J.SCS.2023.104783

Wang SJ, Wang ZH (2021) The spatio-temporal differentiation of urban land expansion and influencing factors in the Yangtze River Delta in the economic transition context. J Nat Resour 36(4):993–1007

Wang YW, Xu JG, Liu D, Zhou YY (2023) Analysis and comparison of the industrial economic resilience in the Taihu Lake Basin under the 2008 financial crisis and the 2018 Sino-US Trade War. Land 12(2):481–481

Xiong M, Cheng SM, Guo H, Zhao JZ (2023) The impact of local government fiscal gaps on public-private partnerships: government demand and private sector risk aversion. Int Public Manag J 26(4):589–608

Xuan D, Jian XY, Fang Y (2023) Can globalization and the green economy hedge natural resources? Functions of population growth and financial development in BRICS countries. Resour Policy. https://doi.org/10.1016/J.RESOURPOL.2023.103414

Yang YY, Chen WK, Gu RD (2023) How does digital infrastructure affect industrial eco-efficiency? Considering the threshold effect of regional collaborative innovation. J Clean Prod. https://doi.org/10.1016/J.JCLEPRO.2023.139248

Yang ZS (2023) Human capital space: a spatial perspective of the dynamics of people and economic relationships. Humanit Soc Sci Commun 10(1):145–145

Article   PubMed   PubMed Central   Google Scholar  

Ye SS, Qian Z (2021) The economic network resilience of the Guanzhong plain city cluster, China: a network analysis from the evolutionary perspective. Growth Change 52(4):2391–2411

Yin Y, Su XY, Xie XQ (2023) The spatial effect of industrial structure upgrading on urban economic resilience. Econ Geogr. 43(8):86–92

Yu HJ, Wang YZ, Li DH (2023) Using the AHP-entropy weight method to study regional marine economy resilience in China. J Coast Res 39(5):960–969

Ženka J, Slach O, Pavlík A (2019) Economic resilience of metropolitan, old industrial, and rural regions in two subsequent recessionary shocks. Eur Plan Stud 27(11):2288–2311

Zhang J (2022) Spatial effects of tourism development on economic resilience: an empirical study of Wenchuan earthquake based on dynamic spatial Durbin model. Nat Hazards 115(1):309–329

Zhang LX, Bi X, Huang ZJ (2020) Urban industrial land use efficiency under the background of economic transformation in the Yangtze River Economic Belt. Resour Sci 42(9):1728–1738

Zhang L, Yao L (2023) A study on the impact of digital technology innovation on urban economic resilience: empirical evidence from 278 prefectural and above cities in China. J Xinxiang Educ Coll 36(05):38–59

Zhang XB, Fu WS, Ma HT (2023) Coupling correlation between economic resilience and industrial structure evolution in the Beijing-Tianjin-Hebei region. Acta Geogr Sin. https://doi.org/10.11821/dlxb202310002

Zheng ZY, Xu YB, Chen JY (2023) Digital economy, industrial structure upgrading and green innovation efficiency of family enterprises. Int Entrep Manag J 20(1):479–503

Zheng HZ, Qian J, Liu GH,Wu YG, Delang, CO, He HM (2023) Housing prices and household consumption: a threshold effect model analysis in central and western China. Humanit Soc Sci Commun. https://doi.org/10.1057/S41599-023-02258-W

Zhou D, Chen A, Wang J (2019) Impact of disaster risks on regional economic resilience in China: a case study of Wenchuan earthquake. Soc. Sci. 8(5):245–254

Zhou Q, Qi ZP (2023) Urban economic resilience and human capital: an exploration of heterogeneity and mechanism in the context of spatial population mobility. Sustain Cities Soc. https://doi.org/10.1016/J.SCS.2023.104983

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China [42271284;41801205;71974071;42171286].Fundamental Scientific Research Business Expense for Higher School of Central Government[CCNU23ZZ009].

Author information

Authors and affiliations.

Faculty of Political Science, School of Public Administration, Central China Normal University, Wuhan, China

Lu Zhang & Guodong Lin

School of Humanities and Law, Northeastern University, Shenyang, China

School of Management, Guangzhou College of Technology and Business, Guangzhou, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: LZ; methodology: WS; software: XL; validation, formal analysis, investigation, data curation, writing-original draft preparation, writing—review and editing, and visualization: GL; supervision: LZ; All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Guodong Lin .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This research was not required to receive any ethical approval because it did not involve human research participants and no primary data were collected. It uses data collected by statistical organizations such as the Statistical Yearbook of Urban Construction in China and the Statistical Year book of Chinese Cities from 2008 to 2021.

Informed consent

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Zhang, L., Lin, G., Lyu, X. et al. Suppression or promotion: research on the impact of industrial structure upgrading on urban economic resilience. Humanit Soc Sci Commun 11 , 843 (2024). https://doi.org/10.1057/s41599-024-03329-2

Download citation

Received : 12 December 2023

Accepted : 11 June 2024

Published : 27 June 2024

DOI : https://doi.org/10.1057/s41599-024-03329-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

The evidence is mounting: humans were responsible for the extinction of large mammals

The debate has raged for decades: Was it humans or climate change that led to the extinction of many species of large mammals, birds, and reptiles that have disappeared from Earth over the past 50,000 years?

By "large," we mean animals that weighed at least 45 kilograms -- known as megafauna. At least 161 species of mammals were driven to extinction during this period. This number is based on the remains found so far.

The largest of them were hit the hardest -- land-dwelling herbivores weighing over a ton, the megaherbivores. Fifty thousand years ago, there were 57 species of megaherbivores. Today, only 11 remain. These remaining 11 species have also seen drastic declines in their populations, but not to the point of complete extinction.

A research group from the Danish National Research Foundation's Center for Ecological Dynamics in a Novel Biosphere (ECONOVO) at Aarhus University now concludes that many of these vanished species were hunted to extinction by humans.

Many different fields of research

They present this conclusion in a review article invited by and published in the scientific journal Cambridge Prisms: Extinction . A review article synthesizes and analyses existing research within a particular field.

In this case, the researchers from Aarhus University incorporated several research fields, including studies directly related to the extinction of large animals, such as:

- The timing of species extinctions

- The animals' dietary preferences

- Climate and habitat requirements

- Genetic estimates of past population sizes

- Evidence of human hunting

Additionally, they included a wide range of studies from other fields necessary to understand the phenomenon, such as:

- Climate history over the past 1-3 million years

- Vegetation history over the past 1-3 million years

- Evolution and dynamics of fauna over the past 66 million years

- Archaeological data on human expansion and lifestyle, including dietary preferences

Climate change played a lesser role

The dramatic climate changes during the last interglacial and glacial periods (known as the late Pleistocene, from 130,000 to 11,000 years ago) certainly affected populations and distributions of both large and small animals and plants worldwide. However, significant extinctions were observed only among the large animals, particularly the largest ones.

An important observation is that the previous, equally dramatic ice ages and interglacials over the past couple of million years did not cause a selective loss of megafauna. Especially at the beginning of the glacial periods, the new cold and dry conditions caused large-scale extinctions in some regions, such as trees in Europe. However, there were no selective extinctions of large animals.

"The large and very selective loss of megafauna over the last 50,000 years is unique over the past 66 million years. Previous periods of climate change did not lead to large, selective extinctions, which argues against a major role for climate in the megafauna extinctions," says Professor Jens-Christian Svenning. He leads ECONOVO and is the lead author of the article. He adds, "Another significant pattern that argues against a role for climate is that the recent megafauna extinctions hit just as hard in climatically stable areas as in unstable areas."

Effective hunters and vulnerable giants

Archaeologists have found traps designed for very large animals, and isotope analyses of ancient human bones and protein residues from spear points show that they hunted and ate the largest mammals.

Jens-Christian Svenning adds, "Early modern humans were effective hunters of even the largest animal species and clearly had the ability to reduce the populations of large animals. These large animals were and are particularly vulnerable to overexploitation because they have long gestation periods, produce very few offspring at a time, and take many years to reach sexual maturity."

The analysis shows that human hunting of large animals such as mammoths, mastodons, and giant sloths was widespread and consistent across the world.

It also shows that the species went extinct at very different times and at different rates around the world. In some local areas, it happened quite quickly, while in other places it took over 10,000 years. But everywhere, it occurred after modern humans arrived, or in Africa's case, after cultural advancements among humans.

…in all types of environments

Species went extinct on all continents except Antarctica and in all types of ecosystems, from tropical forests and savannas to Mediterranean and temperate forests and steppes to arctic ecosystems.

"Many of the extinct species could thrive in various types of environments. Therefore, their extinction cannot be explained by climate changes causing the disappearance of a specific ecosystem type, such as the mammoth steppe -- which also housed only a few megafauna species," explains Jens-Christian Svenning. "Most of the species existed under temperate to tropical conditions and should actually have benefited from the warming at the end of the last ice age."

Consequences and recommendations

The researchers point out that the loss of megafauna has had profound ecological consequences. Large animals play a central role in ecosystems by influencing vegetation structure (e.g., the balance between dense forests and open areas), seed dispersal, and nutrient cycling. Their disappearance has resulted in significant changes in ecosystem structures and functions.

"Our results highlight the need for active conservation and restoration efforts. By reintroducing large mammals, we can help restore ecological balances and support biodiversity, which evolved in ecosystems rich in megafauna," says Jens-Christian Svenning.

FACTS: The numbers of extinct and surviving species come from the freely accessible database PHYLACINE 1.2.1, which lists all known mammals that have lived in the past 129,000 years, including those that have gone extinct recently or are only found in captivity.

• Endangered Animals
• New Species
• Early Climate
• Early Mammals
• Origin of Life
• Toba catastrophe theory
• Temperature record of the past 1000 years
• Endangered species
• Global warming
• Recent single-origin hypothesis
• Right whale

Story Source:

Materials provided by Aarhus University . Original written by Peter Gammelby. Note: Content may be edited for style and length.

Related Multimedia :

• Hunting and extinction of large mammals

Journal Reference :

• Jens-Christian Svenning, Rhys T. Lemoine, Juraj Bergman, Robert Buitenwerf, Elizabeth Le Roux, Erick Lundgren, Ninad Mungi, Rasmus Ø. Pedersen. The late-Quaternary megafauna extinctions: Patterns, causes, ecological consequences and implications for ecosystem management in the Anthropocene . Cambridge Prisms: Extinction , 2024; 2 DOI: 10.1017/ext.2024.4

Cite This Page :

Explore More

• Extinct Humans On Tibetan Plateau: 160,000 Years
• Excellent Memory of Birds
• Seams in Clothing Capture Body Movement
• Moon 'Swirls' Magnetized by Unseen Magmas?
• Inexpensive, Clean, Fast-Charging Batteries
• Giant 'Salamander': Top Predator Before Dinos
• Humans Caused Extinction of Large Mammals
• The Beginnings of Fashion
• Weight Change and Antidepressant Meds
• Crucial Gaps in Climate Risk Assessment Methods

Trending Topics

Strange & offbeat.