how to do critical review of research articles

Get Started

Take the first step and invest in your future.

Online Programs

Offering flexibility & convenience in 51 online degrees & programs.

Prairie Stars

Featuring 15 intercollegiate NCAA Div II athletic teams.

Find your Fit

UIS has over 85 student and 10 greek life organizations, and many volunteer opportunities.

Arts & Culture

Celebrating the arts to create rich cultural experiences on campus.

Give Like a Star

Your generosity helps fuel fundraising for scholarships, programs and new initiatives.

Bragging Rights

UIS was listed No. 1 in Illinois and No. 3 in the Midwest in 2023 rankings.

• Quick links Applicants & Students Important Apps & Links Alumni Faculty and Staff Community Admissions How to Apply Cost & Aid Tuition Calculator Registrar Orientation Visit Campus Academics Register for Class Programs of Study Online Degrees & Programs Graduate Education International Student Services Study Away Student Support Bookstore UIS Life Dining Diversity & Inclusion Get Involved Health & Wellness COVID-19 United in Safety Residence Life Student Life Programs UIS Connection Important Apps UIS Mobile App Advise U Canvas myUIS i-card Balance Pay My Bill - UIS Bursar Self-Service Email Resources Bookstore Box Information Technology Services Library Orbit Policies Webtools Get Connected Area Information Calendar Campus Recreation Departments & Programs (A-Z) Parking UIS Newsroom Connect & Get Involved Update your Info Alumni Events Alumni Networks & Groups Volunteer Opportunities Alumni Board News & Publications Featured Alumni Alumni News UIS Alumni Magazine Resources Order your Transcripts Give Back Alumni Programs Career Development Services & Support Accessibility Services Campus Services Campus Police Facilities & Services Registrar Faculty & Staff Resources Website Project Request Web Services Training & Tools Academic Impressions Career Connect CSA Reporting Cybersecurity Training Faculty Research FERPA Training Website Login Campus Resources Newsroom Campus Calendar Campus Maps i-Card Human Resources Public Relations Webtools Arts & Events UIS Performing Arts Center Visual Arts Gallery Event Calendar Sangamon Experience Center for Lincoln Studies ECCE Speaker Series Community Engagement Center for State Policy and Leadership Illinois Innocence Project Innovate Springfield Central IL Nonprofit Resource Center NPR Illinois Community Resources Child Protection Training Academy Office of Electronic Media University Archives/IRAD Institute for Illinois Public Finance

Request Info

How to Review a Journal Article

• Request Info Request info for....     Undergraduate/Graduate     Online     Study Away     Continuing & Professional Education     International Student Services     General Inquiries

For many kinds of assignments, like a  literature review , you may be asked to offer a critique or review of a journal article. This is an opportunity for you as a scholar to offer your  qualified opinion  and  evaluation  of how another scholar has composed their article, argument, and research. That means you will be expected to go beyond a simple  summary  of the article and evaluate it on a deeper level. As a college student, this might sound intimidating. However, as you engage with the research process, you are becoming immersed in a particular topic, and your insights about the way that topic is presented are valuable and can contribute to the overall conversation surrounding your topic.

IMPORTANT NOTE!!

Some disciplines, like Criminal Justice, may only want you to summarize the article without including your opinion or evaluation. If your assignment is to summarize the article only, please see our literature review handout.

Before getting started on the critique, it is important to review the article thoroughly and critically. To do this, we recommend take notes,  annotating , and reading the article several times before critiquing. As you read, be sure to note important items like the thesis, purpose, research questions, hypotheses, methods, evidence, key findings, major conclusions, tone, and publication information. Depending on your writing context, some of these items may not be applicable.

Questions to Consider

To evaluate a source, consider some of the following questions. They are broken down into different categories, but answering these questions will help you consider what areas to examine. With each category, we recommend identifying the strengths and weaknesses in each since that is a critical part of evaluation.

Evaluating Purpose and Argument

• How well is the purpose made clear in the introduction through background/context and thesis?
• How well does the abstract represent and summarize the article’s major points and argument?
• How well does the objective of the experiment or of the observation fill a need for the field?
• How well is the argument/purpose articulated and discussed throughout the body of the text?
• How well does the discussion maintain cohesion?

Evaluating the Presentation/Organization of Information

• How appropriate and clear is the title of the article?
• Where could the author have benefited from expanding, condensing, or omitting ideas?
• How clear are the author’s statements? Challenge ambiguous statements.
• What underlying assumptions does the author have, and how does this affect the credibility or clarity of their article?
• How objective is the author in his or her discussion of the topic?
• How well does the organization fit the article’s purpose and articulate key goals?

Evaluating Methods

• How appropriate are the study design and methods for the purposes of the study?
• How detailed are the methods being described? Is the author leaving out important steps or considerations?
• Have the procedures been presented in enough detail to enable the reader to duplicate them?

Evaluating Data

• Scan and spot-check calculations. Are the statistical methods appropriate?
• Do you find any content repeated or duplicated?
• How many errors of fact and interpretation does the author include? (You can check on this by looking up the references the author cites).
• What pertinent literature has the author cited, and have they used this literature appropriately?

Following, we have an example of a summary and an evaluation of a research article. Note that in most literature review contexts, the summary and evaluation would be much shorter. This extended example shows the different ways a student can critique and write about an article.

Chik, A. (2012). Digital gameplay for autonomous foreign language learning: Gamers’ and language teachers’ perspectives. In H. Reinders (ed.),  Digital games in language learning and teaching  (pp. 95-114). Eastbourne, UK: Palgrave Macmillan.

Be sure to include the full citation either in a reference page or near your evaluation if writing an  annotated bibliography .

In Chik’s article “Digital Gameplay for Autonomous Foreign Language Learning: Gamers’ and Teachers’ Perspectives”, she explores the ways in which “digital gamers manage gaming and gaming-related activities to assume autonomy in their foreign language learning,” (96) which is presented in contrast to how teachers view the “pedagogical potential” of gaming. The research was described as an “umbrella project” consisting of two parts. The first part examined 34 language teachers’ perspectives who had limited experience with gaming (only five stated they played games regularly) (99). Their data was recorded through a survey, class discussion, and a seven-day gaming trial done by six teachers who recorded their reflections through personal blog posts. The second part explored undergraduate gaming habits of ten Hong Kong students who were regular gamers. Their habits were recorded through language learning histories, videotaped gaming sessions, blog entries of gaming practices, group discussion sessions, stimulated recall sessions on gaming videos, interviews with other gamers, and posts from online discussion forums. The research shows that while students recognize the educational potential of games and have seen benefits of it in their lives, the instructors overall do not see the positive impacts of gaming on foreign language learning.

The summary includes the article’s purpose, methods, results, discussion, and citations when necessary.

This article did a good job representing the undergraduate gamers’ voices through extended quotes and stories. Particularly for the data collection of the undergraduate gamers, there were many opportunities for an in-depth examination of their gaming practices and histories. However, the representation of the teachers in this study was very uneven when compared to the students. Not only were teachers labeled as numbers while the students picked out their own pseudonyms, but also when viewing the data collection, the undergraduate students were more closely examined in comparison to the teachers in the study. While the students have fifteen extended quotes describing their experiences in their research section, the teachers only have two of these instances in their section, which shows just how imbalanced the study is when presenting instructor voices.

Some research methods, like the recorded gaming sessions, were only used with students whereas teachers were only asked to blog about their gaming experiences. This creates a richer narrative for the students while also failing to give instructors the chance to have more nuanced perspectives. This lack of nuance also stems from the emphasis of the non-gamer teachers over the gamer teachers. The non-gamer teachers’ perspectives provide a stark contrast to the undergraduate gamer experiences and fits neatly with the narrative of teachers not valuing gaming as an educational tool. However, the study mentioned five teachers that were regular gamers whose perspectives are left to a short section at the end of the presentation of the teachers’ results. This was an opportunity to give the teacher group a more complex story, and the opportunity was entirely missed.

Additionally, the context of this study was not entirely clear. The instructors were recruited through a master’s level course, but the content of the course and the institution’s background is not discussed. Understanding this context helps us understand the course’s purpose(s) and how those purposes may have influenced the ways in which these teachers interpreted and saw games. It was also unclear how Chik was connected to this masters’ class and to the students. Why these particular teachers and students were recruited was not explicitly defined and also has the potential to skew results in a particular direction.

Overall, I was inclined to agree with the idea that students can benefit from language acquisition through gaming while instructors may not see the instructional value, but I believe the way the research was conducted and portrayed in this article made it very difficult to support Chik’s specific findings.

Some professors like you to begin an evaluation with something positive but isn’t always necessary.

The evaluation is clearly organized and uses transitional phrases when moving to a new topic.

This evaluation includes a summative statement that gives the overall impression of the article at the end, but this can also be placed at the beginning of the evaluation.

This evaluation mainly discusses the representation of data and methods. However, other areas, like organization, are open to critique.

• Queen's University Library
• Research Guides

Introduction to Research: Humanities and Social Sciences

• Critical Reviews
• Choosing Your Research Topic
• Finding Background Information
• Choosing Relevant Keywords
• Connecting Keywords
• Citation Searching
• Articles (General)
• Scholarly Articles
• Films & Videos
• Scholarly vs. Popular Sources
• Scholarly vs. Non-Scholarly Periodicals
• Web Resources
• Annotated Bibliography
• Reflective Writing
• Citing Sources
• Off Campus?

Writing Critical Reviews

What is a Critical Review of a Journal Article?

A critical review of a journal article evaluates the strengths and weaknesses of an article's ideas and content. It provides description, analysis and interpretation that allow readers to assess the article's value.

Before You Read the Article

• What does the title lead you to expect about the article?
• Study any sub-headings to understand how the author organized the content.
• Read the abstract for a summary of the author's arguments.
• Study the list of references to determine what research contributed to the author's arguments. Are the references recent? Do they represent important work in the field?
• If possible, read about the author to learn what authority he or she has to write about the subject.
• Consult Web of Science to see if other writers have cited the author's work. (Please see 'How to use E-Indexes'.) Has the author made an important contribution to the field of study?

Reading the Article: Points to Consider

Read the article carefully. Record your impressions and note sections suitable for quoting.

• Who is the intended audience?
• What is the author's purpose? To survey and summarize research on a topic? To present an argument that builds on past research? To refute another writer's argument?
• Does the author define important terms?
• Is the information in the article fact or opinion? (Facts can be verified, while opinions arise from interpretations of facts.) Does the information seem well-researched or is it unsupported?
• What are the author's central arguments or conclusions? Are they clearly stated? Are they supported by evidence and analysis?
• If the article reports on an experiment or study, does the author clearly outline methodology and the expected result?
• Is the article lacking information or argumentation that you expected to find?
• Is the article organized logically and easy to follow?
• Does the writer's style suit the intended audience? Is the style stilted or unnecessarily complicated?
• Is the author's language objective or charged with emotion and bias?
• If illustrations or charts are used, are they effective in presenting information?

Prepare an Outline

Read over your notes. Choose a statement that expresses the central purpose or thesis of your review. When thinking of a thesis, consider the author's intentions and whether or not you think those intentions were successfully realized. Eliminate all notes that do not relate to your thesis. Organize your remaining points into separate groups such as points about structure, style, or argument. Devise a logical sequence for presenting these ideas. Remember that all of your ideas must support your central thesis.

Write the First Draft

The review should begin with a complete citation of the article. For example:

Platt, Kevin M.F. "History and Despotism, or: Hayden White vs. Ivan the Terrible  and Peter the Great." Rethinking History 3:3 (1999) : 247-269.

NOTE: Use the same bibliographic citation format as you would for any bibliography, works cited or reference list. It will follow a standard documentation style such as MLA or APA.

Be sure to ask your instructor which citation style to use. For frequently used style guides consult Queen's University Library's Citing Sources guide.

The first paragraph may contain:

• a statement of your thesis
• the author's purpose in writing the article
• comments on how the article relates to other work on the same subject
• information about the author's reputation or authority in the field

The body of the review should:

• state your arguments in support of your thesis
• follow the logical development of ideas that you mapped out in your outline
• include quotations from the article which illustrate your main ideas

The concluding paragraph may:

• summarize your review
• restate your thesis

Revise the First Draft

Ideally, you should leave your first draft for a day or two before revising. This allows you to gain a more objective perspective on your ideas. Check for the following when revising:

• grammar and punctuation errors
• organization, logical development and solid support of your thesis
• errors in quotations or in references

You may make major revisions in the organization or content of your review during the revision process. Revising can even lead to a radical change in your central thesis.

NOTE: Prepared by University of Toronto Mississauga Library, Hazel McCallion Academic Learning Centre.

• << Previous: Writing Resources
• Next: Annotated Bibliography >>

Additional Resources

Writing a Critical Review (Allyson Skene, The Writing Centre, U of Toronto at Scarborough)

The Book Review or Article Critique (Margaret Procter, Writing Support, University of Toronto)

Critical Reviews of Journal Articles (Herbert Coutts, University of Alberta)

Writing a Critical Review (The Writing Centre, Queen's University)

• Last Updated: Aug 29, 2023 12:23 PM
• Subjects: Multidisciplinary

How to Write Critical Reviews

When you are asked to write a critical review of a book or article, you will need to identify, summarize, and evaluate the ideas and information the author has presented. In other words, you will be examining another person’s thoughts on a topic from your point of view.

Your stand must go beyond your “gut reaction” to the work and be based on your knowledge (readings, lecture, experience) of the topic as well as on factors such as criteria stated in your assignment or discussed by you and your instructor.

Make your stand clear at the beginning of your review, in your evaluations of specific parts, and in your concluding commentary.

Remember that your goal should be to make a few key points about the book or article, not to discuss everything the author writes.

Understanding the Assignment

To write a good critical review, you will have to engage in the mental processes of analyzing (taking apart) the work–deciding what its major components are and determining how these parts (i.e., paragraphs, sections, or chapters) contribute to the work as a whole.

Analyzing the work will help you focus on how and why the author makes certain points and prevent you from merely summarizing what the author says. Assuming the role of an analytical reader will also help you to determine whether or not the author fulfills the stated purpose of the book or article and enhances your understanding or knowledge of a particular topic.

Be sure to read your assignment thoroughly before you read the article or book. Your instructor may have included specific guidelines for you to follow. Keeping these guidelines in mind as you read the article or book can really help you write your paper!

Also, note where the work connects with what you’ve studied in the course. You can make the most efficient use of your reading and notetaking time if you are an active reader; that is, keep relevant questions in mind and jot down page numbers as well as your responses to ideas that appear to be significant as you read.

Please note: The length of your introduction and overview, the number of points you choose to review, and the length of your conclusion should be proportionate to the page limit stated in your assignment and should reflect the complexity of the material being reviewed as well as the expectations of your reader.

Write the introduction

Below are a few guidelines to help you write the introduction to your critical review.

Introduce your review appropriately

Begin your review with an introduction appropriate to your assignment.

If your assignment asks you to review only one book and not to use outside sources, your introduction will focus on identifying the author, the title, the main topic or issue presented in the book, and the author’s purpose in writing the book.

If your assignment asks you to review the book as it relates to issues or themes discussed in the course, or to review two or more books on the same topic, your introduction must also encompass those expectations.

Explain relationships

For example, before you can review two books on a topic, you must explain to your reader in your introduction how they are related to one another.

Within this shared context (or under this “umbrella”) you can then review comparable aspects of both books, pointing out where the authors agree and differ.

In other words, the more complicated your assignment is, the more your introduction must accomplish.

Finally, the introduction to a book review is always the place for you to establish your position as the reviewer (your thesis about the author’s thesis).

As you write, consider the following questions:

• Is the book a memoir, a treatise, a collection of facts, an extended argument, etc.? Is the article a documentary, a write-up of primary research, a position paper, etc.?
• Who is the author? What does the preface or foreword tell you about the author’s purpose, background, and credentials? What is the author’s approach to the topic (as a journalist? a historian? a researcher?)?
• What is the main topic or problem addressed? How does the work relate to a discipline, to a profession, to a particular audience, or to other works on the topic?
• What is your critical evaluation of the work (your thesis)? Why have you taken that position? What criteria are you basing your position on?

Provide an overview

In your introduction, you will also want to provide an overview. An overview supplies your reader with certain general information not appropriate for including in the introduction but necessary to understanding the body of the review.

Generally, an overview describes your book’s division into chapters, sections, or points of discussion. An overview may also include background information about the topic, about your stand, or about the criteria you will use for evaluation.

The overview and the introduction work together to provide a comprehensive beginning for (a “springboard” into) your review.

• What are the author’s basic premises? What issues are raised, or what themes emerge? What situation (i.e., racism on college campuses) provides a basis for the author’s assertions?
• How informed is my reader? What background information is relevant to the entire book and should be placed here rather than in a body paragraph?

Write the body

The body is the center of your paper, where you draw out your main arguments. Below are some guidelines to help you write it.

Organize using a logical plan

Organize the body of your review according to a logical plan. Here are two options:

• First, summarize, in a series of paragraphs, those major points from the book that you plan to discuss; incorporating each major point into a topic sentence for a paragraph is an effective organizational strategy. Second, discuss and evaluate these points in a following group of paragraphs. (There are two dangers lurking in this pattern–you may allot too many paragraphs to summary and too few to evaluation, or you may re-summarize too many points from the book in your evaluation section.)
• Alternatively, you can summarize and evaluate the major points you have chosen from the book in a point-by-point schema. That means you will discuss and evaluate point one within the same paragraph (or in several if the point is significant and warrants extended discussion) before you summarize and evaluate point two, point three, etc., moving in a logical sequence from point to point to point. Here again, it is effective to use the topic sentence of each paragraph to identify the point from the book that you plan to summarize or evaluate.

Questions to keep in mind as you write

With either organizational pattern, consider the following questions:

• What are the author’s most important points? How do these relate to one another? (Make relationships clear by using transitions: “In contrast,” an equally strong argument,” “moreover,” “a final conclusion,” etc.).
• What types of evidence or information does the author present to support his or her points? Is this evidence convincing, controversial, factual, one-sided, etc.? (Consider the use of primary historical material, case studies, narratives, recent scientific findings, statistics.)
• Where does the author do a good job of conveying factual material as well as personal perspective? Where does the author fail to do so? If solutions to a problem are offered, are they believable, misguided, or promising?
• Which parts of the work (particular arguments, descriptions, chapters, etc.) are most effective and which parts are least effective? Why?
• Where (if at all) does the author convey personal prejudice, support illogical relationships, or present evidence out of its appropriate context?

Keep your opinions distinct and cite your sources

Remember, as you discuss the author’s major points, be sure to distinguish consistently between the author’s opinions and your own.

Keep the summary portions of your discussion concise, remembering that your task as a reviewer is to re-see the author’s work, not to re-tell it.

And, importantly, if you refer to ideas from other books and articles or from lecture and course materials, always document your sources, or else you might wander into the realm of plagiarism.

Include only that material which has relevance for your review and use direct quotations sparingly. The Writing Center has other handouts to help you paraphrase text and introduce quotations.

Write the conclusion

You will want to use the conclusion to state your overall critical evaluation.

You have already discussed the major points the author makes, examined how the author supports arguments, and evaluated the quality or effectiveness of specific aspects of the book or article.

Now you must make an evaluation of the work as a whole, determining such things as whether or not the author achieves the stated or implied purpose and if the work makes a significant contribution to an existing body of knowledge.

Consider the following questions:

• Is the work appropriately subjective or objective according to the author’s purpose?
• How well does the work maintain its stated or implied focus? Does the author present extraneous material? Does the author exclude or ignore relevant information?
• How well has the author achieved the overall purpose of the book or article? What contribution does the work make to an existing body of knowledge or to a specific group of readers? Can you justify the use of this work in a particular course?
• What is the most important final comment you wish to make about the book or article? Do you have any suggestions for the direction of future research in the area? What has reading this work done for you or demonstrated to you?

Academic and Professional Writing

This is an accordion element with a series of buttons that open and close related content panels.

Analysis Papers

Reading Poetry

A Short Guide to Close Reading for Literary Analysis

Using Literary Quotations

Play Reviews

Writing a Rhetorical Précis to Analyze Nonfiction Texts

Incorporating Interview Data

Grant Proposals

Planning and Writing a Grant Proposal: The Basics

Additional Resources for Grants and Proposal Writing

Job Materials and Application Essays

Writing Personal Statements for Ph.D. Programs

• Before you begin: useful tips for writing your essay
• Guided brainstorming exercises
• Get more help with your essay
• Frequently Asked Questions

Resume Writing Tips

CV Writing Tips

Cover Letters

Business Letters

Proposals and Dissertations

Resources for Proposal Writers

Resources for Dissertators

Research Papers

Planning and Writing Research Papers

Quoting and Paraphrasing

Writing Annotated Bibliographies

Creating Poster Presentations

Writing an Abstract for Your Research Paper

Thank-You Notes

Advice for Students Writing Thank-You Notes to Donors

Reading for a Review

Critical Reviews

Writing a Review of Literature

Scientific Reports

Scientific Report Format

Sample Lab Assignment

Writing for the Web

Writing an Effective Blog Post

Writing for Social Media: A Guide for Academics

IOE Writing Centre

• Writing a Critical Review

Writing a Critique

A critique (or critical review) is not to be mistaken for a literature review. A 'critical review', or 'critique', is a complete type of text (or genre), discussing one particular article or book in detail.  In some instances, you may be asked to write a critique of two or three articles (e.g. a comparative critical review). In contrast, a 'literature review', which also needs to be 'critical', is a part of a larger type of text, such as a chapter of your dissertation.

Most importantly: Read your article / book as many times as possible, as this will make the critical review much easier.

1. Read and take notes 2. Organising your writing 3. Summary 4. Evaluation 5. Linguistic features of a critical review 6. Summary language 7. Evaluation language 8. Conclusion language 9. Example extracts from a critical review 10. Further resources

Read and Take Notes

To improve your reading confidence and efficiency, visit our pages on reading.

Further reading: Read Confidently

After you are familiar with the text, make notes on some of the following questions. Choose the questions which seem suitable:

• What kind of article is it (for example does it present data or does it present purely theoretical arguments)?
• What is the main area under discussion?
• What are the main findings?
• What are the stated limitations?
• Where does the author's data and evidence come from? Are they appropriate / sufficient?
• What are the main issues raised by the author?
• What questions are raised?
• How well are these questions addressed?
• What are the major points/interpretations made by the author in terms of the issues raised?
• Is the text balanced? Is it fair / biased?
• Does the author contradict herself?
• How does all this relate to other literature on this topic?
• How does all this relate to your own experience, ideas and views?
• What else has this author written? Do these build / complement this text?
• (Optional) Has anyone else reviewed this article? What did they say? Do I agree with them?

^ Back to top

Organising your writing

You first need to summarise the text that you have read. One reason to summarise the text is that the reader may not have read the text. In your summary, you will

• focus on points within the article that you think are interesting
• summarise the author(s) main ideas or argument
• explain how these ideas / argument have been constructed. (For example, is the author basing her arguments on data that they have collected? Are the main ideas / argument purely theoretical?)

In your summary you might answer the following questions:     Why is this topic important?     Where can this text be located? For example, does it address policy studies?     What other prominent authors also write about this?

Evaluation is the most important part in a critical review.

Use the literature to support your views. You may also use your knowledge of conducting research, and your own experience. Evaluation can be explicit or implicit.

Explicit evaluation

Explicit evaluation involves stating directly (explicitly) how you intend to evaluate the text. e.g. "I will review this article by focusing on the following questions. First, I will examine the extent to which the authors contribute to current thought on Second Language Acquisition (SLA) pedagogy. After that, I will analyse whether the authors' propositions are feasible within overseas SLA classrooms."

Implicit evaluation

Implicit evaluation is less direct. The following section on Linguistic Features of Writing a Critical Review contains language that evaluates the text. A difficult part of evaluation of a published text (and a professional author) is how to do this as a student. There is nothing wrong with making your position as a student explicit and incorporating it into your evaluation. Examples of how you might do this can be found in the section on Linguistic Features of Writing a Critical Review. You need to remember to locate and analyse the author's argument when you are writing your critical review. For example, you need to locate the authors' view of classroom pedagogy as presented in the book / article and not present a critique of views of classroom pedagogy in general.

Linguistic features of a critical review

The following examples come from published critical reviews. Some of them have been adapted for student use.

Summary language

•     This article / book is divided into two / three parts. First...
•     While the title might suggest...
•     The tone appears to be...
•     Title is the first / second volume in the series Title, edited by...The books / articles in this series address...
•     The second / third claim is based on...
•     The author challenges the notion that...
•     The author tries to find a more middle ground / make more modest claims...
•     The article / book begins with a short historical overview of...
•     Numerous authors have recently suggested that...(see Author, Year; Author, Year). Author would also be once such author. With his / her argument that...
•     To refer to title as a...is not to say that it is...
•     This book / article is aimed at... This intended readership...
•     The author's book / article examines the...To do this, the author first...
•     The author develops / suggests a theoretical / pedagogical model to…
•     This book / article positions itself firmly within the field of...
•     The author in a series of subtle arguments, indicates that he / she...
•     The argument is therefore...
•     The author asks "..."
•     With a purely critical / postmodern take on...
•     Topic, as the author points out, can be viewed as...
•     In this recent contribution to the field of...this British author...
•     As a leading author in the field of...
•     This book / article nicely contributes to the field of...and complements other work by this author...
•     The second / third part of...provides / questions / asks the reader...
•     Title is intended to encourage students / researchers to...
•     The approach taken by the author provides the opportunity to examine...in a qualitative / quantitative research framework that nicely complements...
•     The author notes / claims that state support / a focus on pedagogy / the adoption of...remains vital if...
•     According to Author (Year) teaching towards examinations is not as effective as it is in other areas of the curriculum. This is because, as Author (Year) claims that examinations have undue status within the curriculum.
•     According to Author (Year)…is not as effective in some areas of the curriculum / syllabus as others. Therefore the author believes that this is a reason for some school's…

Evaluation language

•     This argument is not entirely convincing, as...furthermore it commodifies / rationalises the...
•     Over the last five / ten years the view of...has increasingly been viewed as 'complicated' (see Author, Year; Author, Year).
•     However, through trying to integrate...with...the author...
•     There are difficulties with such a position.
•     Inevitably, several crucial questions are left unanswered / glossed over by this insightful / timely / interesting / stimulating book / article. Why should...
•     It might have been more relevant for the author to have written this book / article as...
•     This article / book is not without disappointment from those who would view...as...
•     This chosen framework enlightens / clouds...
•     This analysis intends to be...but falls a little short as...
•     The authors rightly conclude that if...
•     A detailed, well-written and rigorous account of...
•     As a Korean student I feel that this article / book very clearly illustrates...
•     The beginning of...provides an informative overview into...
•     The tables / figures do little to help / greatly help the reader...
•     The reaction by scholars who take a...approach might not be so favourable (e.g. Author, Year).
•     This explanation has a few weaknesses that other researchers have pointed out (see Author, Year; Author, Year). The first is...
•     On the other hand, the author wisely suggests / proposes that...By combining these two dimensions...
•     The author's brief introduction to...may leave the intended reader confused as it fails to properly...
•     Despite my inability to...I was greatly interested in...
•     Even where this reader / I disagree(s), the author's effort to...
•     The author thus combines...with...to argue...which seems quite improbable for a number of reasons. First...
•     Perhaps this aversion to...would explain the author's reluctance to...
•     As a second language student from ...I find it slightly ironic that such an anglo-centric view is...
•     The reader is rewarded with...
•     Less convincing is the broad-sweeping generalisation that...
•     There is no denying the author's subject knowledge nor his / her...
•     The author's prose is dense and littered with unnecessary jargon...
•     The author's critique of...might seem harsh but is well supported within the literature (see Author, Year; Author, Year; Author, Year). Aligning herself with the author, Author (Year) states that...
•     As it stands, the central focus of Title is well / poorly supported by its empirical findings...
•     Given the hesitation to generalise to...the limitation of...does not seem problematic...
•     For instance, the term...is never properly defined and the reader left to guess as to whether...
•     Furthermore, to label...as...inadvertently misguides...
•     In addition, this research proves to be timely / especially significant to... as recent government policy / proposals has / have been enacted to...
•     On this well researched / documented basis the author emphasises / proposes that...
•     Nonetheless, other research / scholarship / data tend to counter / contradict this possible trend / assumption...(see Author, Year; Author, Year).
•     Without entering into detail of the..., it should be stated that Title should be read by...others will see little value in...
•     As experimental conditions were not used in the study the word 'significant' misleads the reader.
•     The article / book becomes repetitious in its assertion that...
•     The thread of the author's argument becomes lost in an overuse of empirical data...
•     Almost every argument presented in the final section is largely derivative, providing little to say about...
•     She / he does not seem to take into consideration; however, that there are fundamental differences in the conditions of…
•     As Author (Year) points out, however, it seems to be necessary to look at…
•     This suggest that having low…does not necessarily indicate that…is ineffective.
•     Therefore, the suggestion made by Author (Year)…is difficult to support.
•     When considering all the data presented…it is not clear that the low scores of some students, indeed, reflects…

Conclusion language

•     Overall this article / book is an analytical look at...which within the field of...is often overlooked.
•     Despite its problems, Title offers valuable theoretical insights / interesting examples / a contribution to pedagogy and a starting point for students / researchers of...with an interest in...
•     This detailed and rigorously argued...
•     This first / second volume / book / article by...with an interest in...is highly informative...

Example extracts from a critical review

Writing critically.

If you have been told your writing is not critical enough, it probably means that your writing treats the knowledge claims as if they are true, well supported, and applicable in the context you are writing about. This may not always be the case.

In these two examples, the extracts refer to the same section of text. In each example, the section that refers to a source has been highlighted in bold. The note below the example then explains how the writer has used the source material.    

There is a strong positive effect on students, both educationally and emotionally, when the instructors try to learn to say students' names without making pronunciation errors (Kiang, 2004).

Use of source material in example a: 

This is a simple paraphrase with no critical comment. It looks like the writer agrees with Kiang. (This is not a good example for critical writing, as the writer has not made any critical comment).        

Kiang (2004) gives various examples to support his claim that "the positive emotional and educational impact on students is clear" (p.210) when instructors try to pronounce students' names in the correct way. He quotes one student, Nguyet, as saying that he "felt surprised and happy" (p.211) when the tutor said his name clearly . The emotional effect claimed by Kiang is illustrated in quotes such as these, although the educational impact is supported more indirectly through the chapter. Overall, he provides more examples of students being negatively affected by incorrect pronunciation, and it is difficult to find examples within the text of a positive educational impact as such.

Use of source material in example b: 

The writer describes Kiang's (2004) claim and the examples which he uses to try to support it. The writer then comments that the examples do not seem balanced and may not be enough to support the claims fully. This is a better example of writing which expresses criticality.

^Back to top

Further resources

You may also be interested in our page on criticality, which covers criticality in general, and includes more critical reading questions.

Further reading: Read and Write Critically

We recommend that you do not search for other university guidelines on critical reviews. This is because the expectations may be different at other institutions. Ask your tutor for more guidance or examples if you have further questions.

IOE Writing Centre Online

Self-access resources from the Academic Writing Centre at the UCL Institute of Education.

Anonymous Suggestions Box

Information for Staff

Academic Writing Centre

Academic Writing Centre, UCL Institute of Education [email protected] Twitter:   @AWC_IOE Skype:   awc.ioe

Write a Critical Review

Introduction, how can i improve my critical review, ask us: chat, email, visit or call.

Video: How to Integrate Critical Voice into Your Literature Review

Video: Note-taking and Writing Tips to Avoid Plagiarism

More help: Writing

• Book Writing Appointments Get help on your writing assignments.
• To introduce the source, its main ideas, key details, and its place within the field
• To present your assessment of the quality of the source

In general, the introduction of your critical review should include

• Author(s) name
• Title of the source 
• What is the author's central purpose?
• What methods or theoretical frameworks were used to accomplish this purpose?
• What topic areas, chapters, sections, or key points did the author use to structure the source?
• What were the results or findings of the study?
• How were the results or findings interpreted? How were they related to the original problem (author's view of evidence rather than objective findings)?
• Who conducted the research? What were/are their interests?
• Why did they do this research?
• Was this research pertinent only within the author’s field, or did it have broader (even global) relevance?
• On what prior research was this source-based? What gap is the author attempting to address?
• How important was the research question posed by the researcher?
• Your overall opinion of the quality of the source. Think of this like a thesis or main argument.
• Present your evaluation of the source, providing evidence from the text (or other sources) to support your assessment.

In general, the body of your critical review should include

• Is the material organized logically and with appropriate headings?
• Are there stylistic problems in logical, clarity or language?
• Were the author(s) able to answer the question (test the hypothesis) raised
• What was the objective of the study?
• Does all the information lead coherently to the purpose of the study?
• Are the methods valid for studying the problem or gap?
• Could the study be duplicated from the information provided?
• Is the experimental design logical and reliable?
• How are the data organized? Is it logical and interpretable?
• Do the results reveal what the researcher intended?
• Do the authors present a logical interpretation of the results?
• Have the limitations of the research been addressed?
• Does the study consider other key studies in the field or other research possibilities or directions?
• How was the significance of the work described?
• Follow the structure of the journal article (e.g. Introduction, Methods, Results, Discussion) - highlighting the strengths and weaknesses in each section
• Present the weaknesses of the article, and then the strengths of the article (or vice versa).
• Group your ideas according to different research themes presented in the source
• Group the strengths and weaknesses of the article into the following areas: originality, reliability, validity, relevance, and presentation

Purpose: 

• To summarize the strengths and weaknesses of the article as a whole
• To assert the article’s practical and theoretical significance

In general, the conclusion of your critical review should include

• A restatement of your overall opinion
• A summary of the key strengths and weaknesses of the research that support your overall opinion of the source
• Did the research reported in this source result in the formation of new questions, theories or hypotheses by the authors or other researchers?
• Have other researchers subsequently supported or refuted the observations or interpretations of these authors?
• Did the research provide new factual information, a new understanding of a phenomenon in the field, a new research technique?
• Did the research produce any practical applications? 
• What are the social, political, technological, or medical implications of this research?
• How do you evaluate the significance of the research? 
• Find out what style guide you are required to follow (e.g., APA, MLA, Chicago) and follow the guidelines to create a reference list (may be called a bibliography or works cited).
• Be sure to include citations in the text when you refer to the source itself or external sources. 
• Check out our Cite Your Sources Guide for more information. 
• Read assignment instructions carefully and refer to them throughout the writing process.
• Make an outline of your main sections before you write.
• If your professor does not assign a topic or source, you must choose one yourself. Select a source that interests you and is written clearly so you can understand it.
• << Previous: Start Here
• Last Updated: Sep 26, 2023 10:58 AM
• URL: https://guides.lib.uoguelph.ca/CriticalReview

Suggest an edit to this guide

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

• Jump to menu
• Student Home
• Accept your offer
• How to enrol
• Student ID card
• Set up your IT
• Orientation Week
• Fees & payment
• Academic calendar
• Special consideration
• Transcripts
• The Nucleus: Student Hub
• Referencing
• Essay writing
• Learning abroad & exchange
• Professional development & UNSW Advantage
• Employability
• Financial assistance
• International students
• Equitable learning
• Postgraduate research
• Health Service
• Events & activities
• Emergencies
• Volunteering
• Clubs and societies
• Accommodation
• Health services
• Sport and gym
• Arc student organisation
• Security on campus
• Maps of campus
• Careers portal
• Change password

Structure of a Critical Review

Critical reviews, both short (one page) and long (four pages), usually have a similar structure. Check your assignment instructions for formatting and structural specifications. Headings are usually optional for longer reviews and can be helpful for the reader.

Introduction

The length of an introduction is usually one paragraph for a journal article review and two or three paragraphs for a longer book review. Include a few opening sentences that announce the author(s) and the title, and briefly explain the topic of the text. Present the aim of the text and summarise the main finding or key argument. Conclude the introduction with a brief statement of your evaluation of the text. This can be a positive or negative evaluation or, as is usually the case, a mixed response.

Present a summary of the key points along with a limited number of examples. You can also briefly explain the author’s purpose/intentions throughout the text and you may briefly describe how the text is organised. The summary should only make up about a third of the critical review.

The critique should be a balanced discussion and evaluation of the strengths, weakness and notable features of the text. Remember to base your discussion on specific criteria. Good reviews also include other sources to support your evaluation (remember to reference).

You can choose how to sequence your critique. Here are some examples to get you started:

• Most important to least important conclusions you make about the text.
• If your critique is more positive than negative, then present the negative points first and the positive last.
• If your critique is more negative than positive, then present the positive points first and the negative last.
• If there are both strengths and weakness for each criterion you use, you need to decide overall what your judgement is. For example, you may want to comment on a key idea in the text and have both positive and negative comments. You could begin by stating what is good about the idea and then concede and explain how it is limited in some way. While this example shows a mixed evaluation, overall you are probably being more negative than positive.
• In long reviews, you can address each criterion you choose in a paragraph, including both negative and positive points. For very short critical reviews (one page or less), where your comments will be briefer, include a paragraph of positive aspects  and another of negative.
• You can also include recommendations for how the text can be improved in terms of ideas, research approach; theories or frameworks used can also be included in the critique section.

Conclusion & References

This is usually a very short paragraph.

• Restate your overall opinion of the text.
• Briefly present recommendations.
• If necessary, some further qualification or explanation of your judgement can be included. This can help your critique sound fair and reasonable.

If you have used other sources in you review you should also include a list of references at the end of the review.

Summarising and paraphrasing for the critical review

The best way to summarise

• Scan the text. Look for information that can be deduced from the introduction, conclusion, title, and headings. What do these tell you about the main points of the article?
• Locate the topic sentences and highlight the main points as you read.
• Reread the text and make separate notes of the main points. Examples and evidence do not need to be included at this stage. Usually they are used selectively in your critique.

Paraphrasing means putting it into your own words. Paraphrasing offers an alternative to using direct quotations in your summary (and the critique) and can be an efficient way to integrate your summary notes.

The best way to paraphrase

• Review your summary notes
• Rewrite them in your own words and in complete sentences
• Use reporting verbs and phrases, e.g. 'The author describes…', 'Smith argues that …'.
• Use quotation marks if If you include unique or specialist phrases from the text.

  Next: Some general criteria for evaluating texts

Essay and assignment writing guide.

• Essay writing basics
• Essay and assignment planning
• Answering assignment questions
• Editing checklist
• Structure of a critical review
• General criteria for evaluating
• Sample extracts
• Annotated bibliography
• Reflective writing
• ^ More support

Study Hacks Workshops | All the hacks you need! 7 Feb – 10 Apr 2024

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 20 January 2009

How to critically appraise an article

• Jane M Young 1 &
• Michael J Solomon 2  

Nature Clinical Practice Gastroenterology & Hepatology volume  6 ,  pages 82–91 ( 2009 ) Cite this article

51k Accesses

100 Citations

425 Altmetric

Metrics details

Critical appraisal is a systematic process used to identify the strengths and weaknesses of a research article in order to assess the usefulness and validity of research findings. The most important components of a critical appraisal are an evaluation of the appropriateness of the study design for the research question and a careful assessment of the key methodological features of this design. Other factors that also should be considered include the suitability of the statistical methods used and their subsequent interpretation, potential conflicts of interest and the relevance of the research to one's own practice. This Review presents a 10-step guide to critical appraisal that aims to assist clinicians to identify the most relevant high-quality studies available to guide their clinical practice.

Critical appraisal is a systematic process used to identify the strengths and weaknesses of a research article

Critical appraisal provides a basis for decisions on whether to use the results of a study in clinical practice

Different study designs are prone to various sources of systematic bias

Design-specific, critical-appraisal checklists are useful tools to help assess study quality

Assessments of other factors, including the importance of the research question, the appropriateness of statistical analysis, the legitimacy of conclusions and potential conflicts of interest are an important part of the critical appraisal process

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Making sense of the literature: an introduction to critical appraisal for the primary care practitioner

Kishan Patel & Meera Pajpani

How to appraise the literature: basic principles for the busy clinician - part 2: systematic reviews and meta-analyses

Aslam Alkadhimi, Samuel Reeves & Andrew T. DiBiase

How to appraise the literature: basic principles for the busy clinician - part 1: randomised controlled trials

Druss BG and Marcus SC (2005) Growth and decentralisation of the medical literature: implications for evidence-based medicine. J Med Libr Assoc 93 : 499–501

PubMed   PubMed Central   Google Scholar  

Glasziou PP (2008) Information overload: what's behind it, what's beyond it? Med J Aust 189 : 84–85

PubMed   Google Scholar  

Last JE (Ed.; 2001) A Dictionary of Epidemiology (4th Edn). New York: Oxford University Press

Google Scholar  

Sackett DL et al . (2000). Evidence-based Medicine. How to Practice and Teach EBM . London: Churchill Livingstone

Guyatt G and Rennie D (Eds; 2002). Users' Guides to the Medical Literature: a Manual for Evidence-based Clinical Practice . Chicago: American Medical Association

Greenhalgh T (2000) How to Read a Paper: the Basics of Evidence-based Medicine . London: Blackwell Medicine Books

MacAuley D (1994) READER: an acronym to aid critical reading by general practitioners. Br J Gen Pract 44 : 83–85

CAS   PubMed   PubMed Central   Google Scholar  

Hill A and Spittlehouse C (2001) What is critical appraisal. Evidence-based Medicine 3 : 1–8 [ http://www.evidence-based-medicine.co.uk ] (accessed 25 November 2008)

Public Health Resource Unit (2008) Critical Appraisal Skills Programme (CASP) . [ http://www.phru.nhs.uk/Pages/PHD/CASP.htm ] (accessed 8 August 2008)

National Health and Medical Research Council (2000) How to Review the Evidence: Systematic Identification and Review of the Scientific Literature . Canberra: NHMRC

Elwood JM (1998) Critical Appraisal of Epidemiological Studies and Clinical Trials (2nd Edn). Oxford: Oxford University Press

Agency for Healthcare Research and Quality (2002) Systems to rate the strength of scientific evidence? Evidence Report/Technology Assessment No 47, Publication No 02-E019 Rockville: Agency for Healthcare Research and Quality

Crombie IK (1996) The Pocket Guide to Critical Appraisal: a Handbook for Health Care Professionals . London: Blackwell Medicine Publishing Group

Heller RF et al . (2008) Critical appraisal for public health: a new checklist. Public Health 122 : 92–98

Article   Google Scholar  

MacAuley D et al . (1998) Randomised controlled trial of the READER method of critical appraisal in general practice. BMJ 316 : 1134–37

Article   CAS   Google Scholar  

Parkes J et al . Teaching critical appraisal skills in health care settings (Review). Cochrane Database of Systematic Reviews 2005, Issue 3. Art. No.: cd001270. 10.1002/14651858.cd001270

Mays N and Pope C (2000) Assessing quality in qualitative research. BMJ 320 : 50–52

Hawking SW (2003) On the Shoulders of Giants: the Great Works of Physics and Astronomy . Philadelphia, PN: Penguin

National Health and Medical Research Council (1999) A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines . Canberra: National Health and Medical Research Council

US Preventive Services Taskforce (1996) Guide to clinical preventive services (2nd Edn). Baltimore, MD: Williams & Wilkins

Solomon MJ and McLeod RS (1995) Should we be performing more randomized controlled trials evaluating surgical operations? Surgery 118 : 456–467

Rothman KJ (2002) Epidemiology: an Introduction . Oxford: Oxford University Press

Young JM and Solomon MJ (2003) Improving the evidence-base in surgery: sources of bias in surgical studies. ANZ J Surg 73 : 504–506

Margitic SE et al . (1995) Lessons learned from a prospective meta-analysis. J Am Geriatr Soc 43 : 435–439

Shea B et al . (2001) Assessing the quality of reports of systematic reviews: the QUORUM statement compared to other tools. In Systematic Reviews in Health Care: Meta-analysis in Context 2nd Edition, 122–139 (Eds Egger M. et al .) London: BMJ Books

Chapter   Google Scholar  

Easterbrook PH et al . (1991) Publication bias in clinical research. Lancet 337 : 867–872

Begg CB and Berlin JA (1989) Publication bias and dissemination of clinical research. J Natl Cancer Inst 81 : 107–115

Moher D et al . (2000) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Br J Surg 87 : 1448–1454

Shea BJ et al . (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology 7 : 10 [10.1186/1471-2288-7-10]

Stroup DF et al . (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 283 : 2008–2012

Young JM and Solomon MJ (2003) Improving the evidence-base in surgery: evaluating surgical effectiveness. ANZ J Surg 73 : 507–510

Schulz KF (1995) Subverting randomization in controlled trials. JAMA 274 : 1456–1458

Schulz KF et al . (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273 : 408–412

Moher D et al . (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology 1 : 2 [ http://www.biomedcentral.com/ 1471-2288/1/2 ] (accessed 25 November 2008)

Rochon PA et al . (2005) Reader's guide to critical appraisal of cohort studies: 1. Role and design. BMJ 330 : 895–897

Mamdani M et al . (2005) Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ 330 : 960–962

Normand S et al . (2005) Reader's guide to critical appraisal of cohort studies: 3. Analytical strategies to reduce confounding. BMJ 330 : 1021–1023

von Elm E et al . (2007) Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335 : 806–808

Sutton-Tyrrell K (1991) Assessing bias in case-control studies: proper selection of cases and controls. Stroke 22 : 938–942

Knottnerus J (2003) Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol 56 : 1118–1128

Furukawa TA and Guyatt GH (2006) Sources of bias in diagnostic accuracy studies and the diagnostic process. CMAJ 174 : 481–482

Bossyut PM et al . (2003)The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 138 : W1–W12

STARD statement (Standards for the Reporting of Diagnostic Accuracy Studies). [ http://www.stard-statement.org/ ] (accessed 10 September 2008)

Raftery J (1998) Economic evaluation: an introduction. BMJ 316 : 1013–1014

Palmer S et al . (1999) Economics notes: types of economic evaluation. BMJ 318 : 1349

Russ S et al . (1999) Barriers to participation in randomized controlled trials: a systematic review. J Clin Epidemiol 52 : 1143–1156

Tinmouth JM et al . (2004) Are claims of equivalency in digestive diseases trials supported by the evidence? Gastroentrology 126 : 1700–1710

Kaul S and Diamond GA (2006) Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med 145 : 62–69

Piaggio G et al . (2006) Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA 295 : 1152–1160

Heritier SR et al . (2007) Inclusion of patients in clinical trial analysis: the intention to treat principle. In Interpreting and Reporting Clinical Trials: a Guide to the CONSORT Statement and the Principles of Randomized Controlled Trials , 92–98 (Eds Keech A. et al .) Strawberry Hills, NSW: Australian Medical Publishing Company

National Health and Medical Research Council (2007) National Statement on Ethical Conduct in Human Research 89–90 Canberra: NHMRC

Lo B et al . (2000) Conflict-of-interest policies for investigators in clinical trials. N Engl J Med 343 : 1616–1620

Kim SYH et al . (2004) Potential research participants' views regarding researcher and institutional financial conflicts of interests. J Med Ethics 30 : 73–79

Komesaroff PA and Kerridge IH (2002) Ethical issues concerning the relationships between medical practitioners and the pharmaceutical industry. Med J Aust 176 : 118–121

Little M (1999) Research, ethics and conflicts of interest. J Med Ethics 25 : 259–262

Lemmens T and Singer PA (1998) Bioethics for clinicians: 17. Conflict of interest in research, education and patient care. CMAJ 159 : 960–965

Download references

Author information

Authors and affiliations.

JM Young is an Associate Professor of Public Health and the Executive Director of the Surgical Outcomes Research Centre at the University of Sydney and Sydney South-West Area Health Service, Sydney,

Jane M Young

MJ Solomon is Head of the Surgical Outcomes Research Centre and Director of Colorectal Research at the University of Sydney and Sydney South-West Area Health Service, Sydney, Australia.,

Michael J Solomon

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jane M Young .

Ethics declarations

Competing interests.

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Young, J., Solomon, M. How to critically appraise an article. Nat Rev Gastroenterol Hepatol 6 , 82–91 (2009). https://doi.org/10.1038/ncpgasthep1331

Download citation

Received : 10 August 2008

Accepted : 03 November 2008

Published : 20 January 2009

Issue Date : February 2009

DOI : https://doi.org/10.1038/ncpgasthep1331

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Emergency physicians’ perceptions of critical appraisal skills: a qualitative study.

• Sumintra Wood
• Jacqueline Paulis
• Angela Chen

BMC Medical Education (2022)

An integrative review on individual determinants of enrolment in National Health Insurance Scheme among older adults in Ghana

• Anthony Kwame Morgan
• Anthony Acquah Mensah

BMC Primary Care (2022)

Autopsy findings of COVID-19 in children: a systematic review and meta-analysis

• Anju Khairwa
• Kana Ram Jat

Forensic Science, Medicine and Pathology (2022)

The use of a modified Delphi technique to develop a critical appraisal tool for clinical pharmacokinetic studies

• Alaa Bahaa Eldeen Soliman
• Shane Ashley Pawluk
• Ousama Rachid

International Journal of Clinical Pharmacy (2022)

Critical Appraisal: Analysis of a Prospective Comparative Study Published in IJS

• Ramakrishna Ramakrishna HK
• Swarnalatha MC

Indian Journal of Surgery (2021)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game New
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications
• Critical Reviews

How to Write an Article Review

Last Updated: September 8, 2023 Fact Checked

This article was co-authored by Jake Adams . Jake Adams is an academic tutor and the owner of Simplifi EDU, a Santa Monica, California based online tutoring business offering learning resources and online tutors for academic subjects K-College, SAT & ACT prep, and college admissions applications. With over 14 years of professional tutoring experience, Jake is dedicated to providing his clients the very best online tutoring experience and access to a network of excellent undergraduate and graduate-level tutors from top colleges all over the nation. Jake holds a BS in International Business and Marketing from Pepperdine University. There are 13 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 3,084,548 times.

An article review is both a summary and an evaluation of another writer's article. Teachers often assign article reviews to introduce students to the work of experts in the field. Experts also are often asked to review the work of other professionals. Understanding the main points and arguments of the article is essential for an accurate summation. Logical evaluation of the article's main theme, supporting arguments, and implications for further research is an important element of a review . Here are a few guidelines for writing an article review.

Education specialist Alexander Peterman recommends: "In the case of a review, your objective should be to reflect on the effectiveness of what has already been written, rather than writing to inform your audience about a subject."

Things You Should Know

• Read the article very closely, and then take time to reflect on your evaluation. Consider whether the article effectively achieves what it set out to.
• Write out a full article review by completing your intro, summary, evaluation, and conclusion. Don't forget to add a title, too!
• Proofread your review for mistakes (like grammar and usage), while also cutting down on needless information. [1] X Research source

Preparing to Write Your Review

• Article reviews present more than just an opinion. You will engage with the text to create a response to the scholarly writer's ideas. You will respond to and use ideas, theories, and research from your studies. Your critique of the article will be based on proof and your own thoughtful reasoning.
• An article review only responds to the author's research. It typically does not provide any new research. However, if you are correcting misleading or otherwise incorrect points, some new data may be presented.
• An article review both summarizes and evaluates the article.

• Summarize the article. Focus on the important points, claims, and information.
• Discuss the positive aspects of the article. Think about what the author does well, good points she makes, and insightful observations.
• Identify contradictions, gaps, and inconsistencies in the text. Determine if there is enough data or research included to support the author's claims. Find any unanswered questions left in the article.

• Make note of words or issues you don't understand and questions you have.
• Look up terms or concepts you are unfamiliar with, so you can fully understand the article. Read about concepts in-depth to make sure you understand their full context.

• Pay careful attention to the meaning of the article. Make sure you fully understand the article. The only way to write a good article review is to understand the article.

• With either method, make an outline of the main points made in the article and the supporting research or arguments. It is strictly a restatement of the main points of the article and does not include your opinions.
• After putting the article in your own words, decide which parts of the article you want to discuss in your review. You can focus on the theoretical approach, the content, the presentation or interpretation of evidence, or the style. You will always discuss the main issues of the article, but you can sometimes also focus on certain aspects. This comes in handy if you want to focus the review towards the content of a course.
• Review the summary outline to eliminate unnecessary items. Erase or cross out the less important arguments or supplemental information. Your revised summary can serve as the basis for the summary you provide at the beginning of your review.

• What does the article set out to do?
• What is the theoretical framework or assumptions?
• Are the central concepts clearly defined?
• How adequate is the evidence?
• How does the article fit into the literature and field?
• Does it advance the knowledge of the subject?
• How clear is the author's writing? Don't: include superficial opinions or your personal reaction. Do: pay attention to your biases, so you can overcome them.

Writing the Article Review

• For example, in MLA , a citation may look like: Duvall, John N. "The (Super)Marketplace of Images: Television as Unmediated Mediation in DeLillo's White Noise ." Arizona Quarterly 50.3 (1994): 127-53. Print. [10] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

• For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest.

• Your introduction should only be 10-25% of your review.
• End the introduction with your thesis. Your thesis should address the above issues. For example: Although the author has some good points, his article is biased and contains some misinterpretation of data from others’ analysis of the effectiveness of the condom.

• Use direct quotes from the author sparingly.
• Review the summary you have written. Read over your summary many times to ensure that your words are an accurate description of the author's article.

• Support your critique with evidence from the article or other texts.
• The summary portion is very important for your critique. You must make the author's argument clear in the summary section for your evaluation to make sense.
• Remember, this is not where you say if you liked the article or not. You are assessing the significance and relevance of the article.
• Use a topic sentence and supportive arguments for each opinion. For example, you might address a particular strength in the first sentence of the opinion section, followed by several sentences elaborating on the significance of the point.

• This should only be about 10% of your overall essay.
• For example: This critical review has evaluated the article "Condom use will increase the spread of AIDS" by Anthony Zimmerman. The arguments in the article show the presence of bias, prejudice, argumentative writing without supporting details, and misinformation. These points weaken the author’s arguments and reduce his credibility.

• Make sure you have identified and discussed the 3-4 key issues in the article.

Sample Article Reviews

Expert Q&A

You Might Also Like

• ↑ https://writing.wisc.edu/handbook/grammarpunct/proofreading/
• ↑ https://libguides.cmich.edu/writinghelp/articlereview
• ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/
• ↑ Jake Adams. Academic Tutor & Test Prep Specialist. Expert Interview. 24 July 2020.
• ↑ https://guides.library.queensu.ca/introduction-research/writing/critical
• ↑ https://www.iup.edu/writingcenter/writing-resources/organization-and-structure/creating-an-outline.html
• ↑ https://writing.umn.edu/sws/assets/pdf/quicktips/titles.pdf
• ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_periodicals.html
• ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548565/
• ↑ https://writingcenter.uconn.edu/wp-content/uploads/sites/593/2014/06/How_to_Summarize_a_Research_Article1.pdf
• ↑ https://www.uis.edu/learning-hub/writing-resources/handouts/learning-hub/how-to-review-a-journal-article
• ↑ https://writingcenter.unc.edu/tips-and-tools/editing-and-proofreading/

About This Article

If you have to write an article review, read through the original article closely, taking notes and highlighting important sections as you read. Next, rewrite the article in your own words, either in a long paragraph or as an outline. Open your article review by citing the article, then write an introduction which states the article’s thesis. Next, summarize the article, followed by your opinion about whether the article was clear, thorough, and useful. Finish with a paragraph that summarizes the main points of the article and your opinions. To learn more about what to include in your personal critique of the article, keep reading the article! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Prince Asiedu-Gyan

Apr 22, 2022

Did this article help you?

Sammy James

Sep 12, 2017

Juabin Matey

Aug 30, 2017

Oct 25, 2023

Vanita Meghrajani

Jul 21, 2016

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

Get all the best how-tos!

Sign up for wikiHow's weekly email newsletter

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved March 25, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Turk J Urol
• v.39(Suppl 1); 2013 Sep

How to write a review article?

In the medical sciences, the importance of review articles is rising. When clinicians want to update their knowledge and generate guidelines about a topic, they frequently use reviews as a starting point. The value of a review is associated with what has been done, what has been found and how these findings are presented. Before asking ‘how,’ the question of ‘why’ is more important when starting to write a review. The main and fundamental purpose of writing a review is to create a readable synthesis of the best resources available in the literature for an important research question or a current area of research. Although the idea of writing a review is attractive, it is important to spend time identifying the important questions. Good review methods are critical because they provide an unbiased point of view for the reader regarding the current literature. There is a consensus that a review should be written in a systematic fashion, a notion that is usually followed. In a systematic review with a focused question, the research methods must be clearly described. A ‘methodological filter’ is the best method for identifying the best working style for a research question, and this method reduces the workload when surveying the literature. An essential part of the review process is differentiating good research from bad and leaning on the results of the better studies. The ideal way to synthesize studies is to perform a meta-analysis. In conclusion, when writing a review, it is best to clearly focus on fixed ideas, to use a procedural and critical approach to the literature and to express your findings in an attractive way.

The importance of review articles in health sciences is increasing day by day. Clinicians frequently benefit from review articles to update their knowledge in their field of specialization, and use these articles as a starting point for formulating guidelines. [ 1 , 2 ] The institutions which provide financial support for further investigations resort to these reviews to reveal the need for these researches. [ 3 ] As is the case with all other researches, the value of a review article is related to what is achieved, what is found, and the way of communicating this information. A few studies have evaluated the quality of review articles. Murlow evaluated 50 review articles published in 1985, and 1986, and revealed that none of them had complied with clear-cut scientific criteria. [ 4 ] In 1996 an international group that analyzed articles, demonstrated the aspects of review articles, and meta-analyses that had not complied with scientific criteria, and elaborated QUOROM (QUality Of Reporting Of Meta-analyses) statement which focused on meta-analyses of randomized controlled studies. [ 5 ] Later on this guideline was updated, and named as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). [ 6 ]

Review articles are divided into 2 categories as narrative, and systematic reviews. Narrative reviews are written in an easily readable format, and allow consideration of the subject matter within a large spectrum. However in a systematic review, a very detailed, and comprehensive literature surveying is performed on the selected topic. [ 7 , 8 ] Since it is a result of a more detailed literature surveying with relatively lesser involvement of author’s bias, systematic reviews are considered as gold standard articles. Systematic reviews can be diivded into qualitative, and quantitative reviews. In both of them detailed literature surveying is performed. However in quantitative reviews, study data are collected, and statistically evaluated (ie. meta-analysis). [ 8 ]

Before inquring for the method of preparation of a review article, it is more logical to investigate the motivation behind writing the review article in question. The fundamental rationale of writing a review article is to make a readable synthesis of the best literature sources on an important research inquiry or a topic. This simple definition of a review article contains the following key elements:

• The question(s) to be dealt with
• Methods used to find out, and select the best quality researches so as to respond to these questions.
• To synthetize available, but quite different researches

For the specification of important questions to be answered, number of literature references to be consulted should be more or less determined. Discussions should be conducted with colleagues in the same area of interest, and time should be reserved for the solution of the problem(s). Though starting to write the review article promptly seems to be very alluring, the time you spend for the determination of important issues won’t be a waste of time. [ 9 ]

The PRISMA statement [ 6 ] elaborated to write a well-designed review articles contains a 27-item checklist ( Table 1 ). It will be reasonable to fulfill the requirements of these items during preparation of a review article or a meta-analysis. Thus preparation of a comprehensible article with a high-quality scientific content can be feasible.

PRISMA statement: A 27-item checklist

Contents and format

Important differences exist between systematic, and non-systematic reviews which especially arise from methodologies used in the description of the literature sources. A non-systematic review means use of articles collected for years with the recommendations of your colleagues, while systematic review is based on struggles to search for, and find the best possible researches which will respond to the questions predetermined at the start of the review.

Though a consensus has been reached about the systematic design of the review articles, studies revealed that most of them had not been written in a systematic format. McAlister et al. analyzed review articles in 6 medical journals, and disclosed that in less than one fourth of the review articles, methods of description, evaluation or synthesis of evidence had been provided, one third of them had focused on a clinical topic, and only half of them had provided quantitative data about the extend of the potential benefits. [ 10 ]

Use of proper methodologies in review articles is important in that readers assume an objective attitude towards updated information. We can confront two problems while we are using data from researches in order to answer certain questions. Firstly, we can be prejudiced during selection of research articles or these articles might be biased. To minimize this risk, methodologies used in our reviews should allow us to define, and use researches with minimal degree of bias. The second problem is that, most of the researches have been performed with small sample sizes. In statistical methods in meta-analyses, available researches are combined to increase the statistical power of the study. The problematic aspect of a non-systematic review is that our tendency to give biased responses to the questions, in other words we apt to select the studies with known or favourite results, rather than the best quality investigations among them.

As is the case with many research articles, general format of a systematic review on a single subject includes sections of Introduction, Methods, Results, and Discussion ( Table 2 ).

Structure of a systematic review

Preparation of the review article

Steps, and targets of constructing a good review article are listed in Table 3 . To write a good review article the items in Table 3 should be implemented step by step. [ 11 – 13 ]

Steps of a systematic review

The research question

It might be helpful to divide the research question into components. The most prevalently used format for questions related to the treatment is PICO (P - Patient, Problem or Population; I-Intervention; C-appropriate Comparisons, and O-Outcome measures) procedure. For example In female patients (P) with stress urinary incontinence, comparisons (C) between transobturator, and retropubic midurethral tension-free band surgery (I) as for patients’ satisfaction (O).

Finding Studies

In a systematic review on a focused question, methods of investigation used should be clearly specified.

Ideally, research methods, investigated databases, and key words should be described in the final report. Different databases are used dependent on the topic analyzed. In most of the clinical topics, Medline should be surveyed. However searching through Embase and CINAHL can be also appropriate.

While determining appropriate terms for surveying, PICO elements of the issue to be sought may guide the process. Since in general we are interested in more than one outcome, P, and I can be key elements. In this case we should think about synonyms of P, and I elements, and combine them with a conjunction AND.

One method which might alleviate the workload of surveying process is “methodological filter” which aims to find the best investigation method for each research question. A good example of this method can be found in PubMed interface of Medline. The Clinical Queries tool offers empirically developed filters for five different inquiries as guidelines for etiology, diagnosis, treatment, prognosis or clinical prediction.

Evaluation of the Quality of the Study

As an indispensable component of the review process is to discriminate good, and bad quality researches from each other, and the outcomes should be based on better qualified researches, as far as possible. To achieve this goal you should know the best possible evidence for each type of question The first component of the quality is its general planning/design of the study. General planning/design of a cohort study, a case series or normal study demonstrates variations.

A hierarchy of evidence for different research questions is presented in Table 4 . However this hierarchy is only a first step. After you find good quality research articles, you won’t need to read all the rest of other articles which saves you tons of time. [ 14 ]

Determination of levels of evidence based on the type of the research question

Formulating a Synthesis

Rarely all researches arrive at the same conclusion. In this case a solution should be found. However it is risky to make a decision based on the votes of absolute majority. Indeed, a well-performed large scale study, and a weakly designed one are weighed on the same scale. Therefore, ideally a meta-analysis should be performed to solve apparent differences. Ideally, first of all, one should be focused on the largest, and higher quality study, then other studies should be compared with this basic study.

Conclusions

In conclusion, during writing process of a review article, the procedures to be achieved can be indicated as follows: 1) Get rid of fixed ideas, and obsessions from your head, and view the subject from a large perspective. 2) Research articles in the literature should be approached with a methodological, and critical attitude and 3) finally data should be explained in an attractive way.

• Open access
• Published: 28 March 2024

Racial/ethnic differences in the association between transgender-related U.S. state policies and self-rated health of transgender women

• Wesley M. King 1   na1 ,
• Kristi E. Gamarel 1 ,
• Nancy L. Fleischer 2 ,
• Asa E. Radix 3 ,
• Tonia C. Poteat 4 ,
• Linda M. Chatters 1 , 5 ,
• Don Operario 6 ,
• Sari L. Reisner 2 , 7   na1 ,
• Andrea L. Wirtz 8 &

American Cohort to Study HIV Acquisition Among Transgender Women (LITE) Study Group

BMC Public Health volume  24 , Article number:  911 ( 2024 ) Cite this article

98 Accesses

Metrics details

Policy protections for transgender adults in the United States are consistently associated with positive health outcomes. However, studies over-represent non-Latinx White transgender people and obscure variation in policies’ intended goals. This study examined racial differences in the relationship between transgender-related policies and transgender women’s self-rated health. Guided by Critical Race Theory, we hypothesized that policies conferring access to resources (e.g., healthcare) would be associated with better self-rated health among all participants while policies signifying equality (e.g., nondiscrimination laws) would be associated with better self-rated health only for White participants.

Using cross-sectional data collected between March 2018-December 2020 from 1566 transgender women, we analyzed 7 state-level ‘access policies,’ 5 ‘equality policies,’ and sum indices of each. Participants represented 29 states, and 54.7% were categorized as people of color. We fit a series of multilevel ordinal regression models predicting self-rated health by each policy. Multivariate models were adjusted for relevant covariates at the individual- and state-level. We then tested moderation by race/ethnicity using interaction terms and generated stratified predicted probability plots.

In bivariate models, 4 access policies, 2 equality policies, and both indices were associated with better self-rated health, but associations did not persist in adjusted models. Results from the multivariable models including interaction terms indicated that policies concerning private insurance coverage of gender-affirming care, private insurance nondiscrimination, credit nondiscrimination, and both indices were statistically significantly associated with better self-rated health for White participants and worse self-rated health for participants of color.

Conclusions

The policies included in this analysis do not mitigate racism’s effects on access to resources, indicating they may be less impactful for transgender women of color than White transgender women. Future research and policy advocacy efforts promoting transgender women’s health must center racial equity as well as transgender people of color’s priorities.

Peer Review reports

In the United States (U.S.), policies with particular relevance to transgender populations have been in flux over the past decade. Policy surveillance organizations have noted the erosion of state-level protections for transgender populations [ 1 , 2 , 3 ], a trend with direct implications for transgender population health. State-level protective policies, such as nondiscrimination laws and requirements that health insurance cover medical gender affirmation, are consistently associated with better health among transgender populations [ 4 , 5 , 6 , 7 ]. In contrast, exclusionary policies, such as those that allow healthcare providers to deny care to transgender patients on religious grounds, have been linked to adverse health outcomes such as non-prescribed hormone use, healthcare avoidance, violent victimization, suicidality, and emotional and physical distress [ 8 , 9 , 10 , 11 ]. Further, studies suggest that public debates surrounding adoption of transgender-related policies may heighten depression, anxiety, and PTSD symptoms among transgender populations [ 1 , 2 , 12 ].

Little research has examined differences in the relationship between policies and health among subgroups of transgender people. In particular, the health effects of transgender-related policies on transgender people of color is largely unknown as many studies examining the health effects of transgender-related policies had samples that were upwards of 80% non-Latinx White (hereafter, White) [ 1 , 5 , 6 , 8 ]. Accumulating evidence suggests racial health inequities within transgender populations. Transgender people of color, compared to their White counterparts, have poorer HIV prevention and HIV care continua outcomes [ 13 ], worse self-rated health [ 14 ], more adverse mental health symptoms [ 14 , 15 ], lower access to healthcare [ 16 , 17 ], higher burden of chronic diseases [ 18 ], and higher mortality risk [ 19 ]. Thus, understanding which populations benefit from transgender-related policies is crucial to understanding structural solutions for health equity within transgender populations.

Theoretical framework

This study draws from Critical Race Theory’s critique of liberalism to explore racial differences in the relationship between transgender-related state policies and health among transgender women. This critique holds that policies based in race-blind neoliberal frameworks of inclusion and rights expansion primarily benefit populations that are the least vulnerable to the harms of racism and intersectional oppression [ 20 , 21 ]. Many transgender-related policies are based on the liberal ideal of equal opportunity and may exacerbate social and health inequities [ 22 , 23 ]. For example, the 2020 Bostock v. Clayton County, Georgia decision resulted in transgender inclusion in employment nondiscrimination policies nationwide [ 24 ]. Yet, anti-transgender employment discrimination remains highly prevalent despite transgender-inclusive nondiscrimination policies, and the actual enforcement of these laws does not provide redress for most transgender people who experience workplace discrimination [ 23 , 25 , 26 ]. Employment nondiscrimination laws require complainants to prove employers’ discriminatory intent, a task difficult for those who have financial access to appropriate legal counsel and effectively impossible for those who do not [ 23 ]. Given documented economic inequities between White transgender people and transgender people of color, and specifically Black and Latina transgender women [ 27 , 28 , 29 ], Critical Race Theory’s critique of liberalism would therefore suggest that transgender-inclusive employment nondiscrimination laws are more likely to benefit White transgender women than transgender women of color.

Furthermore, employment discrimination, hate crime laws, and other currently debated transgender-related policies do not address structural vulnerability among transgender people [ 23 ]. Structural vulnerability refers to a depreciated social position created through discrimination and economic exploitation and marked by social, economic, and material hardships (e.g., poverty, violence) [ 30 , 31 ]. Large-scale national studies indicate that employment inequities impacting transgender adults have increased despite the expansion of nondiscrimination policies [ 32 , 33 ]. Similarly, growing lists of states have added gender identity as a protected class in hate crime laws and eliminated “trans panic” defenses (i.e., defendants’ use of discovery of a transgender person’s gender as exculpatory or mitigating evidence) in criminal proceedings; however, annual accounts of fatal violence against transgender women of color continue to grow [ 34 ].

Responding to the noted inadequacies of liberal reforms based on ideals of inclusion and equality, transgender activists have proposed policy agendas focused on building coalitions across axes of oppression to transform or abolish the legal and administrative systems that directly control the lives of the most marginalized transgender people: prisons, welfare programs, job training centers, foster care, housing authorities, and healthcare [ 23 ]. In particular, transgender women of color have identified potentially effective focal points for transgender-related policies. These include equitable access to public and private housing; cultural and structural competence in education, employment, and healthcare settings; and programs that promote safety and recovery from interpersonal violence and other traumas [ 35 , 36 , 37 , 38 ]. These issues align with several existing transgender-related policies, namely those that govern insurance coverage for gender-affirming medical care, institutional sex segregation (e.g., in domestic violence programs), and identity document changes [ 23 ]. In this study, we refer to these policies as access policies because they have direct implications for transgender people’s access to resources critical for wellbeing. We use the term equality policies to refer to policies that signal recognition of transgender people within the existing neoliberal order but without altering their lived experiences, such as nondiscrimination policies and hate crime laws.

Current study

This study seeks to examine the relationship between access and equality policies and self-rated health among transgender women. Due to the lack of population-level data that adequately captures gender identity [ 39 ], we pursued our research aims using a large convenience sample of transgender women. We expect that access policies, which have direct implications for transgender women’s material conditions and social experiences, are more consistently associated with better self-rated health than symbolic equality [ 23 ]. Additionally, given evidence suggesting that transgender women of color are more structurally vulnerable than White transgender women due to their positionality at the intersection of racism, cisgenderism, and misogyny [ 29 , 35 , 37 , 40 , 41 ], we hypothesize that race will moderate the relationship between policies and self-rated health. More specifically, we expect that in comparison to White transgender women, access policies will be more strongly associated with transgender women of color’s health status while equality policies will be less strongly associated [ 23 ]. Finally, we anticipate that any observed relationships between policies and self-rated health will persist when controlling for structural vulnerability and individual- and state-level demographics.

Study design

Data for this analysis were collected through the Leading Innovation for Transgender Women’s Health and Empowerment (LITE) study. Between March 2018 and October 2020, 1,614 transgender women were enrolled in either a 2-year prospective cohort study designed to characterize HIV incidence and risk factors for HIV acquisition or a cross-sectional comparison group of transgender women living with HIV [ 42 ]. LITE initially enrolled participants at six physical study sites in Boston, MA; New York, NY; Baltimore, MD; Washington, DC; Atlanta, GA; and Miami, FL. Beginning in June 2018, participants living in Eastern and Southern U.S. cities could enroll online. Eligibility criteria for participation in the baseline survey included being at least 18 years-old, speaking English or Spanish, identifying as a woman or with a feminine gender identity, and being assigned male sex at birth [ 42 , 43 ]. Data for this cross-sectional analysis comes from the baseline survey of all participants. We made this choice to include participants living with HIV (who were not enrolled into the cohort study) and because follow-up data collection was ongoing at the time of analysis. Individuals were included in this analysis if they provided a valid U.S. zip code of their residence and data on self-rated health, race, and ethnicity, resulting in an analytic sample of 1,566 participants. Study procedures were approved by the Johns Hopkins School of Medicine single Institutional Review Board.

Participant level

Primary outcome: self-rated health.

Self-rated health was assessed with a single item asking whether participants considered their health to be excellent, very good, good, fair, or poor. We combined poor and fair health to account for skewed data, resulting in a 4-point scale which higher numbers indicated better self-rated health. Self-rated health was selected as a study outcome because it is a robust predictor of morbidity and mortality at the population level [ 44 , 45 ].

Demographics

Participants self-reported their race, ethnicity, age, citizenship, and whether they immigrated to the U.S. For this analysis, participants were considered people of color if they selected any race other than or in combination with White or indicated that they were Hispanic/Latina. Participants also reported the zip code of where they currently live, which was used to assign their state of residence and calculate local population density in number of people per square mile using Zip Code Tabulation Area data from the 2016-2020 American Community Survey (ACS) [ 46 ].

Structural vulnerability indicators

Fifteen dichotomous indicators were selected to reflect Bourgois et al.’s (2017) eight domains of structural vulnerability: financial insecurity, residence, risk environments, food access, social network, legal status, education, and discrimination. The structural vulnerability framework conceptualizes such indicators as the individual-level consequences of a structurally subordinated positionality [ 30 ].

Financial insecurity

Participants were asked to indicate their current sources of income or financial support. Those who did not report having a traditional job (either full-time or part-time) were considered unemployed. Additionally, we created a variable indicating reliance on precarious sources of income. Any participants who were unemployed and reported receiving income from unregulated or criminalized forms of employment (e.g., sex work, ‘under the table’ jobs, drug sales) were considered to have informal employment. Finally, participants reported their total income over the past 30 days, which was dichotomized at $1,000 or less, which approximates the federal poverty level for an individual during the study period [ 47 ].

Participants were considered to have unstable housing if they reported currently living anywhere other than housing they owned or rented; this included, for example, living in a homeless or domestic violence shelter, doubling up with friends or family, and living in hotels. Participants also reported the number of days during the last 3 months (site-based participants) or 6 months (online participants) they had difficulty finding a safe place to sleep, which was dichotomized as any vs. none. For this and other time-bounded measures in the survey, the recall window differed between online and site-based participants to reflect planned differences in the length of time between follow-up points in the cohort study due to resource constraints.

Risk environments

In the structural vulnerability framework, risk environments refer to potential for bodily harm, including interpersonal violence [ 30 ]. Participants completed an adapted version of the intimate partner violence scale from the World Health Organization Multi-country Study on Women’s Health and Domestic Violence Against Women; items were modified to ask about violence from all perpetrators [ 48 ]. Emotional abuse was assessed with four items asking whether participants had been insulted, humiliated, intimidated, or threatened to be outed (α=0.91). For example, participants reported whether someone had ever “belittled or humiliated [them] in front of other people” and, if so, whether this had happened in the past 3 months (site-based participants) or past 6 months (online participants). Lifetime and recent physical violence were assessed with six items asking whether participants had been slapped, pushed, punched, kicked, choked, or attacked with a weapon (α=0.96). A sample physical violence item is “Has anyone ever… hit you with a fist or something else that could hurt you?” Finally, lifetime and recent sexual violence were assessed with four items asking whether participants had been physically forced to have sex, had been degraded or humiliated during sex, had unwanted sex out of fear, or had unwanted sex because someone told them it was their right (α=0.95). A sample sexual violence item is “Has anyone ever… had sexual intercourse or did something sexual you did not want to because you were afraid of what they might do?” Each type of interpersonal violence was considered an indicator of risk environment if participants reported any experience within each category within the past 3 months (site-based participants) or past 6 months (online participants).

Food access

Participants were considered food insecure if they reported running out of food or money for food by the end of the month sometimes, most of the time, or almost always [ 49 ]. Additionally, Supplemental Nutrition Assistance Program/Electronic Benefit Transfer (SNAP/EBT) reported as a source of income or support in the past 3 months (site-based participants) or 6 months (online participants) was also considered an indicator of food access.

Social network

Participants completed the 5-item California Health Interview Survey social support measure, which assessed general social support with items such as “Thinking about the last 6 months, how often have you had someone available to understand your problems?” [ 50 ]. Participants rated items on a 4-point scale, which were then summed for an overall score (α=0.91). To create an indicator reflecting Bourgois et al. 2017’s conceptualization of absence of social support as a structural vulnerability, participants with a mean score in the bottom quartile of the sample (7 out of a possible 20) were considered socially isolated [ 30 ].

Legal status

Participants reported whether they had ever been held in prison, jail, juvenile detention, or immigration custody at any point in their lives. Those that had were considered to have been incarcerated. Additionally, participants rated the extent to which their legal forms of identification (e.g., driver’s license) list their name and gender. Those who reported that none of their forms of identification listed either were considered to not have any of their legal gender affirmation needs met. Finally, not having U.S. citizenship was considered a marker of structural vulnerability.

Participants selected one of the following options to report their educational background: did not complete 8 th grade, completed 8 th grade, some high school, completed high school (received a diploma or GED), some college or associate degree, completed college (Bachelor’s degree), technical/vocational school, some graduate school, or completed graduate school. Participants were considered educationally structurally vulnerable if they reported completing less than high school.

Discrimination

Discrimination was assessed with the 9-item Intersectional Discrimination Index: Anticipated Discrimination subscale [ 51 ]. Participants rated items like “I may be denied a bank account, loan, or mortgage because of who I am” on a 4-point scale (α=0.94). To create an indicator reflecting Bourgois et al.’s (2017) conceptualization of discrimination as a form of structural vulnerability, those in the top quartile of the sample (27 out of a possible 36) were considered to have high anticipated discrimination.

Policy data

Policy data were extrapolated from reports published by the Movement Advancement Project and cross-referenced with state legal texts [ 24 ]. Transgender-related state policies were selected based on (1) their applicability to transgender adults, (2) variation across the states represented in the dataset, and (3) their ability to be categorized as access or equality policies. This resulted in seven access policies and five equality policies. Each of the access policies govern transgender adults’ ability to receive either medical care or legal gender affirmation. Each of the equality policies reflect transgender people’s inclusion in nondiscrimination or criminal justice laws. For ease of interpretation, all policies were coded dichotomously such that 1 represents the theoretically most favorable policy environment for transgender people and 0 represents all other environments (Table 1 ). States in which legal authorities (e.g., human rights commissions, state supreme courts) have interpreted nondiscrimination laws covering sexual orientation and/or sex to include gender identity were coded as ‘1’. Additionally, we created composite measures totaling all access policies (‘access policy index’, α=0.86) and all equality policies (‘equality policy index’, α=0.86). The average distribution of these measures across the study period is presented in Fig. 1 with darker colors indicating higher average scores.

Average distribution of access policies and equality policies among states represented in the LITE Cohort, March 2018-October 2020

State-level covariates

State-level covariates were selected based on their potential confounding influence on self-rated health. The proportion of the state population identifying as Black, the proportion identifying as Latinx, the percent of each state’s population with a bachelor’s degree, and state’s household income inequality was derived from 2016-2020 ACS data [ 46 ]. States’ overall unemployment rate for 2019 was taken from Bureau of Labor Statistics data [ 52 ].

Analysis began by calculating descriptive statistics for all variables of interest to categorize the sample by individual-level demographics, structural vulnerability indicators, and self-rated health. ANOVA and chi-square tests were used to determine if there were significant differences between White participants and participants of color across these variables.

Multilevel modeling

Subsequent analyses used multilevel models to account for participant clustering by state. Participants were assigned to 46 state-time clusters based on their state of residence and the time at which they completed the survey. This clustering accounted for changes in laws that occurred during the period over which participants completed their baseline surveys. For example, there were 14 participants from Connecticut, 10 of whom completed the survey prior to Connecticut updating requirements for gender marker changes on driver’s licenses, and 11 of whom did so prior to Connecticut’s prohibition of transgender panic defenses. Therefore, participants from Connecticut were assigned to three different state-time clusters.

Operationalizing structural vulnerability

Multilevel latent class analysis (LCA) was performed in MPlus 8.8 using all structural vulnerability indicators. We chose to use LCA rather than treat each indicator as an independent covariate as evidence suggests underlying patterns of structural vulnerability are better predictors of health outcomes than single indicators among transgender women [ 53 ]. Aligned with this analytic choice, we dichotomized all indicators of structural vulnerability as described above. Beginning with a single-class model, models with up to 5 classes were evaluated using Bayesian information criterion (BIC), sample-size adjusted BIC, and Lo-Mendell-Rubin likelihood ratio tests [ 54 ]. All models adjusted for whether participants completed study procedures online or at a study site. Participants were assigned to latent classes based on their highest posterior predicted probability of class membership.

Regression models

We fit multilevel ordinal regression models in Stata 17.0 to test the association between policy variables and self-rated health. First, each policy variable was entered as the only independent variable in models accounting for clustering at the state-time level. Brant tests indicated that none of the policy variables violated the parallel regression assumption. Then, we fit adjusted models including person-level covariates (age, race, local population density, online vs. site-based participation, immigration history, and structural vulnerability class membership) and state-level covariates (racial demographics, proportion of adults with a bachelor’s degree, income inequality, and unemployment rate). Adjusted models used complete case analysis as missingness was less than 1% across all variables. Finally, we added policy x person of color interactions terms to each model to determine if relationships between policy and self-rated health were consistent across White participants and participants of color.

Sample characteristics

Structural vulnerability indicators were highly prevalent in the sample. For example, 36.9% of participants reported housing instability and 22.2% had a history of incarceration ( Appendix 1). In LCA, a two-class model best fit the data ( Appendix 2). The predicted probabilities of most structural vulnerability indicators were substantially higher in Class 2 than Class 1. For example, the predicted probability of being currently unemployed was 0.748 in Class 2 compared to 0.239 in Class 1, and the predicted probability of food insecurity was 0.364 in Class 2 compared to 0.065 in Class 1. Therefore, Class 2 was labeled “High Vulnerability” and Class 1 was labeled “Low Vulnerability.” Slightly over half of participants were assigned to the Low Vulnerability class (56.1%).

Over half of the analytic sample was comprised of participants of color (54.7%, n =856). Among participants of color, 38.1% ( n =326) were Black, 36.5% ( n =312) were Latina (any race), 4.1% ( n =35) were Asian/Pacific Islander, 0.6% ( n =5) were American Indian/Alaskan Native, and 20.8% ( n =178) were multiracial or reported another race. There were statistically significant differences between participants of color and White participants across most individual-level variables such that participants of color were significantly older, more likely to have immigrated to the U.S., participate at a study site vs. online, live in more densely populated areas, and be assigned to the High Vulnerability Class than White participants (Table 2 ). Furthermore, the distribution of participants across self-rated health categories differed across race/ethnicity ( p <0.001); for example, 23.3% of participants of color described their health as “excellent” compared to 12.5% of White participants.

Transgender-related policies and self-rated health

In unadjusted models, several access and equality policies and both policy indices were associated with better self-rated health (Table 3 ). The associated access policies included requirements that private insurers cover gender-affirming care (OR=1.39, 95% CI: 1.04-1.85), Medicaid coverage of gender-affirming care (OR=1.34, 95% CI: 1.02-1.75), name change requirements not being dependent on applicants’ criminal records (OR=1.56, 95% CI: 1.24-1.97), and accessible birth certificate gender marker change requirements (OR=1.36, 95% CI: 1.01-1.83). Equality policies associated with better self-rated health included gender identity protections in credit nondiscrimination law (OR=1.38, 95% CI: 1.02-1.86) and hate crime law (OR=1.58, 95% CI: 1.19-2.10). Each additional access policy was associated with a 10% increase in the odds of being in the next highest self-rated health category (95% CI: 1.03-1.16 and 1.02-1.18) and each additional equality policy was associated with a 9% increase in these odds (95% CI: 1.01-1.72). Additionally, participants of color had a higher odds of better self-rated health category compared to White participants (95% CI: 1.14-1.72). None of these associations persisted when adjusting for individual- and state-level covariates.

When including policy x person of color interaction terms, requirements that private insurers cover gender-affirming healthcare (OR=1.38, 95% CI: 1.01-1.88) were associated with better self-rated health (Table 4 ). In this model, the interaction term indicated a statistically weaker association with self-rated health for participants of color than White participants (OR=0.64, 95% CI: 0.43-0.95). Additionally, the interaction between policy and race/ethnicity on self-rated health was statistically significant for prohibitions on private insurers denying coverage on the basis of gender identity (OR=0.51, 95% CI: 0.35-0.76) and credit nondiscrimination laws (OR=0.59, 95% CI: 0.40-0.88), the access policy index (OR=0.91, 95% CI: 0.83-<1.00), and the equality policy index (OR: 0.89, 95% CI: 0.80-<1.00).

We include predicted probability plots to aid interpretations of these results. For both the Access Policy Index (Fig. 2 ) and the Equality Policy Index (Fig. 3 ), the probability of reporting ‘very good’ and ‘excellent’ health increases as each index increases for White transgender women but decreases or remains constant for transgender women of color. As follows, the probability of reporting ‘fair/poor’ and ‘good’ health decreases as each index increases for White transgender women but increases or remains constant for transgender women of color. A similar trend appears for each of the individual policies with significant interaction terms (Fig. 4 ).

Predicted probability of self-rated general health categories by access policy index and race/ethnicity

Predicted probability of self-rated general health categories by equality policy index and race/ethnicity

Distribution of predicted probabilities of self-rated general health categories by select transgender-related state policies and race/ethnicity

This analysis of adult transgender women in the U.S. found that several transgender-related state policies governing access to resources and equality under the law were associated with better self-rated health in bivariable regression models. When adjusting for individual- and state-level covariates including structural vulnerability, these associations did not persist. However, adding policy x person of color interaction terms to the adjusted models revealed that race/ethnicity moderated the relationship between several transgender-related state policies and self-rated health such that the relationships were positive for White transgender women and negative for transgender women of color. Contrary to our hypotheses, this finding was true for both access and equality policies. Consistent with Critical Race Theory’s critique of liberalism [ 20 , 21 ], these findings suggest that transgender-related state policies may have a protective effect on self-rated health only for White transgender women.

All policies included in this analysis are nonracial in that they do not contain language about race or racism [ 20 ]. This colorblindness disregards how violence, discrimination, and access to resources are qualitatively different for transgender women of color than White transgender women due to the centrality of racism in the political, economic, and cultural structures in the U.S. [ 20 , 55 , 56 ]. For example, we found that the relationships between two policies regarding private health insurance and self-rated health was positive for White transgender women and negative for transgender women of color. This may be because White adults are more likely to have private insurance than Black or Latinx adults due to greater access to employer subsidies and greater ability to afford purchased insurance [ 57 ]. Transgender-related policies regarding private insurance coverage are therefore potentially more relevant to White transgender women’s access to healthcare.

Our distinction between access and equality policies was based on prior literature highlighting how nondiscrimination and hate crime laws strengthen the carceral state and fail to redistribute resources from those in power to structurally vulnerable transgender people [ 22 , 23 , 58 ]. We hypothesized that access policies would be more strongly associated with better self-rated health for transgender women of color than White transgender women because these policies aim to eliminate barriers to important social determinants of health for “all” transgender people, agnostic to racial differences: medical and legal gender affirmation and health insurance [ 59 ]. However, the access policies included in this study may be ineffective for addressing barriers specific to transgender women of color such as structural, institutional, and interpersonal gendered racism in healthcare settings, schools, and the criminal-legal system that may influence their health status and drive racial health inequities within transgender populations [ 37 , 41 , 60 , 61 ].

Policies that structure the distribution of and access to social, economic, and political resources for all people of color may be more relevant to the lives of transgender women of color than the transgender-related ‘access policies’ we analyzed in this study [ 62 ]. Structural-racism related policies may have unique impacts on transgender women of color as intersectional racism, cisgenderism, and misogyny impact how they are enforced. For example, compared to cisgender, heterosexual people, transgender women are disproportionately subject to police contact, harassment, and arrest for “walking while transgender” under the pretext of enforcing solicitation laws, and police hyper-surveillance of low-income communities of color compounds this risk [ 63 ]. Critical legal scholars have described how solicitation laws and other policies used to justify ‘quality-of-life’ or ‘broken windows’ policing function to intimidate, control, and financially exploit people of color with intersecting marginalized identities, including transgender women of color [ 64 , 65 , 66 , 67 ]. These include laws criminalizing behaviors deemed signifiers of disorder or immorality under hegemonic White supremacy and cisheteropatriarchy (e.g., loitering, vagrancy), laws criminalizing engagement in survival economies (e.g., sex work), laws governing law enforcement conduct (e.g., stop-and-frisk, racial and ethnic profiling), and laws structuring the legal systems through which those charged with de minimis offenses (i.e., those typically punished via fines and/or short incarceration periods) are sentenced [ 62 , 67 , 68 , 69 , 70 , 71 ]. How the full scope of these laws are enacted and enforced against transgender women of color remains poorly documented and understudied [ 67 ]. Further research is needed on how these laws influence population level health outcomes [ 72 ].

Overall, our results indicate that both access and equality policies may be more health-promoting for the self-rated health of White transgender women as compared to transgender women of color. These findings suggest that existing transgender-specific policies may create paths for less marginalized transgender women (e.g., White) to navigate existing oppressive structures such as healthcare, health insurance, and credit systems [ 23 ]. Policies that effectively promote justice and liberation for all transgender people will need to reorder, disrupt, or dismantle these systems to effectively redistribute resources vital to structurally vulnerable transgender people’s wellbeing [ 23 ]. For example, many of the policy demands in the Trans Agenda for Liberation concern abolition of the criminal-legal system in ways that would increase transgender people of color’s—specifically Black transgender women’s—access to employment, housing, and other economic resources and decrease their exposure to interpersonal violence [ 38 ]. Such demands include decriminalizing sex work; ending practices such as monetary sanctions, cash bond, pretrial detention, and solitary confinement; removing immigration restrictions and eliminating immigrant detention; and redistributing public safety funds from policing to community-based alternatives based in restorative/transformative justice practices [ 38 ]. The results of this study highlight the need for policy research pertaining to transgender health that uses intersectionality frameworks to understand how both transgender-specific and non-transgender specific laws differentially impact health for transgender women of color [ 72 , 73 , 74 ].

Limitations

Findings must be interpreted in light of several limitations. First, our data came from a convenience sample of transgender women participating in a study of HIV incidence, and 21 states were not represented in the data. We chose this data source because national health surveillance systems do not allow for the identification of large enough samples of transgender people of color to adequately power analyses. Consequently, our findings lack generalizability to other geographies and transgender populations (e.g., transgender men). Additionally, although this study is among the first to decompose policy effects on health for White and transgender people of color, we acknowledge the diversity within the latter category which our analyses were not powered to explore.

Another major limitation is our use of self-reported general health as our primary outcome. This measure is an established predictor of many clinical outcomes and mortality and is widely used as outcomes in studies assessing the impact of social determinants of health [ 75 , 76 ]. Because self-reported general health measures reflect a range of potentially underlying health conditions, its use is also appropriate for research examining structural determinants of health. However, no studies have validated use of these measures in trans health research or with trans people of color. Notably, in this sample, participants of color reported better health than White participants, which may reflect established differences in how racial/ethnic groups respond to these survey items [ 77 ]. Research on the burden and structural drivers of more specific health outcomes including cardiovascular disease, metabolic diseases, mental health conditions, and cancer among trans populations of color is urgently needed [ 78 ].

Furthermore, while our use of LCA to operationalize and adjust for structural vulnerability is a notable strength of this study, some variables had different recall windows for online (6 months) and site-based (3 months) participants. Site-based participants may therefore have been more likely to be misclassified as Low Vulnerability as their recall windows for the items regarding risk environment, income sources, and difficulty finding a safe place to sleep were three months shorter than online participants. We attempted to mitigate this issue by adjusting for study modality in the LCA and all multivariable models. Additionally, our null findings regarding birth certificate gender marker changes likely reflect that these laws pertain to state of birth rather than current residence; future studies should consider mobility and migration among participants in evaluating this policy. Finally, our cross-sectional study design precludes any conclusions regarding causation, and we did not consider the length of time prior to data collection in which states had enacted these policies. Future research should consider quasi-experimental approaches to evaluating transgender-related policies’ health impact, including the potential mediating role of structural vulnerability or indicators of socioeconomic status.

Policies that promote transgender people’s access to resources and inclusion in existing legal and socioeconomic systems may have differential benefits on the self-rated health of White transgender women compared to transgender women of color. Future evaluations of transgender-related policies must consider the role of race and racism in the function, enforcement, and health impact of these policies [ 62 ]. Transgender health research and political advocacy efforts must extend their focus beyond policies and practices that only implicate transgender identity or gender affirmation and towards those that impact transgender people of color’s material conditions to effectively promote health equity.

Availability of data and materials

The policy data supporting this study are available upon reasonable request from the corresponding author. The participant data supporting this study may be available upon reasonable request via the LITE Study website: https://www.litestudy.org .

Hughto JM, Meyers DJ, Mimiaga MJ, Reisner SL, Cahill S. Uncertainty and confusion regarding transgender non-discrimination policies: implications for the mental health of transgender Americans. Sexual Res Soc Policy. 2022;19:1069-79.

Paceley MS, Dikitsas ZA, Greenwood E, McInroy LB, Fish JN, Williams N, et al. The perceived health implications of policies and rhetoric targeting transgender and gender diverse youth: a community-based qualitative study. Transgender Health. 2023;8(1):100-3.

Movement Advancement Project. Mapping LGBTQ Equality: 2010 to 2020, United States. 2020.

Google Scholar  

McDowell A, Raifman J, Progovac AM, Rose S. Association of nondiscrimination policies with mental health among gender minority individuals. JAMA Psychiatry. 2020;77(9):952–8.

Article   PubMed   Google Scholar  

Goldenberg T, Reisner S, Harper G, Gamarel K, Stephenson R. State policies and healthcare use among transgender people in the U.S. Am J Prev Med. 2020;59(2):247–59.

Article   PubMed   PubMed Central   Google Scholar  

Goldenberg T, Reisner S, Harper G, Gamarel K, Stephenson R. State-level transgender-specific policies, race/ethnicity, and use of medical gender affirmation services among trasngender and other gender-diverse people in the United States. Millbank Quarterly. 2020;98(3):802–46.

Article   Google Scholar  

Du Bois SN, Yoder W, Guy AA, Manser K, Ramos S. Examining associations between state-level transgender policies and transgender health. Transgend Health. 2018;3(1):220–4.

Hughes L, Gamarel K, King W, Goldenberg T, Jaccard J, Geronimus A. State-level policy stigma and non-prescribed hormones use among trans populations in the United States: a mediational analysis of insurance and anticipated stigma. Ann Behav Med. 2022;56(6):592-604.

Ledesma E, Ford CL. Health implications of housing assignments for incarcerated transgender women. Am J Public Health. 2020;110(5):650–4.

Reisner S, Hughto J, Dunham E, Heflin K, Begenyi J, Coffey-Esquivel J, et al. Legal protections in public accomodations settings: a critical public health issue for transgender adn gender-nonconforming people. Milbank Quarterly. 2015;93(3):484–515.

Drakeford L. Correctional policy and attempted suicide among transgender individuals. J Correct Health Care. 2018;24(2):171–82.

Tebbe EA, Simone M, Wilson E, Hunsicker M. A dangerous visibility: moderating effects of antitrans legislative efforts on trans and gender-diverse mental health. Psychology of Sexual Orientation and Gender Diversity. 2022;9(3):259-71.

Becasen JS, Denard CL, Mullins MM, Higa DH, Sipe TA. Estimating the prevalence of HIV and sexual behaviors among the US transgender population: a systematic review and meta-analysis, 2006-2017. Am J Public Health. 2018;109(1):e1-e8.

Lett E, Dowshen N, Baker K. Intersectionality and health inequities for gender minority blacks in the U.S. Am J Prev Med. 2020;59(5):639–47.

Park IY, Speer R, Whitfield DL, Kattari L, Walls EN, Christensen C. Predictors of bullying, depression, and suicide attempts among youth: the intersection of race/ethnicity by gender identity. Child Youth Serv Rev. 2022;139:106536.

Lett E, Abrams M, Gold A, Fullerton F, Everhart A. Ethnoracial inequities in access to gender-affirming mental health care and psychological distress among transgender adults. Soc Psychiatry Psychiatr Epidemiol. 2022;57(5):963–71.

Lett E, Asabor E, Beltran S, Dowshen N. Characterizing health inequities for the U.S. transgender Hispanic population using the behavioral risk factor surveillance system. Transgend Health. 2021;6(5):275–83.

Seelman KL, Young SR, Tesene M, Alvarez-Hernandez LR, Kattari L. A comparison of health disparities among transgender adults in Colorado (USA) by race and income. Int J Transgend. 2017;18(2):199–214.

Hughes L, King W, Gamarel K, Geronimus A, Panagiotou O, Hughto JUS. Black-white differences in mortality risk among transgender and cisgender people in private insurance, 2011–2019. Am J Public Health. 2022;112(10):1507–14.

Bonilla-Silva E. The Structure of Racism in Color-Blind, “Post-Racial” America. Am Behav Sci. 2015;59(11):1358–76.

Freeman A. Legitimizing Racial Discrimination through Antidiscrimination Law: A Critical Review of Supreme Court Doctrine. In: Crenshaw K, Gotanda N, Peller G, Thomas K, editors. Critical Race Theory: The Key Writings that Informed the Movement. New York: The New Press; 1995.

Ashley F. Don’t be so hateful: the insufficiency of anti-discrimination and hate crime laws in improving trans well-being. Univ Toronto Law J. 2018;68(1):1–36.

Spade D. Normal Life: Administrative Violence, Critical Trans Politics, and the Limits of Law. Durham: Duke University Press; 2015.

Book   Google Scholar  

Movement Advancement Project. Equality Maps Snapshot: LGBTQ Equality By State. 2023.

Rosich GR. Sexual citizenship theory and employment discrimination among transgender-identified people. Societies. 2020;10(1):17.

Davis M, Wertz K. When laws are not enough: a study of the economy health of transgender people and the need for a multidisciplinary approach to economic justice. Seattle Journal for Social Justice. 2010;8(2):467–95.

Rosentel K, VandeVusse A, Hill B. Racial and socioeconomic inequity in the spatial distribution of LGBTQ human services: an exploratory analysis of LGBTQ services in Chicago. Sexual Res Soc Policy. 2019;17(1):87–103.

Irving D, Hoo N. Doing trans-economic justice: a critique of anti-discrimination laws and inclusive employment policies. Can J Law Soc. 2020;35(2):197–220.

Badgett M, Choi S, Willson B. LGBT poverty in the United States: A study of differences between sexual orientation and gender identity groups. Los Angeles: The Williams Institute; 2019.

Bourgois P, Holmes SM, Sue K, Quesada J. Structural vulnerability: operationalizing the concept to address health disparities in clinical care. Acad Med. 2017;92(3):299–307.

Quesada J, Hart LK, Bourgois P. Structural vulnerability and health: Latino migrant laborers in the United States. Med Anthropol. 2011;30(4):339–62.

James S, Herman J, Rankin S, Keisling M, Mottet L, Anafi MA. The Report of the 2015 US Transgender Survey. Washington, DC: National Center for Transgender Equality; 2016.

Grant J, Mottet L, Tanis J, Harrison J, Herman J, Keisling M. Injustice at Every Turn: A Report of the National Transgender Discrimination Survey. Washington: National Center for Transgender Equality and National Gay and Lesbian Task Force; 2011.

Human Rights Campaign. Fatal Violence Against the Transgender and Gender Non-Conforming Community in 2021 Washington, DC2021 [Available from: https://www.hrc.org/resources/fatal-violence-against-the-transgender-and-gender-non-conforming-community-in-2021 .

Lacombe-Duncan A, Jadwin-Cakmak L, Trammell R, Burks C, Rivera B, Reyes L, et al. “…Everybody Else Is More Privileged. Then It’s Us…”: a Qualitative Study Exploring Community Responses to Social Determinants of Health Inequities and Intersectional Exclusion Among Trans Women of Color in Detroit, Michigan. Sexual Res Soc Policy. 2021;19:1419-39.

Simons JD, Grant L, Rodas JM. Transgender people of color: experiences and coping during the school-age years. J LGBTQ Issues Counseling. 2021;15(1):16–37.

Yarbrough D. The carceral production of transgender poverty: how racialized gender policing deprives transgender women of housing and safety. Punishment & Society. 2021.

Bradford M, Stephens A. Trans Agenda for Liberation. Oakland: Transgender Law Center; 2021.

Lett E, Everhart A. Considerations for transgender population health research based on US national surveys. Ann Epidemiol. 2022;65:65–71.

Reisner S, Bailey Z, Sevelius J. Racial/ethnic disparities in history of incarceration, experiences of victimization, and associated health indicators among transgender women in the U.S. Women’s Health. 2014;54(8):750–67.

Rosentel K, López-Martínez I, Crosby R, Salazar L, Hill B. Black transgender women and the school-to-prison pipeline: exploring the relationship between anti-trans experiences in school and adverse criminal-legal system outcomes. Sexual Res Soc Policy. 2020;18:481-94.

Wirtz AL, Poteat T, Radix A, Althoff KN, Cannon CM, Wawrzyniak AJ, et al. American cohort to study HIV acquisition among transgender women in high-risk areas (The LITE Study): protocol for a multisite prospective cohort study in the Eastern and Southern United States. JMIR Res Protoc. 2019;8(10):e14704.

Wirtz AL, Cooney EE, Stevenson M, Radix A, Poteat T, Wawrzyniak AJ, et al. Digital epidemiologic research on multilevel risks for HIV acquisition and other health outcomes among transgender women in Eastern and Southern United States: protocol for an online cohort. JMIR Res Protoc. 2021;10(4):e29152.

Schnittker J, Bacak V. The increasing predictive validity of self-rated health. PLoS One. 2014;9(1):e84933.

Lorem G, Cook S, Leon DA, Emaus N, Schirmer H. Self-reported health as a predictor of mortality: A cohort study of its relation to other health measurements and observation time. Sci Rep. 2020;10(1):4886.

Article   CAS   PubMed   PubMed Central   Google Scholar  

US Census Bureau. 2020 American Community Survey 5-Year Public Use Microdata Samples. 2022.

US Department of Health and Human Services. Poverty Guidelines Washington, D.C. 2022 [Available from: https://aspe.hhs.gov/topics/poverty-economic-mobility/poverty-guidelines .

Garcia-Moreno C, Jansen HAFM, Ellsberg M, Heise L, Watts C. WHO Multi-country Study on Women’s Health and Domestic Violence Against Women. Switzerland: World Health Organization; 2005.

Coleman-Jensen A, Rabbitt M, Hales L, Gregory C. Survey Tools 2020 [Available from: https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/survey-tools/ .

California Health Interview Survey. CHIS 2003 Adult Questionnaire Version 11.4. Los Angeles: UCLA Center for Health Policy Research; 2011.

Scheim A, Bauer G. The Intersectional Discrimination Index: Development and validation of measures of self-reported enacted and anticipated discrimination for intercategorical analysis. Soc Sci Med. 2019;226:225–35.

US Bureau of Labor Statistics. Occupational Employment and Wage Statistics Washington, DC. 2021 [updated March 31, 2021. Available from: https://www.bls.gov/oes/ .

King W, Jadwin-Cakmak L, Trammell R, Gamarel K. Structural vulnerability as a conceptual framework for transgender health research: findings from a community needs assessment of transgender women of colour in Detroit. Cult Health Sex. 2023;25(6)681-97.

Nylund KL, Asparouhov T, Muthen BO. Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Structural Equation Modeling. 2007;14(4):535–69.

Ladson-Billings G, Tate W. Toward a critical race theory of education. Teachers College Record. 1995;97(1):47–68.

Krell E. Is transmisogyny killing trans women of color? Black trans feminisms and the exigencies of white femininity. Transgender Studies Quarterly. 2017;4(2):226–42.

Buchmueller TC, Levinson ZM, Levy HG, Wolfe BL. Effect of the affordable care act on racial and ethnic disparities in health insurance coverage. Am J Public Health. 2016;106(8):1416–21.

Vitulli E. A defining moment in civil rights history? The employment non-discrimination act, trans-inclusion, and homonormativity. Sexual Res Soc Policy. 2010;7(3):155–67.

Reisner S, Radix A, Deutsch M. Integrated and gender-affirming transgender clinical care and research. J Acquir Immune Defic Syndr. 2016;72(Suppl 3):S235-42.

Agénor M, Geffen SR, Zubizarreta D, Jones R, Giraldo S, McGuirk A, et al. Experiences of and resistance to multiple discrimination in health care settings among transmasculine people of color. BMC Health Serv Res. 2022;22(1):369.

Howard SD, Lee KL, Nathan AG, Wenger HC, Chin MH, Cook SC. Healthcare experiences of transgender people of color. J Gen Intern Med. 2019;34(10):2068–74.

Agénor M, Perkins C, Stamoulis C, Hall R, Samnaliev M, Berland S, et al. Developing a database of structural racism-related state laws for health equity research and practice in the United States. Public Health Rep. 2021;136(4):428–40.

Daum CW. The war on solicitation and intersectional subjection: quality-of-life policing as a tool to control transgender populations. New Political Sci. 2015;37(4):562–81.

Collins R. Strolling while poor: how broken-windows policing created new crime in Baltimore. Georgetown J Poverty Law Policy. 2007;14(3):419–40.

Oberman J, Johnson K. The never ending tale: racism and inequality in the era of broken windows. Cardozo Law Re. 2016;37:1075–91.

Stewart-Winter T. Queer law and order: sex, criminality, and policing in the late twentieth-century United States. J Am History. 2015;102(1):61–72.

Ritchie AJ. Invisible No More: Police Violence against Black Women and Women of Color. Boston: Beacon Press; 2017.

Thompson J. Broken policing. New Labor Forum. 2015;24(2):42–7.

Howell KB. The costs of “Broken Windows” policing: Twenty years and counting. Cardozo Law Rev. 2016;37:1059–73.

Martin KD, Sykes BL, Shannon S, Edwards F, Harris A. Monetary sanctions: legal financial obligations in US systems of justice. Annu Rev Criminol. 2018;1:471–95.

Kurdyla V. Advocating for transgender immigrants in detention centers: cisnormativity as a tool for racialized social control. Am Behav Sci. 2022;66(13):1777–96.

Poteat T, Simmons A. Intersectional structural stigma, community priorities, and opportunities for transgender health equity: findings from TRANSforming the Carolinas. J Law Med Ethics. 2022;50(3):443–55.

Rosentel K, Fuller C, Bowers S, Moore A, Hill B. Police enforcement of sex work criminalization laws in an “End Demand” City: the persistence of quality-of-life policing and seller arrests. Arch Sex Behav. 2021;50(5):1973–90.

Wesp LM, Malcoe LH, Elliott A, Poteat T. Intersectionality research for transgender health justice: a theory-driven conceptual framework for structural analysis of transgender health inequities. Transgend Health. 2019;4(1):287–96.

DeSalvo K, Bloser N, Reynolds K, He J, Muntner P. Mortality prediction with a single general self-rated health question: a meta-analysis. J Gen Intern Med. 2006;20:267–75.

Wind K, Poland B, KakemZadeh F, Jackson SJ, Tomlinson G, Jadad A. Using self-reported health as a social determinants of health outcome: a scoping review of reviews. Health Promot Int. 2023;38(6):daad165.

Erving CL, Zajdel R. Assessing the validity of self-rated health across ethnic groups: implications for health disparities research. J Racial Ethn Health Disparities. 2022;9(2):462–77.

Farvid P, Vance TA, Klein SL, Nikiforova Y, Rubin LR, Lopez FG. The health and wellbeing of transgender and gender non-conforming people of colour in the United States: a systematic literature search and review. J Commun Appl Soc Psychol. 2021;31(6):703–31.

Download references

Acknowledgements

The authors would like to express their gratitude to the transgender women who participated in this study. This study would not be possible without their experiences and participation. We appreciate the continued involvement and contributions of the Community Advisory Board that supports and guides this study. We also want to thank all the research staff who spent their time and effort to actualize this study and connect with study participants in meaningful ways.

American Cohort to Study HIV Acquisition Among Transgender Women (LITE) Study

Sari L. Reisner, ScD b,g (consortium representative), Andrea L. Wirtz, PhD MHS h , Keri N. Althoff, PhD MPH h , Chris Beyrer, MD h , James Case, MHS i , Erin Cooney, PhD j , Meg Stevenson, MSPH h , Dee Adams, MSPH h , Oliver B. Laeyendecker, PhD k , Charlotte Gaydos, DrPH k , Tonia C. Poteat, PhD MPH PA-C d , Kenneth Mayer, MD l , Asa E. Radix, MD PhD c , Christopher Cannon, MPH, ACRP-CP m , Jason Schneider, MD, FACP n , J. Sonya Haw, MD n , Allan Rodriguez, MD o , Andrew J. Wawrzyniak, PhD p , The LITE Community Advisory Board, which includes the following individuals: Sherri Meeks, Sydney Shackelford, Nala Toussaint, SaVanna Wanzer, as well as those who have remained anonymous

i Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD 21205, USA

j Department of International Health, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD 21205, USA

k Johns Hopkins University School of Medicine, 733 North Broadway, Baltimore, MD 21205, UAS

l Fenway Health, 1340 Boylston St, Boston, MA 02215, USA

m Whitman-Walker, 1525 14 th St. NW, Washington, DC, 20005, USA

n Department of Medicine, Emory University, 49 Jesse Hill Jr Dr SE, Atlanta, GA 30303, USA

o Department of Medicine, Miller School of Medicine, University of Miami, 1600 NW 10 th Ave, #1140, Miami, FL, 33136, USA

p Department of Psychiatry and Behavioral Sciences, Miller School of Medicine, University of Miami, 1600 NW 10 th Ave, #1140, Miami, FL, 33136, USA

Research reported in this publication was jointly supported by the National Institute of Allergy and Infectious Diseases, the National Institute of Mental Health, and the National Institute of Child Health and Human Development of the National Institutes of Health under Award Number UG3/UH3AI133669 (Wirtz/Reisner). This research was supported in part by NICHD of the National Institutes of Health under award numbers T32HD007339 and P2CHD041028, awarded to the Population Studies Center at the University of Michigan. Research reported in this publication was also supported by HIV/AIDS, Hepatitis, STD, and TB Administration (HAHSTA), Washington DC Department of Health. The LITE study also appreciates support from the Centers for AIDS Research at partner institutions, including JHU (P30AI094189), Emory University (P30AI050409), Harvard University (P30AI060354), DC CFAR (AI117970), and the University of Miami (P30AI073961). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or HAHSTA.

Author information

Wesley M. King and Sari L. Reisner consortium representative.

Authors and Affiliations

Department of Health Behavior and Health Education, University of Michigan School of Public Health, 1415 Washington Heights, Ann Arbor, MI, 48109, USA

Wesley M. King, Kristi E. Gamarel & Linda M. Chatters

Department of Epidemiology, University of Michigan School of Public Health, 1415 Washington Heights, Ann Arbor, MI, 48109, USA

Nancy L. Fleischer & Sari L. Reisner

Callen-Lorde Community Health Center, 356 West 18th Street, New York, NY, 10011, USA

Asa E. Radix

Duke University School of Nursing, Box 3322 DUMC, Durham, NC, 27710, USA

Tonia C. Poteat

University of Michigan School of Social Work, 1080 South University Avenue, Ann Arbor, MI, 48109, USA

Linda M. Chatters

Department of Behavioral, Social, and Health Education Sciences, Rollins School of Public Health, Emory University, 1518 Clifton Road, Atlanta, GA, 30322, USA

Don Operario

Department of Epidemiology, Harvard T.H. Chan School of Public Health, 677 Huntington Avenue, Boston, MA, 02115, USA

• Sari L. Reisner

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD, 21205, USA

Andrea L. Wirtz

You can also search for this author in PubMed   Google Scholar

• , Andrea L. Wirtz
• , Keri N. Althoff
• , Chris Beyrer
• , James Case
• , Erin Cooney
• , Meg Stevenson
• , Dee Adams
• , Oliver B. Laeyendecker
• , Charlotte Gaydos
• , Tonia C. Poteat
• , Kenneth Mayer
• , Asa E. Radix
• , Christopher Cannon
• , Jason Schneider
• , J. Sonya Haw
• , Allan Rodriguez
• , Andrew J. Wawrzyniak
• , Sherri Meeks
• , Sydney Shackelford
• , Nala Toussaint
•  & SaVanna Wanzer

Contributions

WK conceptualized this study with support from KG, NF, and DO. WK and NF created the analysis plan. WK performed all analyses including preparing tables and figures. AW, SR, AR, TP, and the LITE Study Group conceptualized, acquired funding for, and conducted the original study from which data for these analyses were used. WK produced the original manuscript draft, which was reviewed and revised by all authors.

Corresponding author

Correspondence to Wesley M. King .

Ethics declarations

Ethics approval and consent to participate.

Study procedures including informed consent to participate were approved by the Johns Hopkins School of Medicine single Institutional Review Board.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• a Selected model

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

King, W.M., Gamarel, K.E., Fleischer, N.L. et al. Racial/ethnic differences in the association between transgender-related U.S. state policies and self-rated health of transgender women. BMC Public Health 24 , 911 (2024). https://doi.org/10.1186/s12889-024-18317-z

Download citation

Received : 18 October 2023

Accepted : 09 March 2024

Published : 28 March 2024

DOI : https://doi.org/10.1186/s12889-024-18317-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Transgender
• U.S. state policies
• Self-rated health

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 23 March 2024

Technology, data, people, and partnerships in addressing unmet social needs within Medicaid Managed Care

• Rachel Hogg-Graham 1 ,
• Allison M. Scott 2 ,
• Emily R. Clear 1 ,
• Elizabeth N. Riley 1 &
• Teresa M. Waters 3  

BMC Health Services Research volume  24 , Article number:  368 ( 2024 ) Cite this article

197 Accesses

Metrics details

Individuals with unmet social needs experience adverse health outcomes and are subject to greater inequities in health and social outcomes. Given the high prevalence of unmet needs among Medicaid enrollees, many Medicaid managed care organizations (MCOs) are now screening enrollees for unmet social needs and connecting them to community-based organizations (CBOs) with knowledge and resources to address identified needs. The use of screening and referral technology and data sharing are often considered key components in programs integrating health and social services. Despite this emphasis on technology and data collection, research suggests substantial barriers exist in operationalizing effective systems.

We used qualitative methods to examine cross-sector perspectives on the use of data and technology to facilitate MCO and CBO partnerships in Kentucky, a state with high Medicaid enrollment, to address enrollee social needs. We recruited participants through targeted sampling, and conducted 46 in-depth interviews with 26 representatives from all six Kentucky MCOs and 20 CBO leaders. Qualitative descriptive analysis, an inductive approach, was used to identify salient themes.

We found that MCOs and CBOs have differing levels of need for data, varying incentives for collecting and sharing data, and differing valuations of what data can or should do. Four themes emerged from interviewees’ descriptions of how they use data, including 1) to screen for patient needs, 2) to case manage, 3) to evaluate the effectiveness of programs, and 4) to partner with each other. Underlying these data use themes were areas of alignment between MCOs/CBOs, areas of incongruence, and areas of tension (both practical and ideological). The inability to interface with community partners for data privacy and ownership concerns contributes to division. Our findings suggest a disconnect between MCOs and CBOs regarding terms of their technology interfacing despite their shared mission of meeting the unmet social needs of enrollees.

Conclusions

While data and technology can be used to identify enrollee needs and determine the most critical need, it is not sufficient in resolving challenges. People and relationships across sectors are vital in connecting enrollees with the community resources to resolve unmet needs.

Peer Review reports

Introduction

Individuals with unmet social needs, like food and housing insecurity and transportation challenges, experience higher rates of adverse health outcomes [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] and are subject to greater inequities in health and social outcomes [ 8 ]. Unmet social needs are especially prevalent among Medicaid enrollees [ 9 ]. For this reason, state Medicaid programs are particularly interested in testing strategies that encourage and incentivize Medicaid managed care organizations (MCOs) to identify and address the complex social needs of enrollees [ 10 , 11 ]. Many Medicaid MCOs are now screening enrollees for their unmet social needs and connecting them to community-based organizations (CBOs) better equipped with knowledge and resources to address these needs [ 12 , 13 ].

The use of screening and referral technology and data sharing are often considered key components in programs integrating health and social services to address social needs [ 12 , 14 ]. Data sharing infrastructure has been highlighted as a way to streamline coordination and social need resolution [ 12 , 14 ]. In some instances, successful integration has facilitated strong connections between health and social services organizations, ensuring that patients move efficiently between sectors [ 14 , 15 , 16 ]. Despite this emphasis on technology and data collection and some positive integration, research suggests substantial barriers exist in operationalizing effective systems [ 12 , 17 ]. CBOs often have limited resources, financial and personnel, to put toward the use of advanced social need screening and referral systems [ 12 , 17 , 18 , 19 ]. The reliance on grant funding and other time-limited resource streams likely presents another barrier in the adoption of tools [ 17 ]. CBOs can also be hesitant to adopt technology and data systems owned by MCOs, hospitals, and other clinically oriented organizations because of data privacy and HIPAA-related issues [ 16 , 20 ].

Research examining health and community partnerships has identified technology adoption by CBOs and other social services organizations as an important barrier to collaboration [ 14 , 15 , 17 ]. Most prior studies examining data and technology include clinical organization perspectives on the use of tools but do not include robust information from community partners [ 12 , 14 , 16 ]. Further, those studies that do include perspectives from multiple organization types on the integration of health and social services are not focused on adopting screening and referral systems. Technology typically emerges in subthemes, and the evidence included does not provide in-depth information on benefits and challenges from both community and clinical partners [ 17 ].

This study examines CBO and MCO perspectives on the use of technology in social need screening and referral. The qualitative analysis presented here is part of a larger mixed methods study examining how Kentucky (KY) MCOs address unmet social needs in partnership with community organizations [ 21 ]. KY offers a unique opportunity to examine strategies addressing Medicaid enrollee needs. Just under 29% of all KY residents are enrolled in Medicaid, making it the third highest enrollment among US states [ 22 ]. KY is also geographically diverse, with distinct urban, rural, and Appalachian regions.

Setting and study population

A project Stakeholder Advisory Board (SAB), including representatives from all Medicaid MCOs, academia, a community-based organization, the State Department for Medicaid Services, and enrollees, met quarterly to provide expertise, guide research, and assist with the dissemination of study results. MCO representatives serving on our SAB were asked to 1) identify individuals in their organization leading efforts to address unmet social needs and population health outcomes among their enrollees and 2) identify CBOs they work closely with in their social need referral process. As part of a targeted sampling strategy, identified contacts were invited via email by the research team to participate in key informant interviews to discuss how MCOs and CBOs address social needs. Inclusion criteria were that participants were at least 18 years old, were employed at an MCO/CBO in Kentucky, and were willing to engage in an interview in English. A total of 32 MCO contacts were invited and 33 CBOs, giving us response rates of 81% and 58% respectively.

Participants

Our sample of 46 participants comprised 26 representatives from 6 MCOs (ranging from 3 to 6 participants per MCO) and 20 representatives from 19 unique CBOs. MCO participants represented various organizational roles, including vice presidents, directors, population health, case management, and community engagement. CBO participants represented roles including directors, Chief Executive Officers, Chief Operating Officers, Medical Coordinators, Presidents, Chief Engagement officers, program managers, and outreach coordinators. The services provided by community-based organizations included food security, health, housing, employment, and work readiness, refugee and immigrant services, and community support; many CBOs addressed multiple social needs. CBO interviewees represented organizations operating in both urban and rural areas of the state.

Data collection

In-depth one-on-one interviews with 46 stakeholders from identified CBOs ( n  = 20) and MCOs ( n  = 26) were conducted between May 24, 2021, and November 8, 2021. Interviews were conducted via Zoom, audio-recorded, and transcribed verbatim. The qualitative researcher and facilitator conducting these interviews have extensive training and experience with structural interviewing using a semi-structured interview guide. The guide used was developed for this study [ 23 ].

Data analysis

We conducted an iterative content analysis of the transcribed interview data using qualitative descriptive analysis [ 24 ], an inductive, low-inference method designed to gain an accurate understanding of a phenomenon in the everyday terms of stakeholders. Our data analysis unfolded in two stages. The first stage involved open coding [ 25 ], in which the transcripts were independently coded by two authors and one study team member (AM, ER, and HS), who then met to discuss and reach consensus on the central themes in the data related to technology and data sharing. In this meeting, the authors identified the themes of to screen for patient needs, to case manage, to evaluate the effectiveness of programs, and to partner with each other. The second stage of analysis involved focused coding, with the three individuals again independently coding transcripts for subthemes within each identified central theme. The coders met again to compare findings and finalize themes (and subthemes for Theme 4). At this time, we recognized that there were areas of alignment, incongruence, and tension between the responses of participants from MCOs and CBOs, and we reached agreement in this meeting about which themes demonstrated each dynamic. Finally, all authors met a third time to review the subthemes and select illustrative quotations for each. All analytic decisions were made through discussion until consensus was reached. We used the team-based approach to reaching consensus, which considered dependability and trustworthiness of the data [ 26 ]. This paper focuses on responses addressing technology platforms and data sharing to support MCO and CBO partnerships.

We identified several themes related to the use of technology and data in MCO-CBO partnerships to address enrollee social needs. MCOs and CBOs noted differing levels of need for data, differing incentives for collecting and sharing data, and differing valuations of what data can or should do. MCO and CBO interviewees described how they collect and use data in their work, which fell into four major themes: to screen for patient needs, to case manage, to evaluate the effectiveness of programs, and to partner with each other. Within these themes, the interview responses illuminated areas of alignment between MCOs/CBOs, incongruence, and tension (both practical and ideological; see Table  1 ).

Theme 1. Alignment on collecting data to identify and prioritize patient needs

Using data to identify and prioritize patient needs was largely an area of alignment for MCOs and CBOs. All MCOs and nearly all CBOs recognized the value of data in this area. As one CBO noted,

“By completing the needs assessment with our families, it helps the case managers understand your immediate needs.”

Similarly, MCOs often used the data for targeted programming and social needs referrals,

“ When our members are enrolled, we attempt to engage them in our health risk assessment. And so that health risk assessment is going to not only ask them questions about their specific health, but also about some additional needs that would help us be able to identify them at enrollment and also to be able to target them for programs and other [benefits].”

Several MCO and CBO interviewees also discussed using the data to understand individual enrollee/client needs and to track overall trends among their clients. As one MCO shared,

“The end of 2021, we had a tremendous amount of referrals for food. And so maybe we need to look at doing some of our community investment work and partnering with additional providers and community partners that are in that space for next year.”

There were some differences between MCOs and CBOs in the formality and degree to which social need data was collected. MCO interviewees, particularly those on the front lines of this work, could describe detailed and comprehensive data screening metrics for patient needs and how needs were tracked in their data systems. Using data on patient needs to identify areas for intervention was described as an essential part of patient care:

“We use the screening data, not just to meet the individual member need, but to also inform health equity and types of programs that we bring to play...”

CBO interviewees, on the other hand, had greater variability in their responses about the importance of using data on social needs at an organizational-level. Most described data as having potential value but stopped short of calling it essential for their operations. One CBO stated,

“I don't know what I would do with the information if we had it.”

Conversely, one food-oriented CBO reported that they collect demographic data and use that to help with distribution,

“So think about the local pantry that I talked about earlier. Because we know, we drive a truck into [KY County]. We know that the last five times that we've been in [KY County], we saw, on average, 150 households at each of those five visits. That tells us how much product to put on the truck so that we don't run out.”

Theme 2. Differences in organizational capacity, mission, and resources influenced variability in data use to support case management

Using data to support case management activities was an area of both alignment and incongruence between MCOs and CBOs. All MCOs and many CBOs saw value in using data systems to identify resources available, track referrals and follow-ups, keep notes, and stay in contact with patients. However, there was considerable variability in the sophistication of the data systems. Most MCOs reported elaborate data tracking systems designed specifically for screening, referral, and tracking (e.g., combining medical records applications with Unite Us [ 27 ] or Find Help (formerly Aunt Bertha [ 28 ]). Some CBOs have systems designed specifically for tracking data (e.g., Electronic Health Systems or Vesta [ 29 ]), whereas others employ systems not designed specifically for tracking (e.g., Microsoft Excel spreadsheets). Most CBOs used informal data collection to screen for needs (e.g., Post-it notes, memory, a hand-written planner), and several CBOs reported that they did not use formal data systems to screen and track patient needs at all,

“Are you kidding me? No books. What I usually tell anybody who's working with me is to either email me or text me, and that's my filing system.”

MCO interviewees were more likely to report using data analytics to support and enhance case management. Frontline MCO workers spoke about this aspect of data use more often than executives, and many saw data systems as the answer to case management problems. As one MCO stated,

“We do have a case management system that keeps track. So, we are able to schedule calls. They're able to pop back up on a calling queue, so that we're able to check in with members and attempt to continuously reach out to them. So, that's kind of how we try to make sure that those members don't fall through the cracks by continuously following up.”

Most CBOs indicated that case management occurred but was more personalized and less attached to data and technology use,

“We have a database that we use for client notes. We just record case notes in there. Some of our caseworkers keep basic Excel spreadsheets on their specific clients and what they're working on. Most of that would be informal.”

Only one MCO specifically mentioned the limits of data systems for tracking and the need for a personal touch in case management, a perspective more in line with most CBO interviewees. The MCO shared this when discussing platform capabilities, stating,

“We have a case management platform, of course, where we document everything, because just like everywhere else, if you don't write it down, it didn't happen, but a lot of it is just that manual follow-up and that human touch.”

The variability in tracking system sophistication and capabilities between MCOs and CBOs was also frequently highlighted as one of the critical challenges in collaboration and a notable source of frustration for both sides. When discussing their partnerships with MCOs and data sharing, one CBO stated,

“They really wanted to know about it. And so had to spend considerable time with them about, ‘This is what we do, this is how stuff works.’ And including it's like, ‘No, we can't track. We have no way of tracking [MCO] clientele through the [KY food security] program’."

While MCO interviewees often noted this tension in collaboration, they were aware that capacity and resources typically made it harder for CBOs to track and collect data. One MCO interviewee noted,

“I think the challenge is just the data piece and the complexity of the regulations that we have to navigate, all for good reason. When you're talking about how to best leverage those community resources, if we can't kind of have those data exchanges, it makes it so much more difficult. And so when you're trying to get at outcomes or have simplified referral processes, it just makes it harder because you may not be able to get through, they may not have the HIPAA, the high-tech clearance or whatever it is. It's expensive for them to have to do that.”

Theme 3. Funding and reimbursement structures shaped how MCOs and CBOs used data to evaluate program effectiveness

We found limited alignment between MCO and CBO perspectives on using data to evaluate social need programming and partnerships. Instead, evaluation was an area fraught with incongruencies and tension between the two sectors. The financial incentives and pressures for using data differ substantially between MCOs and CBOs. MCOs reported using data to evaluate the financial impact or effectiveness of programs (particularly claims data/utilization metrics) and partnerships to justify investments or show MCO executives that meeting unmet social needs is good business. As one MCO interviewee explained,

“I think every anything that we’re doing with the community-based partner, we’re studying all that. We’re studying the reduction, so I’m able to say, okay, because we have this member in this [CBO program], in this residential treatment program, not only mama’s healthier, baby is not born exposed to opiates, no NICU, ER utilization down. I think that’s the neat thing, there’s your answer, right?”

One reason MCOs seem to be driving data collection for demonstrated effectiveness/return on investment is that they are heavily regulated in terms of how they can invest funds,

“We are doing payment innovation, we want to take money out of what’s being spent on health care and invest it into social services and that is not easy.”

As another MCO highlighted continued investment often depends on what they can demonstrate,

“Sometimes, there are finance guidelines, right? Like when I’m fighting for my budget, they’ll say, ‘Well, where’s the return on investment numbers?’.”

Conversely, only a few CBOs used data-driven evaluation to support their financial operations. When CBOs did report using data for evaluation, it was typically in relation to using outcomes data in grant writing to gain funding specifically from MCOs, data which may not serve any other useful purpose for the CBO. As one CBO stated,

“Another kind of pain point, and for like one of the managed care companies that we contract with, they give us $8,000 a year. But the requirements to receive that $8,000 is very data heavy. We have to go through and pull all this data, get different releases signed with the participants. It’s great to have extra money, but it’s also a lot of work and nothing really being tied to it, if that makes sense. They just want the data to be able to review and any good outcomes and success stories and stuff like that, which is great. But it’s a lot of work for not a lot of money.”

Theme 4. Tension in using data to partner with other MCOs and CBOs

Both MCO and CBO interviewees described several reasons why they engage in data sharing within MCO-CBO partnerships (e.g., to garner funding, demonstrate effectiveness, or enhance case management), even if the values and importance placed on data sharing differed between agency types. When data sharing existed or was being contemplated, interviewees still described several barriers to sharing, both practical and ideological.

Overwhelmingly, CBO interviewees expressed a perception that they had to report data to the MCOs to prove impact so MCOs would maintain the partnership or provide funding. The first subtheme revealed a notable ideological difference between the MCOs/CBOs regarding whether data was useful to evaluate program effectiveness . While data-driven evaluation is routine and relied upon by most MCOs, many CBO interviewees perceived that data and metrics could harm their operations, diverting time and energy from serving clients and that there is much about program effectiveness that simply cannot be captured using formal data tracking systems. When discussing the course of their partnerships with MCOs, one CBO highlighted,

“So what does that support look like? Well, it is financial support for it. And, initially, it was very much focused on their clientele with [MCO] clientele and trying to track metrics about the impact that having access to better nutrition was going to have on the outcomes for their folks, right? So over the course of two years, I mean, we were able to show, "we," and I mean that collectively, we're able to show that it does have a positive impact. I mean, for [MCO], I think it's safe to say that they realize that it is more cost-effective to invest upfront in increasing access to healthy food better than the back end, to drugs and health care costs and all that kind of stuff. So they have, again, they have maintained that partnership.”

Indeed, most MCOs expressed wanting data from their CBO partners to justify the relationship and a reluctance to build relationships if data capacity is not present. One MCO discussed this directly, stating,

“They come us and they send us their flyer and they're like, "We want [MCO] to partner with us on our heart walk and we want you to give us $20,000." We still get a lot of people that do that because that's their old business model. Most of the time, we don't engage with those types of organizations. I always say, we want to hear from someone and I will take a meeting always if a community-based organization says, "We have an evidence-based solution that is solving for X," or "We have a solution that is solving for X and we want to work with you to help us prove that it's evidence-based," or we have research capabilities...”

Subtheme 2 illustrates how underlying the data sharing tension between CBOs and MCOs are challenges related to the need for more effective and user-friendly interfacing between tracking and referral systems, as well as the limited capacity of CBOs to track and analyze data . As mentioned, the sophistication of CBO data systems is highly variable, and even those organizations with more advanced tracking systems struggle with data sharing. When asked about data sharing, one CBO noted,

“Well that's another pain point. In my history, in my experience, every health plan has their own data system that don't talk to one another, that are very convoluted and messy. Right now we're filling stuff in on an Excel spreadsheet.”

Several MCOs also highlighted this as a challenge. As one MCO stated,

“Our system is designed to deal with hospital systems and health care providers, there's many different levels. I mean we go through a pretty comprehensive system and you have to have all kinds of, meet all kinds of requirements, share data, and different pieces that for a small community-based organization providing housing services, they might not even have the capacity to meet those requirements.”

Although some CBOs reported sharing data with MCOs willingly and saw this sharing as a natural facet of their partnership, other CBOs described significant concerns about data privacy and ownership ( subtheme 3 ). They noted how important data privacy was to the clients they served and how their organization valued serving their clients without the need to collect personal data or share it. Some CBO interviewees indicated that sharing or even collecting private client data might compromise their ability to do their work and serve their clients well,

“We respect their privacy, and we will never do any sharing of their data. In fact, a lot of people who come to us, one of the reasons they're with us is because we do not require them to show an ID.”

Subtheme 4 revealed how CBO and MCO interviewees expressed concerns about relying on data and technology as the solution to social need screening and referral systems building . Interviewees felt that data does not adequately capture utilization or partnership benefits. Primarily, this was attributed to issues related to data quality. One MCO interviewee highlighted this when discussing the challenges of understanding the quality of social need services:

“We also don't have a really long track record of managing quality for this type of provider. We have very distinct report cards and quality cards for every hospital in the state of Kentucky. I can tell you what the outcomes for [Hospital 1] compared to [Hospital 2] and compared to [Hospital 3]. We have very clear metrics on those types of things. We do not have that for the sort of soft services, especially since we don't pay for them.”

Most CBOs articulated challenges with data quality centered on their perception that data does not tell the whole story about what is happening at their organization and in the community. As one CBO noted,

“ We have a people problem. And I think right now there are a lot of hospitals and other organizations, MCOs, that want to kind of tech their way out of this. [T]hey're looking for technological [solutions] to try to streamline and expand services to folks. And that's just not really the answer. You need people.”

MCO interviewees recognized that databases and their tracking systems may be limited in what they capture. In subtheme 5 , several noted their technological ability to comprehensively track organizations in a community as a significant limitation . Maintaining accurate data has also been challenging because of community organization turnover and closures. As one MCO highlighted,

“These national repositories don't have the local knowledge so they don't know the churches that do the hot meals and they don't know the small organizations that are getting up and off their feet and tied to this one or that one, or it's an offshoot of whatever. There are some smaller organizations that don't always get into those big directories and you don't always know about them unless you have boots on the ground, people who live and work in the community and actually know what those are.”

Similarly, another MCO highlighted CBO data capacity as a major challenge in their partnerships, stating,

“Biggest challenges. I guess, you could say data might be the challenges, to close the loop around the return on investment on some of these organizations that are not ... They just don't have the staffing, or the professional leadership, if you will, to do all the tracking. The ones that do, do it very well. The ones that don't, it's just that they don't have the resources.”

In the final subtheme, all MCO interviewees acknowledged that CBOs are doing good work , even if that cannot be quantified, and the ability to share that data is often related to CBO capacity and resources. One MCO shared,

“[Food Pantry CBO] who's just like [Named Female] and her husband [Named Male], they might be the greatest people and we might know that members like going there versus the other food bank because [Named Female] like bakes brownies and gives them a hug and we want to quantify that but also it's just not realistic because they don't have the infrastructure sometimes that's needed to prove the business case, solidify the partnership and ultimately inform policy.”

Our study found alignment as well as discordance between MCOs and CBOs about how and when to leverage technology and data despite their shared mission to meet the unmet social needs of enrollees. Our findings offer important insights regarding why data and technology may create a barrier to effective MCO-CBO partnerships, potentially hindering efforts to improve health and social outcomes. They also provide guidance and identify key considerations for developing programs and partnerships that may be more effective in coordinating efforts between the two organizations.

As we observed in Themes 1 (Alignment on collecting data to identify and prioritize patient needs) and 2 (Differences in organizational capacity, mission, and resources influenced variability in data use to support case management), results suggest that data and technology can be important tools in screening and referral for social needs, but they are far from a universal panacea. Our data indicate that both logistical and cultural disconnects between MCOs and CBOs significantly limit data collection and sharing for coordination of services. On the logistical side, CBOs have extremely limited capacity (software, workforce) to collect and share data. Several participants reported serious concerns with collecting and sharing confidential client information. To make matters worse, MCOs use a range of proprietary and sophisticated referral and tracking systems that severely tax the resources and capacity of CBOs. On the cultural side, while MCOs view data and technology as essential to partnering with CBOs to meet enrollee social needs, CBOs do not. In fact, as we found in Theme 3 (Funding and reimbursement structures shaped how MCOs and CBOs used data to evaluate program effectiveness), many CBOs see data collection as a necessary evil to garner funding from potential donors. Instead, they emphasize the relationship-honoring aspects of their work as a core value.

Solutions that only focus on providing data collection and tracking technology to CBOs are unlikely to be completely successful because they fail to address the disparate cultures found in MCOs vs. CBOs. This conclusion is robustly supported by Theme 4 from our analysis (Tension in using data to partner with other MCOs and CBOs).In many ways, CBOs may view MCO efforts to grow their technological capacity as imposing profit-seeking values, norms, and structure rather than seeking true understanding and partnership. CBOs’ low enthusiasm for and capacity to use data can create difficulty for MCOs when MCOs rely on CBOs for data to justify their funding streams and partnerships. This fundamental disconnect is likely to severely impede partnership efforts without reevaluating the strengths and values each sector brings to the collaborative [ 30 ].

Successful partnerships are built on shared interest and trust [ 31 ]. Our study suggests a strong alignment between MCOs and CBOs in addressing the social needs of highly vulnerable Medicaid beneficiaries. This values alignment may offer a foundation for partnership. Our work underscores a key finding across studies on cross-sector partnerships integrating health and social services, more work must be done to build trust and understand each other’s organizational values [ 17 , 19 , 32 ]. MCOs and CBOs need each other to address social determinants of health (SDOH) effectively. MCOs have the resources and responsibility for finding more effective ways to support their beneficiaries. CBOs are ‘on the ground’ and have the trust of the clients they serve (many of whom are Medicaid enrollees). Forums that create a level playing field for both types of organizations and facilitate safe conversations to build trust are essential.

The Department of Health and Human Services (DHHS) has developed a three-pronged strategy for addressing SDOH: (1) better data, (2) improving health and social services connections, and (3) whole-of-government collaborations [ 8 ]. Our study suggests that their second strategy is essential and could be far more difficult than many imagine. Facilitating honest conversations about identifying and addressing the challenges in building these connections is a critical first step. Because many challenges involve “hearts and minds” and organizational culture, addressing these challenges will need to be a slow and iterative process. Moving forward, organizations like MCOs and other clinical partners must carefully consider how data and social need screening and referral technology can be a value-add to CBOs and not another burden on their already strained capacity.

Limitations

While our sample included at least one representative from all six state MCOs and nineteen different CBOs, the generalizability of study results may not apply to other states. However, many of the MCOs in KY operate in national markets and often use similar strategies in different geographic areas. Insights likely shed light on similar efforts and challenges in other states and markets. Future studies examining the use of data and technology nationally in social need resolutions would provide confirmation of the results we present and any potential geographic variability. Additionally, participant perspectives may not necessarily represent their MCOs or CBOs. Finally, our cross-sectional view of technology and referral platforms provides a snapshot of current processes; a more in-depth longitudinal study would capture changes over time as technology constantly evolves.

Despite a shared mission to meet unmet social needs, MCOs and CBOs do not agree on how and when to leverage technology and data. This discordance is a significant barrier to effective partnerships. Technology offers powerful tools for identifying and prioritizing enrollee needs and connecting them with services. However, trust and a shared understanding of organizational cultures and goals are critically needed to allow technology to realize its potential. Current efforts to build effective MCO-CBO partnerships should focus on creating a level playing field for all organizations and a space for honest conversations that can build strong connections and sustainable relationships across sectors.

Availability of data and materials

Deidentified aggregated data is available from the corresponding author ([email protected]) on reasonable request.

Gottlieb L, Tobey R, Cantor J, Hessler D, Adler NE. Integrating social and medical data to improve population health: opportunities and barriers. Health Aff. 2016;35(11):2116–23.

Article   Google Scholar  

Seligman HK, Laraia BA, Kushel MB. Food insecurity is associated with chronic disease among low-income NHANES participants. J Nutr. 2010;140(2):304–10.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Silverman J, Krieger J, Kiefer M, Hebert P, Robinson J, Nelson K. The relationship between food insecurity and depression, diabetes distress and medication adherence among low-income patients with poorly-controlled diabetes. J Gen Intern Med. 2015;30:1476–80.

Article   PubMed   PubMed Central   Google Scholar  

Berkowitz SA, Hulberg AC, Hong C, Stowell BJ, Tirozzi KJ, Traore CY, Atlas SJ. Addressing basic resource needs to improve primary care quality: a community collaboration programme. BMJ Qual Saf. 2016;25(3):164–72.

Article   PubMed   Google Scholar  

Cole MB, Nguyen KH. Unmet social needs among low-income adults in the United States: Associations with health care access and quality. Health Serv Res. 2020;55:873–82.

Fiori KP, Heller CG, Rehm CD, Parsons A, Flattau A, Braganza S, Lue K, Lauria M, Racine A. Unmet social needs and no-show visits in primary care in a US northeastern urban health system, 2018–2019. Am J Public Health. 2020;110(S2):S242–50.

Alley DE, Asomugha CN, Conway PH, Sanghavi DM. Accountable health communities—addressing social needs through Medicare and Medicaid. N Engl J Med. 2016;374(1):8–11.

Article   CAS   PubMed   Google Scholar  

De Lew N, Sommers BD. Addressing social determinants of health in federal programs. InJAMA Health Forum 2022;3(3):e221064-e221064). American Medical Association.

Thompson T, McQueen A, Croston M, Luke A, Caito N, Quinn K, Funaro J, Kreuter MW. Social needs and health-related outcomes among Medicaid beneficiaries. Health Educ Behav. 2019;46(3):436–44.

Moreno-Camacho CA, Montoya-Torres JR, Jaegler A, Gondran N. Sustainability metrics for real case applications of the supply chain network design problem: A systematic literature review. J Clean Prod. 2019;10(231):600–18.

Apenteng BA, Kimsey L, Opoku ST, Owens C, Peden AH, Mase WA. Addressing the social needs of Medicaid enrollees through managed care: lessons and promising practices from the field. Popul Health Manag. 2022;25(1):119–25.

Cartier Y, Fichtenberg C, Gottlieb LM. Implementing Community Resource Referral Technology: Facilitators And Barriers Described By Early Adopters: A review of new technology platforms to facilitate referrals from health care organizations to social service organizations. Health Aff. 2020;39(4):662–9.

Center for Health Care Strategies [Internet]. Supporting social service and health care partnerships to address health-related social needs: case study series. [updated 2018; cited 2023 Nov 9] Available from: https://www.chcs.org/project/partnership-healthy-outcomes-bridging-community-based-human-services-health-care/ . Accessed March 2, 2023.

Klein S, Hostetter M. Leveraging Technology to Find Solutions to Patients’ Unmet Social Needs. The Commonwealth Fund; June 21, 2017. Available from: https://www.commonwealthfund.org/publications/2017/jun/leveraging-technology-find-solutions-patients-unmet-social-needs

Blavin F, Smith LB, Ramos C, Ozanich G, Horn A. Opportunities to Improve Data Interoperability and Integration to Support Value-Based Care. 2022.

Massar RE, Berry CA, Paul MM. Social needs screening and referral in pediatric primary care clinics: a multiple case study. BMC Health Serv Res. 2022;22(1):1369.

Hogg‐Graham R, Edwards K, L Ely T, Mochizuki M, Varda D. Exploring the capacity of community‐based organisations to absorb health system patient referrals for unmet social needs. Health Soc Care Commun. 2021;29(2):487–95.

Amarashingham R, Xie B, Karam A, Nguyen N, Kapoor B. Using community partnerships to integrate health and social services for high-need, high-cost patients. Issue Brief (Commonw Fund). 2018;2018:1–11.

PubMed   Google Scholar  

Agonafer EP, Carson SL, Nunez V, Poole K, Hong CS, Morales M, et al. Community-based organizations’ perspectives on improving health and social service integration. BMC Public Health. 2021;21(1):1–12.

Petchel S, Gelmon S, Goldberg B. The Organizational Risks Of Cross-Sector Partnerships: A Comparison Of Health And Human Services Perspectives: A legal and policy review to identify potential funding streams specifically for Accountable Communities For Health infrastructure activities. Health Aff. 2020;39(4):574–81.

Hogg-Graham R, Scott AM, Stahl H, Riley E, Clear ER, Waters TM. COVID-19 and MCO-community partnerships to address enrollee social needs. Am J Managed Care. 2023;29(3).

KFF. Medicaid State Fact Sheets 2023 [Available from: https://www.kff.org/interactive/medicaid-state-fact-sheets/ .

Hogg-Graham R, Scott AM, Waters TM. Medicaid Managed Care Organizations and Community Based Organizations Social Need Strategies Interview Guide. University of Kentucky. 2021.

Sandelowski M. Using qualitative research. Qual Health Res. 2004;14(10):1366–86.

Thornberg R, Charmaz K. Grounded theory and theoretical coding. The SAGE handbook of qualitative data analysis. 2014;2014(5):153–69.

Cascio MA, Lee E, Vaudrin N, Freedman DA. A team-based approach to open coding: Considerations for creating intercoder consensus. Field Methods. 2019;31(2):116–30.

Unite Us. Cross-sector collaboration software powered by community. [updated 2023; cited 2023 Nov 9]. Available from: https://uniteus.com/ .

FindHelp.org. Social Care Technology [Internet]. [updated 2023; cited 2023 Nov 9]. Available from: https://company.findhelp.com/ .

The Partnership Center. Vesta [Internet]. [updated 2023; cited 2023 Nov 9]. 2023; Available from: https://thepcl.net/vesta.html .

Varda DM, Retrum JH. Collaborative performance as a function of network members’ perceptions of success. Public Perform Manag Rev. 2015;38(4):632–53.

Varda DM, Chandra A, Stern SA, Lurie N. Core dimensions of connectivity in public health collaboratives. J Public Health Manag Pract. 2008;14(5):E1-7.

Byhoff E, Taylor LA. Massachusetts community-based organization perspectives on Medicaid redesign. Am J Prev Med. 2019;57(6):S74–81.

Download references

Acknowledgements

The authors would like to thank the Study Advisory Board for their help in guiding the research.

This research was supported by a Robert Wood Johnson Foundation grant as part of the Research in Transforming Health and Health Systems Program (Grant ID 77256). Research reported in this publication was also supported by the Kentucky Cabinet for Health and Family Services, Department for Medicaid Services under Agreement C2517 titled “Medicaid Managed Care Organizational Strategies to Address Enrollee Unmet Social Needs.” The content is solely the responsibility of the authors and does not necessarily represent the official views of the Cabinet for Health and Family Services, Department for Medicaid Services.

Author information

Authors and affiliations.

Department of Health Management and Policy, College of Public Health, University of Kentucky, 111 Washington Ave, 107B, Lexington, KY, USA

Rachel Hogg-Graham, Emily R. Clear & Elizabeth N. Riley

Department of Communication, University of Kentucky, Lexington, KY, USA

Allison M. Scott

Institute for Public and Preventive Health, Augusta University, Augusta, GA, USA

Teresa M. Waters

You can also search for this author in PubMed   Google Scholar

Contributions

Concept and design (RH-G, AMS, TMW); acquisition of data (RH-G, AMS, ERC, TMW); analysis and interpretation of data (RH-G, AMS, ER, TMW); drafting of the manuscript (RH-G, AMS, ER, ERC, TMW); critical revision of the manuscript for important intellectual content (RH-G, AMS, ER, TMW); provision of patients or study materials (RH-G, ERC); obtaining funding (RH-G, TMW); administrative, technical, or logistic support (RH-G, ERC, TMW); and supervision (RH-G).

Corresponding author

Correspondence to Rachel Hogg-Graham .

Ethics declarations

Ethics approval and consent to participate.

All research activities involving human subjects have been reviewed and approved by the Institutional Review Board at the University of Kentucky. Informed consent was verbally obtained by all participants. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hogg-Graham, R., Scott, A.M., Clear, E.R. et al. Technology, data, people, and partnerships in addressing unmet social needs within Medicaid Managed Care. BMC Health Serv Res 24 , 368 (2024). https://doi.org/10.1186/s12913-024-10705-w

Download citation

Received : 13 November 2023

Accepted : 11 February 2024

Published : 23 March 2024

DOI : https://doi.org/10.1186/s12913-024-10705-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social determinants of health
• Managed care organizations
• Health care organizations and systems

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

How People Are Really Using GenAI

• Marc Zao-Sanders

The top 100 use cases as reported by users on Reddit, Quora, and other forums.

There are many use cases for generative AI, spanning a vast number of areas of domestic and work life. Looking through thousands of comments on sites such as Reddit and Quora, the author’s team found that the use of this technology is as wide-ranging as the problems we encounter in our lives. The 100 categories they identified can be divided into six top-level themes, which give an immediate sense of what generative AI is being used for: Technical Assistance & Troubleshooting (23%), Content Creation & Editing (22%), Personal & Professional Support (17%), Learning & Education (15%), Creativity & Recreation (13%), Research, Analysis & Decision Making (10%).

It’s been a little over a year since ChatGPT brought generative AI into the mainstream. In that time, we’ve ridden a wave of excitement about the current utility and future impact of large language models (LLMs). These tools already have hundreds of millions of weekly users, analysts are projecting a multi-trillion dollar contribution to the economy, and there’s now a growing array of credible competitors to OpenAI.

• Marc Zao-Sanders is CEO and co-founder of filtered.com , which develops algorithmic technology to make sense of corporate skills and learning content. He’s the author of Timeboxing – The Power of Doing One Thing at a Time . Find Marc on LinkedIn or at www.marczaosanders.com .

Partner Center

• High contrast
• Our mandate
• Our history
• Annual report
• PRESS CENTRE

Search UNICEF

For every child, answers

Research and foresight that drive change for children

Latest work

Youth, Protests and the Polycrisis

Exploring how youth protests can help to build public support for change

Early Childhood Education Systems in Pacific Islands

Status report

Cash Plus Model for Safe Transitions to Healthy Adulthood

Examining the impacts of “Ujana Salama” (‘Safe Youth’ in Swahili), a cash plus programme targeting adolescents in the United Republic of Tanzania

The Impact of Valor Criança

Social cash transfer pilot programme in Angola

The Impact of the Cash Transfer Intervention

In the commune of Nsélé in Kinshasa, Democratic Republic of the Congo

Mitigating the socioeconomic impacts of COVID-19

With a cash transfer in peri-urban Kinshasa, Democratic Republic of the Congo

Prospects for Children: A Global Outlook 2024

Cooperation in a Fragmented World: Discover the eight trends that will define the year ahead for children and young people.

Data Must Speak: Chad

Reports and project briefs

Areas of work

Adolescent participation and civic engagement

Child protection

Climate crisis and the environment

Digital technology

Education and human capital

Health and well-being

Inclusion and equity

Poverty and social protection

Social and behaviour change

The State of the World’s Children

UNICEF’s flagship report – the most comprehensive analysis of global trends affecting children

Changing Childhood Project

What is childhood like today?

Prospects for Children: Global Outlook

An annual analysis of trends shaping the world and their impact on children

Report Card

Understanding child well-being everywhere

Our approach

UNICEF Innocenti works for and with children and young people to seek solutions to their most pressing challenges. As we focus on the rights and lives of children and young people, we always ask: Who else can we include? Will this work cause unintended harm? Are there events that could surprise us? Does this work drive change?

Events and insights

Six ways to make Loss and Damage finance work for children

Climate change is hurting kids. Here is how we can address the harm

The Antidote to Ageism

Understanding the importance of intergenerational collaboration

Celebrating women in education

A closer look at female teachers and school leaders

As they move

Child and youth experiences of migration, displacement and return in Afghanistan

Launch of UNICEF's Youth Foresight Playbook

28 November 2023, Dubai Future Forum

Expert consultation on age-related public expenditure

12-13 April 2023

The Third Annual KIX Symposium

12-13 October 2022

UNICEF at the International SBCC Summit 2022

5-9 December 2022

Become an Innocenti Insider

Receive the latest research and event invites in your inbox once a month

Advertisement

Engineers Raise Questions About Bridge’s Construction as Inquiry Begins

In reviewing images of the Francis Scott Key Bridge, some structural engineers said that its piers, which are essential to the structure’s integrity, appeared to lack protective barriers.

• Share full article

By James Glanz and Annie Correal

• March 26, 2024

The large container ship that collided with the Key Bridge in Baltimore, leading to its near-total collapse, appeared to strike a critical component, known as a pylon or pier, according to several engineers who have reviewed footage of the incident.

Without the pier, they said, it was impossible for other components of the bridge to assume the load and keep the bridge standing.

The piers on a bridge act as a kind of leg and are what is known as “nonredundant” parts of a bridge’s structure. If a pier is somehow taken out, there is nothing to compensate for the missing structural support, and a collapse of the bridge is all but inevitable, most of the analysts said.

How Fenders Might Have Protected Against Bridge Collapse

The Francis Scott Key Bridge did not have an obvious fender system, or protective barriers, to redirect or prevent a ship from crashing into the bridge piers.

Yet the collapse in Baltimore on Tuesday might have been avoided, some of the engineers said, if the piers had been better able to block, deflect or withstand such a collision. And some of the engineers questioned whether the bridge’s piers had adequate blocking devices that are known with a self-explanatory name: fenders.

In bridge engineering, fenders can be anything from simple pyramids of rocks piled around the pylons to major concrete rings padded with slats of wood, designed to shield the bridge’s supports from damage by water or collisions.

It was not clear whether any such protection built around the bridge’s piers was sufficient to guard against even a glancing hit from a 95,000-gross-ton container vessel.

And the U.S. secretary of transportation, Pete Buttigieg, expressed doubt on Tuesday that any bridge could have withstood such a serious collision.

“This is a unique circumstance. I do not know of a bridge that has been constructed to withstand a direct impact from a vessel of this size,” he told reporters.

Yet a different perspective emerged in initial comments by the investigators who will be sorting out what happened in the collapse.

Jennifer Homendy, the chair of the National Transportation Safety Board, said protective structures would be a part of the investigation into the collapse. “There’s some questions about the structure of the bridge — protective structure around the bridge or around the piers to make sure there isn’t a collapse,” she said, responding to a reporter’s question.

“We are aware of what a structure should have. Part of our investigation will be how was this bridge constructed? It will look at the structure itself. Should there be any sort of safety improvements? All of that will be part of our investigation.”

The Maryland Transportation Authority did not immediately respond to a request for comment on the design of the piers in Baltimore, and did not say whether any fenders were installed to protect them.

Between 1960 and 2015, there were 35 major bridge collapses worldwide because of ship or barge collisions, resulting in the deaths of 342 people, according to a 2018 report from the World Association for Waterborne Transport Infrastructure, a scientific and technical organization.

The deadliest crash took place in 1983, when a passenger ship collided with a railroad bridge on the Volga River in Russia, killing 176 people, according to the report.

It was only after “a marked increase in the frequency and severity of vessel collisions with bridges” that attempts to study and address the risks were initiated in the 1980s, said the report’s authors, Michael Knott and Mikele Winters.

A widely circulated video of the Key Bridge failure drew attention to the disastrous collapse of the upper bridge structure. But engineers who reviewed the footage said that did not appear to be the culprit in the disaster. Instead, they said, the superstructure failure was most likely a secondary effect of the pier crumbling beneath it after the collision.

Engineers who reviewed images of the bridge both before and after the collapse said no significant fender structures were visible. Only fairly small structures were visible in photos taken at the foot of the pier, and they did not appear to be substantial enough to be able to stop a large ship, some of them said. They said the structures may have served another purpose entirely — like preventing water from scouring and undermining the pier’s foundation.

Benjamin W. Schafer, a professor of engineering at Johns Hopkins University, said, after looking at images of the bridge taken before the disaster, “If you zoom further out, you can see these large cylinders that sort of define the shipping channel. They are to direct the ships and they are part of the bridge structure. Some would say those are protective structures. But I haven’t seen any evidence of fenders myself.”

In some bridges, engineers may elect, instead of fendering, “the alternative of making the pier exceptionally strong,” said Shankar Nair, a structural engineer with over half a century of experience who is a member of the National Academy of Engineering. But the visual evidence so far, he and others said, suggested that the pier was simply not strong enough to survive the collision.

The structure’s apparent vulnerability left some engineers dumbfounded.

“This is a huge shock,” Dr. Nair said. “A bridge of that size and importance should not collapse when hit by an errant vessel.”

The importance of sturdy fenders on bridge piers was backed up by a similar accident that occurred in 2013 when a 752-foot-long tanker collided with a support of the San Francisco-Oakland Bay Bridge. According to a National Transportation Safety Board report on the incident, the support stood — although $1.4 million in damage was done to the fendering system, which cushioned the impact.

In other cases when collisions lead to full or partial collapses, shortcomings in the fendering system are usually involved, said Matthys Levy, a longtime structural engineer and co-author of “Why Buildings Fall Down.”

“It’s usually an issue of fendering,” Mr. Levy said. “The fendering is not strong enough.”

According to a description of the Key Bridge by an American Society of Civil Engineers manual, the 8,636-foot-long structure in Baltimore was opened to traffic in 1977. The steel span above it, a design known as a truss, can be vulnerable to failure itself — damage to individual elements of the truss can theoretically cascade into a wider collapse. But that did not appear to be the case in Baltimore, engineers who reviewed the footage said: The truss, they said, was simply unable to remain intact when the pier was taken out beneath it.

Tuesday’s collapse raises the question “of how vulnerable are the piers and what is done or should have been done to protect them in the event of something like this,” said Donald O. Dusenberry, a consulting engineer who has investigated many bridge failures.

Mr. Dusenberry, in pointing to the issue of fender protection, said that it was impossible to make a full determination of what was installed without reviewing structural drawings of the bridge.

But images taken before the disaster, he said, suggested that small barriers that could be seen rising around the bridge’s piers, roughly at water level, would be unlikely to be able to stop a large ship. Effective fenders, he said, had to be far enough from the pier to keep the bow of a large ship from striking the pier, and large enough to absorb the energy of a collision. Assuming nothing had changed since the prior pictures were taken, he said, the visible structures did not seem up to that task.

“Maybe it would stop a ferry or something like that,” he said. “Not a massive, oceangoing cargo ship.”

One of the catastrophes prompting scrutiny of the issue of bridge collisions was the collapse of the Sunshine Skyway Bridge in Tampa, Fla., in 1980.

The structure collapsed when a cargo ship hit a pier, bringing down part of the main span and killing 35 people. Seven years later, a shrimp boat hit a bumper erected on the bridge built to replace it.

While catastrophic collisions garner the most attention, vessel collision accidents with bridges are not uncommon and regularly cause damage that, according to the 2018 report, “varies from minor to significant but does not necessarily result in collapse of the structure or loss of life.”

Mr. Schafer, the professor of engineering at Johns Hopkins, said fenders were undeniably important to preventing catastrophic collisions but that the size of the vessel that hits a bridge plays a critical role.

“When people think about fenders, they’re thinking about something that is similar in scale, in size, to the supporting concrete structure itself,” Mr. Schafer said. “So, you know, if that is 30-feet across, you might think of a fender which is like 30 feet as well. Right?”

The problem, he said, comes with trying to design protection against something so large as a container ship. “Could we design something that’s big enough to divert a runaway cargo ship? Yes. Would it be of a scale that’s practical? Probably not.”

Rather than build bigger fenders, Mr. Schafer said, the key is to divert ships before they get dangerously close to the piers and fenders. “That would be the physical answer,” he said. “The better answer is to have the people and the processes in place, so it never happens .”

James Glanz is a Times international and investigative reporter covering major disasters, conflict and deadly failures of technology. More about James Glanz

Annie Correal reports from the U.S. and Latin America for The Times. More about Annie Correal