Log in using your username and password
- Search More Search for this keyword Advanced search
- Latest content
- For authors
- Browse by collection
- BMJ Journals
You are here
- Volume 3, Issue 8
- Systematic literature review of templates for reporting prehospital major incident medical management
- Article Text
- Article info
- Citation Tools
- Rapid Responses
- Article metrics
- Sabina Fattah 1 , 2 ,
- Marius Rehn 1 , 3 , 4 ,
- Eirik Reierth 5 ,
- Torben Wisborg 2 , 6
- 1 Department of Research and Development , Norwegian Air Ambulance Foundation, Drøbak , Norway
- 2 Anaesthesia and Critical Care Research Group, Faculty of Health Sciences , University of Tromsø, Tromsø , Norway
- 3 Network of Medical Sciences , Field of Pre-hospital Critical Care, University of Stavanger, Stavanger , Norway
- 4 Department of Anaesthesia and Intensive Care , Akershus University Hospital, Lørenskog , Norway
- 5 Science and Health Library , University Library of Tromsø, University of Tromsø, Tromsø , Norway
- 6 Department of Anaesthesiology and Intensive Care , Hammerfest Hospital, Finnmark Health Trust, Hammerfest , Norway
- Correspondence to Sabina Fattah; sabina.fattah{at}norskluftambulanse.no
Objective To identify and describe the content of templates for reporting prehospital major incident medical management.
Design Systematic literature review according to PRISMA guidelines.
Data sources PubMed/MEDLINE, EMBASE, CINAHL, Scopus and Web of Knowledge. Grey literature was also searched.
Eligibility criteria for selected studies Templates published after 1 January 1990 and up to 19 March 2012. Non-English language literature, except Scandinavian; literature without an available abstract; and literature reporting only psychological aspects were excluded.
Results The main database search identified 8497 articles, among which 8389 were excluded based on title and abstract. An additional 96 were excluded based on the full-text. The remaining 12 articles were included in the analysis. A total of 107 articles were identified in the grey literature and excluded. The reference lists for the included articles identified five additional articles. A relevant article published after completing the search was also included. In the 18 articles included in the study, 10 different templates or sets of data are described: 2 methodologies for assessing major incident responses, 3 templates intended for reporting from exercises, 2 guidelines for reporting in medical journals, 2 analyses of previous disasters and 1 Utstein-style template.
Conclusions More than one template exists for generating reports. The limitations of the existing templates involve internal and external validity, and none of them have been tested for feasibility in real-life incidents.
Trial registration The review is registered in PROSPERO (registration number: CRD42012002051).
- Disaster Medicine
- Emergencies
- Major Incidents
- Mass Casualty Incidents
- Data Collection
- Health Care Management
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
http://dx.doi.org/10.1136/bmjopen-2013-002658
Statistics from Altmetric.com
Request permissions.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Article summary
Article focus.
Identify templates that enable systematic and uniform reporting of prehospital major incident medical management.
Appraise the data fields in the included templates regarding internal and external validity.
Key messages
Templates for reporting major incident medical management exist in different formats, but none have been tested for feasibility.
A template for generating reports from the prehospital phase with clearly defined data variables enabling comparative analysis is needed.
Strengths and limitations of the study
A systematic review following the preferred reporting items for systematic reviews and meta-analyses guidelines.
The protocol was published and deviations from protocol are revealed in the study report.
Only English and Scandinavian language literature was included.
Introduction
Major incidents, such as natural disasters, accidents and terrorist attacks, affect millions of lives each year. In 2011, natural disasters alone killed more than 30 000 people and injured 244 million people worldwide. The 332 natural disasters in 2011 caused the highest economic damage ever recorded; Asia was the continent most often hit, followed by the Americas, Africa, Europe and Oceania. This regional distribution of disaster resembles the profile observed from 2001 to 2010. Over the last decade, China, the USA, the Philippines, India and Indonesia were the five countries most frequently hit by natural disasters. 1
Although disaster medicine can be traced back to the Middle Ages, it has become a distinct scientific discipline in only the last 60 years. 2 An evolving trend in disaster medicine calls for improved reporting of major incidents in order to increase the level of science within this field. 3–8 Previous expert group processes defined uniform data sets for reporting in both emergency medicine in general 9 , 10 and in disaster medicine specifically. 11 , 12 These templates were designated as Utstein-style templates after the Norwegian monastery where they were developed. Qualitative research methods have also been used to identify areas within prehospital critical care and major incident management that require further research. 13 , 14 A recent review identified data from mass gatherings as non-uniform and called for consistent data to enable future research. 15 The importance of evaluating disaster exercises using predefined, high-quality data has also been discussed as a potential for improving disaster health management 16 and for comparing outcomes from different exercises. 17 The analysis of standardised data from previous incidents can allow decision-makers to make well-informed decisions. 18
This systematic review was designed to identify and describe the content of templates for reporting prehospital major incident medical management. The questions being asked in this systematic review were: which data are reported in the existing templates (data extraction), and are the templates internally and externally valid with regard to the methodology with which they were developed and the data they are reporting (quality appraisal)? The need for a template for uniform reporting was assessed based on the findings. To the best of our knowledge, no similar studies have been performed or registered in the Cochrane or Prospero databases.
Search strategy
A systematic literature search was performed to identify templates published after 1 January 1990 and up to 19 March 2012. 19 The controlled vocabulary of Medical Subject Headings (MeSH) from PubMed, including subheadings, publication types and supplementary concepts, was used. The search was performed between 24 February 2012 and 19 March 2012. A systematic search of the grey literature was performed 25–29 June 2012.
In the main database search, three sets of entry terms were applied and combined ( figure 1 ). The first set of entry terms describes major incidents. The second set of entry terms describes templates. In addition to the MeSH terms in the first two sets, a third set of entry terms with free search phrases was included. For the grey literature search, only two sets of entry terms were combined. 19
- Download figure
- Open in new tab
- Download powerpoint
Search strategy. The two first sets of entry terms consisted of 15 terms each, and the third set of eight free search phrases. Combining these three sets resulted in 225 individual searches in each database. *If any of the individual searches returned more than 700 results, the search was performed again with a fourth entry term ( disaster prevention ) using the Boolean operator AND.
Inclusion criteria
Templates reporting prehospital major incident medical management.
Templates published after 1 January 1990 (inclusive) and until the date of the literature search.
Exclusion criteria
All non-English language literature, except Scandinavian.
Literature without an available abstract.
Literature reporting only psychological aspects.
Deviations from protocol in search strategy
Combining the three sets of entry terms resulted in 225 individual searches in each database. If any of these individual searches returned more than 700 results, the search was performed again with a fourth entry term ( disaster prevention ) using the Boolean operator AND ( figure 1 ).
In Scopus, two entry terms, “questionnaires” and “learning”, were excluded due to a large number of irrelevant results, and all searches were limited to the subject areas of life sciences, health sciences and physical sciences. Searches in Scopus were further limited to article title, abstract and keywords. In Web of Knowledge (ISI), all searches were limited to articles and reviews. The term “disaster prevention” was used to refine and decrease the number of search results in four of the individual searches performed in this database. ProQuest Research Library was excluded as it returned too many irrelevant results and the most relevant subjects were covered by the searches performed in PubMed/MEDLINE, Web of Knowledge and Scopus.
The grey literature databases revealed a broad range of quality and searchability. The System for Information on Grey Literature in Europe (OpenSIGLE) was excluded due to the need to order the documents from the original source or a library. Only the document title was available on the web page, making it difficult to determine which documents to order.
Deviations in the search strategy were necessary in order to make the systematic literature review feasible, as a larger number of findings might have made completion of the study impossible.
Search findings
The search was performed according to the deviations described above. A total of 10 136 results from each individual database search were sent to Endnote X5 (Thomson Reuters, New York, USA). After removing duplicates, the number of results was 8497. The grey literature search returned 107 results ( figure 2 ). A total of 18 articles were included for data extraction and quality appraisal.
Flow diagram depicting the different stages of the systematic literature review.
Analysis of identified literature
One author scanned the titles and abstracts of the identified literature. Literature not complying with the inclusion criteria was excluded. The full text was obtained for uncertain articles, and inclusion was subject to consensus among three of the authors. Data analysis was performed according to the participants, interventions, comparisons, outcomes, study design (PICOS) methodology as described in PRISMA guidelines. 20 , 21 In this case, the participants were all the identified templates for reporting major incident medical management. Our intervention, comparisons and outcomes were carried out using the data extraction and quality appraisal variables described in methods and depicted in figures 3 and 4 . From each template, 34 data items were extracted according to a predefined set of questions described in the study protocol 19 ( figure 3 ). These data were classified into four categories: demographics, incident characteristics, system characteristics and descriptors of patient characteristics. After data extraction, quality appraisal was conducted using a checklist 22 designed by the authors prior to data collection 19 ( figure 4 ). This checklist was based on authors’ assumptions of the data relevant to report in a template. One author performed data extraction and quality appraisal; the results were checked by a second author. The contact authors of articles that provided an email address were asked whether the template had been used in real-life incidents. The reference lists of the included literature were scanned and relevant literature included. A quantitative synthesis (meta-analysis) was not performed. The preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines were followed. 20 , 21
Data extraction from the included literature. ✓ Yes; × no; MI, major incident.
Quality appraisal of the included literature. ✓ Yes, × no, ? unclear, * study is ongoing.
Deviations from protocol regarding quality appraisal
The study protocol proposed to appraise whether the medical outcomes predicted by the templates were valid and to evaluate the outcome of using the templates. Both of these questions proved difficult to answer and were removed from the appraisal.
A total of 12 articles were included from the main database search, 23–34 five articles were identified from the literature lists of included articles, 35–39 and one relevant article was published after the literature search was completed 12 ( figure 2 ). The total of 18 articles included 10 different templates or guidelines for reporting ( figure 5 ).
An overview of the included literature.
Data extraction
The results of data extraction are shown in figure 3 . In addition, under each of the four categories (pre-event information, incident information, system characteristics, patient characteristics), other information that was not extracted by the predefined questions was registered. Six of the 10 templates contained other pre-event information, such as climate, child mortality rate and descriptions of hazards. Seven of the 10 templates contained other incident information, such as a description of the incident. All templates included more system characteristics than what we extracted, including on-site medical care, distribution of casualties, independent action by medical disaster response personnel, continuation of day-to-day care, decision flow and information management. Seven templates contained other descriptors of patient characteristics, such as different triage systems used, description of psychological reactions and morbidity using hospital data.
Quality appraisal
The appraisal using a predefined checklist is shown in figure 4 . The first five questions regarding internal validity indicated that two of the templates contained none of the data we were looking for, four templates contained one of the data items we found relevant and the remaining four templates included three or more data items included in our list of desirable information. The 11 items regarding external validity were also heterogeneous in regard to which and how many of the items each template contained.
Use of templates
We succeeded in contacting the authors of seven templates. According to the authors, five of these templates 24–30 , 31 , 33–35 , 37 , 39 were used in other publications and one 12 is currently being used to retrospectively evaluate disaster management. One has not been used in other publications. 32 DISAST-CIR 24–30 , 35 is routinely used to report each mass casualty incident in the registry of the Israeli Defence Force Home Front and Ministry of Health. Guidelines for reporting health crises and critical health events 37 have been used to report international disasters, but these publications were not available as official publications at the time of correspondence with the authors. The protocol for reports of major accidents and disasters 33 was published previously in the International Journal of Disaster Medicine 40 , 41 and used for a report in this journal. 42 It has also been used in the European Journal of Trauma and Emergency Surgery 43 , 44 and mentioned in an editorial in the same journal. 45 Two of the templates 31 , 34 , 39 are routinely used for reporting from exercises. Data on medical management during a mass casualty incident exercise 31 are normally used to assess the healthcare system in a mass casualty incident simulation and exercise. An online registry for the healthcare system is designed using this instrument. Performance indicators for major incident medical management 34 , 39 have also been used in additional publications. 17 , 46–50 For the three publications lacking author email, 23 , 36 , 38 we were unable to attain information on whether they have been used.
We identified 10 templates for reporting prehospital major incident medical management that were heterogeneous with regard to the data they reported. The quality appraisal revealed that, for most of the templates, the methodology for developing them was not clearly explained. In addition, the data variables were not clearly defined for all templates, and the rationale for choosing the data variables was only explained for half of the templates. Only three of the articles describe the handling of missing data and two depict whether an ethics committee approved the templates. All of these factors are important for internal validity, but the results were also heterogeneous for external validity. We chose to interpret that the templates were developed in the regions affiliated with the authors, though this was not specified. Only two templates stated in which region they were intended to be used. None of the articles discussed the clinical credibility of the template, and no feasibility studies have been performed. In all cases, the use of the template as a tool for evaluation was mentioned.
The data extraction and quality appraisal variables were based on the authors’ assumptions on what is important in a template for reporting major incident medical management. Data variables for reporting should be uniformly defined in order to improve research and allow analysis of data; this is the ideological basis for several previous projects to standardise data for scientific use. 9–12 We also believe it is important that templates are preapproved by ethics committees to allow immediate reporting and rapid dissemination of data on the potential for improvement. For a template to be used, it needs to be both clinically credible and feasible. Ideally, if a template is to be used in a specific region, it ought to be developed together with experts from that region; if this is not possible, feasibility studies regarding regional differences could be performed. Reporting should be done by representatives with in-depth local knowledge and directly involved in responding to or managing the major incident. The ultimate goal of reporting is that an evaluation of the response be undertaken to identify areas for improvement, enabling those responsible in similar settings to improve their preparedness. For this kind of evaluation to occur, comparable, standardised reports that allow for research need to be published. Thus far, reporting on the scale needed for comparisons has not been achieved.
Limitations
Not all of the included literature was intended for prospectively reporting real-life incidents. However, in order to not overlook potentially relevant aspects of major incident reporting, the literature aimed to report from exercises 31 , 34 , 36 , 39 and literature using a systematic method for reporting in general was included. 23 , 32 A clear weakness was that templates may exist in languages other than those included. We invite others to identify these templates. Only literature in which an abstract was available was included. With more that 8000 articles identified in the search, reviewing full articles at the initial stage was not feasible. Another limitation was that only one author performed the initial review of the literature for inclusion. One author performed data extraction and the appraisal and a second author checked the results, but this can still allow room for subjective interpretations of the content of the templates. The aim of the appraisal was to systematically extract information that the authors thought would be important for reporting major incident medical management. However, neither the data extraction nor the quality appraisal represents a validated set of data or gold standard.
Conclusions
Our findings show that more than one template exists for generating reports from the medical management of major incidents. Limitations are present in the existing templates regarding internal and external validity, and none of them have been tested for feasibility in real-life incidents. Uniform reporting can allow the analysis and comparison of medical management for different major incidents and identify areas that need improvement. Indirectly, this information can lead to better resource use and improved outcomes for patients and society. The identified templates may be used as a basis for designing a template that is specifically aimed at prehospital medical care and at generating reports in such a quantity that comparative analysis can be performed. The work to create such a template seems warranted and is now underway.
- Guha-Sapir D ,
- Ashton RW ,
- Farmer JC ,
- Stratton SJ
- Castren M ,
- Hubloue I ,
- Lennquist S
- Langhelle A ,
- Herlitz J ,
- Ringdal KG ,
- Lefering R ,
- Debacker M ,
- Thompson J ,
- Mackway-Jones K ,
- Legemaate GA ,
- Burkle FM Jr . ,
- Radestad M ,
- Nilsson H ,
- Reierth E ,
- Liberati A ,
- Tetzlaff J ,
- Altman DG ,
- Whiting P ,
- Harbord R ,
- Anderson PB
- Schwartz D ,
- Pinkert M ,
- Bar-Dayan Y
- Ostfeld I ,
- Zaltsman E ,
- Ingrassia PL ,
- Juffermans J ,
- Kulling P ,
- Birnbaum M ,
- Backman K ,
- Albertsson P ,
- Petterson S ,
- Peclard E ,
- Marres GMH ,
- France JM ,
- Nichols D ,
- Örtenwall P ,
- Vilkström T
Contributors SF, MR and TW conceived the idea and designed the study. ER designed and conducted the search strategy for the literature search. SF screened the identified literature. TW, MR and SF considered the eligibility of uncertain literature. SF performed data extraction and quality appraisal of the included literature. TW and MR checked these results and were mentors in the process. SF, MR, ER and TW approved the final version of the manuscript.
Funding The Norwegian Air Ambulance employs SF and MR as research fellows. ER and TW received departmental funding only. No additional funding was obtained.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Read the full text or download the PDF:
- Macquarie University Library
- Subject and Research Guides
Systematic Reviews
- Step 10: Templates, Write & Publish
- Step 1: Check Protocols & Guides
- Step 2: Form a Question
- Step 3: Develop a Search Strategy & Criteria
- Grey Literature
- Documenting the Search
- Step 5: Export Results with EndNote, Mendeley
- Step 6: PRISMA Flow Diagram & Screen
- Step 7: Extract Data
- Step 8: Appraise Studies & Assess Risk of Bias
- Step 9: Synthesise & Interpret, Meta-analyses
- Non-Health Systematic Reviews
Writing Your Review
When writing up your systematic review keep in mind the specific guidelines for structuring your review. Systematic review standards are elements that should be reported in any published systematic review. Also there may be other 'Instructions to Authors' provided by the journals or organisations in which you plan to publish.
The PRISMA statement can be followed to ensure reliable coverage of the systematic review methods, results and conclusions. The PRISMA Elaboration and Explanation (E&E) document that accompanies the PRISMA statement provides instructions for each of the items on the checklist.
- PRISMA Statement
- PRISMA Elaboration and Explanation Document
- Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement
Strategic Publishing
- RV2 Checklist To help make informed decisions before publishing, MQ has developed the RV2 checklist to assist with selecting appropriate peer-reviewed journals.
- Strategic Publishing Guide Strategic publishing guide by DVC-R and the Library
- Think Check Submit Think. Check. Submit. helps researchers identify trusted journals for their research. Through a range of tools and practical resources, this international, cross-sector initiative aims to educate researchers, promote integrity, and build trust in credible research and publications.
Templates for SRs
This link is to best practice examples from Cochrane on how to write about your searches in a Cochrane review.
You can adapt examples to suit your systematic review.
- Search Text Templates for Reviews and Updates
Further reading
- Article: Choosing the right journal for your systematic review
- << Previous: Step 9: Synthesise & Interpret, Meta-analyses
- Next: Non-Health Systematic Reviews >>
- Last Updated: Feb 24, 2023 4:29 PM
- URL: https://libguides.mq.edu.au/systematic_reviews
- Staff portal
Systematic style literature reviews for education and social sciences
- Different types of literature review
- Developing the research question
- Search strategies
- Recording your systematic searching
- Systematic reading of the literature
- Writing your systematic literature review
- Software tools
- Citing your sources
Overview of Systematic-style Literature Reviews
This is a guideline for writing a systematic qualitative or quantitative literature review chapter. S ystematic literature reviews concentrate most of the writing in describing the methods used to collect, analyse and present the literature, so it is a good place to start. As with all academic work, remember that all of your material should be referenced and permission to reproduce charts and images obtained.
Before you start:-
- Speak with your supervisor to ensure that their requirements are met and that specific elements of your literature review and research are covered.
- If you are writing a journal article, then you should check the journal author guidelines before you start the writing process.
If you would like to know more about systematic reviews, please see the introduction to this guide .
If you are writing a systemic review for health please use the information on the Systematic-style Reviews page.
When to use a quantitative writing approach
If you are writing a scoping review, systematic quantitative, integrative, iterative, rapid, umbrella, meta-analysis or meta-synthesis then use the quantitative approach to writing the literature review. As these methods have different approaches and are applied to different discipline areas, you may need to combine it with a qualitative style literature review.
For Health research, like Cochrane Reviews or Campbell Collaborations, then use the information on the Systematic-style Reviews page. For narrative literature reviews please use the Griffith Graduate Research Guide or the Library Writing Guide .
Quantitative style literature review
Introduction
- Results & Analysis
- Results & Analysis Structure:
- Inserting Journal Articles
- Useful Resources
Outline: Systematic Quantitative Literature Review Chapter
The following is a simple pattern to follow for writing your systematic quantitative literature review. Specific sections are detailed in each tab. As with all academic work, remember that all material should be referenced and permission to reproduce charts and images obtained.
Introduction ( 15-20% of your total available word count )
Insert Journal A rticle (75 - 80% of word count)
*If you are embedding an article then it would replace the remaining sections and you would include a chapter conclusion
Methods ( 15 – 2 5 % of word count)
Results & Analysis (these can be separate chapters or sections) 2 0 - 25 %
Discussion (20% of word count)
Conclusion (3 – 5% of word count)
When writing your quantitative literature review it i s easier to start with the methods and then the introduction .
S tate the topic for review , the type of systematic review you are undertaking; quantitative or qualitative , weighted, meta - analysis or meta- synth esis and include whether this is a new or updated review .
Describe the hypothesis, theory or problem being tested and identify the aims of the review
State the research questions and expected outcome
Explain the significance of the review to your discipline and audience
Define terms you are using, including debates and agreements about the terminology
Describe the research frontiers or state of the field being research ed
Describe how you have grouped the literature in terms of theory, topics, chronology, methodology, or findings and order you have placed your main sections of writing in
This section is crucial because it provides a full , transparent explanation o f the research method by using summari es and descri ptions of the protocol s used to find , filter, analyse and quality test the literature . This is important as it provides a way for you to describe how comprehensive your review was. It also provides future researchers with a way to reproduce an update d literature review for the same topic . It may not be needed in your final submitted thesis but may be used to inform future journal articles .
Section introduction
Name the t ype of literature review , research topic and aims.
State time frames if they are relevant
State any guidelines or the reporting framework you have used e.g., Prisma , AMSTAR
Indicate the process for developing your research question , e.g., pico , spider
If your review is repeating an existing protocol, updat ing a review protocol or is a new protocol
The logical order that you have placed your writing in
Section Body
Describe t he system you used to find and collect the literature . Use tables or diagrams for data and refer to them from the text. If they are too large to place in the text, place in appendices and refer to the appendix.
If you have regist e r ed the protocols of the review (PROSPERO) , provide the details
Relevant search strings, key terms, filters, and refinements you used
T ools you used, where you searched, and which databases or search engines you us ed
Explain your i nclusion and exclusion criteria, key themes you were reading for , or changes in criteria
*if this is an updated review, describe revisions made to the protocol, why they were made and the effect.
O utline the number of documents found and then kept
Explain t he system used to critically appraise articles you used from your literature search
H ow you assessed quality of literature, for e.g., primary studies or transparency of data
Categories or variables you used to compare content across the literature for e.g., costs, outcomes, drug concentrations
Whether any articles were subsequently excluded from your review and the reason for exclusion
Explain subsequent add itions or updates after the initial search
Articulate any bias in the methods you have used and the effect on the data, results, findings, or outcomes .
Section Summary
Condense the key criteria or limitations , relate them to your research aims or questions, state the relevance to your overall thesis claim .
Section i ntroduction (approximately 1 paragraph)
Outline the literature you kept .
Location, authors, subject areas, types of research, research methods used in literature … count and calculate percentages (of xyz ) of coverage on topics, methods, or oth er study characteristics
Name what variables you were look ing for (if applicable)
Where the frontier of research is (found literature)
Where the gaps are in the research (literature not found)
Fresh perspectives on existing literature
S ummarise and s tate the most important variables you found in the literature . The categories will depend on your discipline, topic, purpose, and search results.
Use tables and graphs to represent the most important literature and concepts. Place large data sets into appendices and use links to raw data .
This article about types of tables commonly used may be helpful.
This article includes examples of figures ( pictograms and graphs) that may be helpful.
Section body
Provide an o verview of the literature you included using sentences and paragraphs. Then insert a logical series of paragraphs, about 1-3 for each section, explaining the literature and limits (see the Results and Analysis Structure tab).
Section summary (approximately 1 paragraph)
S ummarise the key results, relate the results to your research aims or questions, state the relevance to your overall thesis claim .
Results & analysis of the literature
There are differences between academic disciplines about addressing analysis and results at the same time or having two separate sections. Check with your supervisor or consult previously published literature reviews in your field. Essentially, these sections are about what you found and how you categorised the data. This changes with each discipline and field, but common options are: -
Research aims
Key variables
Research question
Research question and then sub-themes in the data
Theory elaboration
Science & Health tend to o rganise results by research questions and apply a similar logical sequence to each answer .
R esearch Q uestion 1 : state the question
Describe the # of studies involved, study characteristics, participant characteristics, efficacy of intervention or efficacy of treatment , therapy, or pharmaceuticals.
Describe each sub-theme from the literature you found about the research question.
or
Theoretical application 1 : state the i mpact on the data or system .
Describe research purpose, research questions or aims. State the main ideas found in the literature . This could be the efficacy of a drug-x, new perspectives or highlight key contradictory data . Outline the order you have put the writing in. This could be aims, themes or questions – it will depend on the discipline norms .
Body of discussion
I mportant to write about:
T he results from the review you have conducted and how they apply to what has been done in other studies
T he limitations of studies included in your review
If relevant to your research topic and questions:
G ive suggestions how contradictions may be resolved by future research
H ow this review has (or has not) generated a novel perspective on the subject
This c hanges with each discipline and field , but c ommon options are: -
R esearch aims – often rephrased as answers and literature applied to the aims . Rørtveit et al. (2015), provide a useful example of this structure.
K ey variables – often summarised and compared across the research you found. An example of this structure is used by Feldman et al. (2021) .
R esearch question – summary of relevant literature applied to the research question . Gauffriau (2021 ) provides a good example of this structure.
Research question and then sub-themes in the literature you found. Slabbert and Du Perez (2021) use this type of structure.
Theory elaboration – explain how the theory you are using applies to the research data . Pitt et al. (2017) use this structure.
If relevant to your research topic and questions:
State limitations of the review and how it affects the reliability of the whole review
Future direction of research
( T hese elements can appear separately or be integrated into the structured response )
A brief and direct interpretation may be made by summarising
Main findings identified from the research (present study)
Evidence of research gap s (future research that may be expanded in future chapters )
Any new themes emerging from the data of your review
Implications for future research , professional or clinical practice
Introduction ( 15 - 20% of word count)
Describe how you have grouped the literature in terms of theory, topics, chronology, methodology, or findings and order you have placed your main sections of writing in
Insert Journal A rticle
If you have published your systematic quantitative literature review, then inse rt the article in here .
Please confirm the requirements for including articles in a thesis for your academic discipline with your supervisor and the publisher.
Conclusion (3 – 5% of word count )
A brief and direct interpretation may be made by summarising:
Implications for future research , professional or clinical practice
L ink the article to the chapter, thesis argument and the remaining chapters in the thesis.
Abstract (3-5% of total word count)
Abstracts are used for journal articles . They appear first in the article but are written last. These are either a narrative style or structured by sub-titles. If you have editorial or publication guidelines, then follow those guidelines.
If you are writing a chapter in a thesis then they usually do not have an abstract. However, similar information would likely be included in the chapter introduction.
Narratives may include: -
What specific problem or research question was being addressed
Why the research was conducted
Research Aims
W hat methods were used to solve the problem or to answer the question
What res ults were obtained
What the results mean to your discipl ine and reader
State your argument or claim
Structured abstracts may include: -
Background – previous research
Aim – purpose of the study
Method – the system and summary of process
Results – what was found
Conclusions – how your results meet your aims
Value/Recommendations – what is new or of value to your discipline, practical implications, or limitations
Useful writing links :
Piper, R. (2013). How to write a systematic literature review: a guide for medical students. National Student Association for Medical Research.
RMIT Systematic Review Guides provide useful guide and annotations of medical examples of systematic reviews.
Griffith University style guide provides information about publication and tone of Griffith documents
Advice about conducting systematic reviews:
The Cochrane Library
Two recommended checklists for critical appraisal of study design; CAMARDADES and CONSORT
Systematic Quantitative Literature Review (SQLR) method
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Griffith University RED workshops about Meta-analysis, meta-synthesis, & multiverse analyses and systematic style reviews are available
A British Medical Journal article on meta-analysis
Qualitative style literature review
Outline: Systematic Qualitative Literature Review Chapter
The following is a simple pattern to follow for writing your systematic qualitative literature review. Specific sections are detailed in each tab. As with all academic work, remember that all material should be referenced and permission to reproduce charts and images obtained.
Introduction ( 15 - 20% of word count)
Results & Analysis of the literature ( 2 0 - 25 % of word count) these can be separate sections
Discussion ( 20% of word count)
When writing your qualitative literature review it i s sometimes easier to start with the methods .
S tate the topic for review , the type of systematic review you are undertaking ( i.e., quantitative or qualitative , weighted, meta - analysis or meta- synth esis or other type of review ) and include whether this is a new or updated review .
For a qualitative review, d escribe the questions that are being asked to explore the central phenomen on or concept under review and identify the aims and expected outcome of the review .
Explain the significance of the review to your academic field and audience . How will it contribute to our understanding of this topic?
Describe the research frontiers or state of the field being research ed .
Define terms you are using, including debates and agreements about the terminology (this part may need separate paragraphs).
Describe how you have grouped the literature in terms of theory, topics, chronology, methodology, or findings . Provide a preview of t he order of the main sub sections in the literature review .
This section is crucial because it provides a full , transparent explanation o f the research method . S ummari es and descri ptions of the protocol s used to find , filter, analyse and quality test the literature are described . Not only does this information add credibility to your research, it also provides a way for future researchers to reproduce an update d literature review for the same topic . It may be used to inform journal articles , which may be included in your thesis .
S ection introduction (1 paragraph, 5-8 sentences)
Outline the t ype of literature review , topic and aims .
State time frames if they are relevant .
State any guidelines or the reporting framework you have used e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) .
Indicate the process for developing your research question , including any standardised systematic search strategies e.g., patient/population, intervention, comparison, outcomes (PICO) , or Sample, Phenomenon of Interest, Design, Evaluation, Research type (SPIDER) .
Indicate whether this is repeating an existing protocol, an update of a review protocol , or a new protocol .
Justify the choice and sequence of the subsections in this section .
S ection body (1 or more paragraphs)
Describe t he system you used to find and collect the literature . Use tables or diagrams for data , then cross-reference them in your paragraphs by refer ring readers to them. If they are too large to place in the text, place in appendices and refer readers to the appendix.
If you r topic is health-related, and you have regist e r ed the protocols of the review (PROSPERO) , provide the details .
Using sentences, d escribe r elevant search strings, key terms, filters and refinements you used .
Detail the t ools you used, where you searched, and which databases or search engines you us ed .
Describe the c riteria used to analyse and quality test the literature .
Explain your i nclusion and exclusion criteria, key themes you were reading for , or changes in criteria .
I f this is an updated review, describe revisions made to the protocol, why they were made and the effect.
O utline the number of documents found and then kept for each search string .
Explain t he critical appraisal system , including h ow you evaluated the studies f ound in your literature search , e.g., JBI critical appraisal checklist for qualitative research
- List the c ategories used to compare content across the literature e.g. outcomes , social aspects or competencies .
State w hether any articles were subsequently excluded and the reason for exclusion .
Explain subsequent add itions or updates after the initial search .
Ar ticulate any bias in the methods you have used for collecting, analysing or presenting information . E xplain how you managed this , as well as any potential effect on the data, results, findings or outcomes.
S ection summary (1 paragraph, 5 - 8 sentences)
Condense the key criteria or limitations , relate them to your research aims or questions, and state the relevance of the whole section to your main claim or academic argument .
Section i ntroduction (approximately 1 paragraph)
Describe the type of literature review, topic and aims.
Describe the theory , situation or problem being addressed .
Indicate the main research question s and expected outcome .
Explain the significance of your results to your academic field and audience .
Describe what this section addresses, results , analysis, or both .
Describe the logic you have used to organise the section .
For each theme o utline and describe: (approximately 1 paragraph)
L ocation, authors, subject areas, types of research, research methods used across the literature
W here the research frontier s are within the literature you found
W here the gaps are in the research (literature you didn’t find )
F resh perspectives on existing literature
T he most important themes you found in the literature . The categories will depend on your discipline, topic, purpose and search results.
T he reasoning for prioritising the themes or sub-themes.
Condense the key results , relate the results to your research aims or questions and state the relevance to your overall claim.
T h ere are differences between academic disciplines about address ing both elements at the same time or having two separat e sections . Check with your supervisor or look at published literature reviews in your field. Essent ially, t hese sections are about what you found and how you categorised the data. Interpretation of the findings is usually found in the Discussion section. Qualitative literature reviews are usually organised thematically.
Literature themes:
Thematically organised literature tend s to apply the same logical sequence of material to each theme.
Several themes may need to be articulated. Each one c ould have its own sub- section and sub-heading .
Structures :
There ar e two commonly used options.
Theme 1
Describe the number of studies involved, study or research design, effect or outcome of the research included (this will depend on what you aimed to achieve) and its rela tionship to your research topic or method . A useful example of this structure is used by Macaro et al., (2017 , p. 46 ) in their article .
Theme 2
Describe what you expected to find, what you didn’t find, then what you did find . A useful example of this structure is used by Yang et al. ( 2017, p. 96 ) in their article.
T ables and figures :
Create subsets of your data (literature you found) to represent the most important literature and concepts .
Place raw or large data sets into appendices and use links to raw data .
Use figures or tables to illustrate your process and findings.
Discussion s ection introduction (1 paragraph, 5-8 sentences) :
Summarise the research topic, purpose and aims.
Describe the theory, situation or problem being addressed .
State the main ideas in the discussion .
Outline the logic al order you have used to organise the section ; usually this is a similar logic to the results and analysis section – it will depend on the norms of your academic disciplin e
Discussion section b ody (1- 3 paragraphs) :
It is i mportant to include the following :
Provide a general interpretation of the main findings, including the strength of evidence.
Apply the results from the review you have conducted to what has been done in other studies .
I dentify and discuss the limitations of studies included in the review .
Consistent structure must be applied throughout the body to help readers navigate your writing .
Structures: there are four commonly used options
Research aims – often rephrased as answers and literature applied to the aims Rørtveit et al. (2015, p. 205 ), provide a useful example of this structure.
Key themes – often literature is summarised and applied to research questions or aims of the research like Cantali , (2019, p. 46)
Research question – summary of relevant literature applied to each research question Macaro et al., (2017 , p. 66 ) have a good example in their article.
Theory elaboration – explain how the theory you are using applies to the research data . Pitt et al. (2017, p. 2402) provide a useful example of this structure.
Discussion section summary ( 1 paragraph )
A brief and direct interpretation may be made by summarising:
Main findings identified from the research (present study) .
Evidence of research gap s (future research) .
Any new themes emerging from the data of your review .
Implications for future research o r clinical or professional practice .
A brief and direct interpretation may be made by summarising:
Main findings identified from the research (present study) .
Evidence of research gaps (future research) .
Any new themes emerging from the data of your review .
Limitations of the review .
Implications for future research or clinical or professional practice .
Insert Journal A rticle ( 75 - 80% of word count)
If you have published your systematic literature review, then inse rt the article in here .
Abstract (3 - 5% of total word count)
Griffith University RED workshops about qualitative systematic reviews, m eta-analysis, meta-synthesis, & multiverse analyses and systematic style reviews are available .
Campbell Collaboration for Education
Two recommended checklists for critical appraisal of study design; SPIDER
- << Previous: Systematic reading of the literature
- Next: Software tools >>
- Last Updated: Nov 1, 2022 10:43 AM
- URL: https://libraryguides.griffith.edu.au/systematic-literature-reviews-for-education
- Open Access
- Published: 01 August 2019
A step by step guide for conducting a systematic review and meta-analysis with simulation data
- Gehad Mohamed Tawfik 1 , 2 ,
- Kadek Agus Surya Dila 2 , 3 ,
- Muawia Yousif Fadlelmola Mohamed 2 , 4 ,
- Dao Ngoc Hien Tam 2 , 5 ,
- Nguyen Dang Kien 2 , 6 ,
- Ali Mahmoud Ahmed 2 , 7 &
- Nguyen Tien Huy 8 , 9 , 10
Tropical Medicine and Health volume 47 , Article number: 46 ( 2019 ) Cite this article
549k Accesses
140 Citations
93 Altmetric
Metrics details
The massive abundance of studies relating to tropical medicine and health has increased strikingly over the last few decades. In the field of tropical medicine and health, a well-conducted systematic review and meta-analysis (SR/MA) is considered a feasible solution for keeping clinicians abreast of current evidence-based medicine. Understanding of SR/MA steps is of paramount importance for its conduction. It is not easy to be done as there are obstacles that could face the researcher. To solve those hindrances, this methodology study aimed to provide a step-by-step approach mainly for beginners and junior researchers, in the field of tropical medicine and other health care fields, on how to properly conduct a SR/MA, in which all the steps here depicts our experience and expertise combined with the already well-known and accepted international guidance.
We suggest that all steps of SR/MA should be done independently by 2–3 reviewers’ discussion, to ensure data quality and accuracy.
SR/MA steps include the development of research question, forming criteria, search strategy, searching databases, protocol registration, title, abstract, full-text screening, manual searching, extracting data, quality assessment, data checking, statistical analysis, double data checking, and manuscript writing.
Introduction
The amount of studies published in the biomedical literature, especially tropical medicine and health, has increased strikingly over the last few decades. This massive abundance of literature makes clinical medicine increasingly complex, and knowledge from various researches is often needed to inform a particular clinical decision. However, available studies are often heterogeneous with regard to their design, operational quality, and subjects under study and may handle the research question in a different way, which adds to the complexity of evidence and conclusion synthesis [ 1 ].
Systematic review and meta-analyses (SR/MAs) have a high level of evidence as represented by the evidence-based pyramid. Therefore, a well-conducted SR/MA is considered a feasible solution in keeping health clinicians ahead regarding contemporary evidence-based medicine.
Differing from a systematic review, unsystematic narrative review tends to be descriptive, in which the authors select frequently articles based on their point of view which leads to its poor quality. A systematic review, on the other hand, is defined as a review using a systematic method to summarize evidence on questions with a detailed and comprehensive plan of study. Furthermore, despite the increasing guidelines for effectively conducting a systematic review, we found that basic steps often start from framing question, then identifying relevant work which consists of criteria development and search for articles, appraise the quality of included studies, summarize the evidence, and interpret the results [ 2 , 3 ]. However, those simple steps are not easy to be reached in reality. There are many troubles that a researcher could be struggled with which has no detailed indication.
Conducting a SR/MA in tropical medicine and health may be difficult especially for young researchers; therefore, understanding of its essential steps is crucial. It is not easy to be done as there are obstacles that could face the researcher. To solve those hindrances, we recommend a flow diagram (Fig. 1 ) which illustrates a detailed and step-by-step the stages for SR/MA studies. This methodology study aimed to provide a step-by-step approach mainly for beginners and junior researchers, in the field of tropical medicine and other health care fields, on how to properly and succinctly conduct a SR/MA; all the steps here depicts our experience and expertise combined with the already well known and accepted international guidance.
Detailed flow diagram guideline for systematic review and meta-analysis steps. Note : Star icon refers to “2–3 reviewers screen independently”
Methods and results
Detailed steps for conducting any systematic review and meta-analysis.
We searched the methods reported in published SR/MA in tropical medicine and other healthcare fields besides the published guidelines like Cochrane guidelines {Higgins, 2011 #7} [ 4 ] to collect the best low-bias method for each step of SR/MA conduction steps. Furthermore, we used guidelines that we apply in studies for all SR/MA steps. We combined these methods in order to conclude and conduct a detailed flow diagram that shows the SR/MA steps how being conducted.
Any SR/MA must follow the widely accepted Preferred Reporting Items for Systematic Review and Meta-analysis statement (PRISMA checklist 2009) (Additional file 5 : Table S1) [ 5 ].
We proposed our methods according to a valid explanatory simulation example choosing the topic of “evaluating safety of Ebola vaccine,” as it is known that Ebola is a very rare tropical disease but fatal. All the explained methods feature the standards followed internationally, with our compiled experience in the conduct of SR beside it, which we think proved some validity. This is a SR under conduct by a couple of researchers teaming in a research group, moreover, as the outbreak of Ebola which took place (2013–2016) in Africa resulted in a significant mortality and morbidity. Furthermore, since there are many published and ongoing trials assessing the safety of Ebola vaccines, we thought this would provide a great opportunity to tackle this hotly debated issue. Moreover, Ebola started to fire again and new fatal outbreak appeared in the Democratic Republic of Congo since August 2018, which caused infection to more than 1000 people according to the World Health Organization, and 629 people have been killed till now. Hence, it is considered the second worst Ebola outbreak, after the first one in West Africa in 2014 , which infected more than 26,000 and killed about 11,300 people along outbreak course.
Research question and objectives
Like other study designs, the research question of SR/MA should be feasible, interesting, novel, ethical, and relevant. Therefore, a clear, logical, and well-defined research question should be formulated. Usually, two common tools are used: PICO or SPIDER. PICO (Population, Intervention, Comparison, Outcome) is used mostly in quantitative evidence synthesis. Authors demonstrated that PICO holds more sensitivity than the more specific SPIDER approach [ 6 ]. SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) was proposed as a method for qualitative and mixed methods search.
We here recommend a combined approach of using either one or both the SPIDER and PICO tools to retrieve a comprehensive search depending on time and resources limitations. When we apply this to our assumed research topic, being of qualitative nature, the use of SPIDER approach is more valid.
PICO is usually used for systematic review and meta-analysis of clinical trial study. For the observational study (without intervention or comparator), in many tropical and epidemiological questions, it is usually enough to use P (Patient) and O (outcome) only to formulate a research question. We must indicate clearly the population (P), then intervention (I) or exposure. Next, it is necessary to compare (C) the indicated intervention with other interventions, i.e., placebo. Finally, we need to clarify which are our relevant outcomes.
To facilitate comprehension, we choose the Ebola virus disease (EVD) as an example. Currently, the vaccine for EVD is being developed and under phase I, II, and III clinical trials; we want to know whether this vaccine is safe and can induce sufficient immunogenicity to the subjects.
An example of a research question for SR/MA based on PICO for this issue is as follows: How is the safety and immunogenicity of Ebola vaccine in human? (P: healthy subjects (human), I: vaccination, C: placebo, O: safety or adverse effects)
Preliminary research and idea validation
We recommend a preliminary search to identify relevant articles, ensure the validity of the proposed idea, avoid duplication of previously addressed questions, and assure that we have enough articles for conducting its analysis. Moreover, themes should focus on relevant and important health-care issues, consider global needs and values, reflect the current science, and be consistent with the adopted review methods. Gaining familiarity with a deep understanding of the study field through relevant videos and discussions is of paramount importance for better retrieval of results. If we ignore this step, our study could be canceled whenever we find out a similar study published before. This means we are wasting our time to deal with a problem that has been tackled for a long time.
To do this, we can start by doing a simple search in PubMed or Google Scholar with search terms Ebola AND vaccine. While doing this step, we identify a systematic review and meta-analysis of determinant factors influencing antibody response from vaccination of Ebola vaccine in non-human primate and human [ 7 ], which is a relevant paper to read to get a deeper insight and identify gaps for better formulation of our research question or purpose. We can still conduct systematic review and meta-analysis of Ebola vaccine because we evaluate safety as a different outcome and different population (only human).
Inclusion and exclusion criteria
Eligibility criteria are based on the PICO approach, study design, and date. Exclusion criteria mostly are unrelated, duplicated, unavailable full texts, or abstract-only papers. These exclusions should be stated in advance to refrain the researcher from bias. The inclusion criteria would be articles with the target patients, investigated interventions, or the comparison between two studied interventions. Briefly, it would be articles which contain information answering our research question. But the most important is that it should be clear and sufficient information, including positive or negative, to answer the question.
For the topic we have chosen, we can make inclusion criteria: (1) any clinical trial evaluating the safety of Ebola vaccine and (2) no restriction regarding country, patient age, race, gender, publication language, and date. Exclusion criteria are as follows: (1) study of Ebola vaccine in non-human subjects or in vitro studies; (2) study with data not reliably extracted, duplicate, or overlapping data; (3) abstract-only papers as preceding papers, conference, editorial, and author response theses and books; (4) articles without available full text available; and (5) case reports, case series, and systematic review studies. The PRISMA flow diagram template that is used in SR/MA studies can be found in Fig. 2 .
PRISMA flow diagram of studies’ screening and selection
Search strategy
A standard search strategy is used in PubMed, then later it is modified according to each specific database to get the best relevant results. The basic search strategy is built based on the research question formulation (i.e., PICO or PICOS). Search strategies are constructed to include free-text terms (e.g., in the title and abstract) and any appropriate subject indexing (e.g., MeSH) expected to retrieve eligible studies, with the help of an expert in the review topic field or an information specialist. Additionally, we advise not to use terms for the Outcomes as their inclusion might hinder the database being searched to retrieve eligible studies because the used outcome is not mentioned obviously in the articles.
The improvement of the search term is made while doing a trial search and looking for another relevant term within each concept from retrieved papers. To search for a clinical trial, we can use these descriptors in PubMed: “clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH terms] OR “clinical trial”[All Fields]. After some rounds of trial and refinement of search term, we formulate the final search term for PubMed as follows: (ebola OR ebola virus OR ebola virus disease OR EVD) AND (vaccine OR vaccination OR vaccinated OR immunization) AND (“clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH Terms] OR “clinical trial”[All Fields]). Because the study for this topic is limited, we do not include outcome term (safety and immunogenicity) in the search term to capture more studies.
Search databases, import all results to a library, and exporting to an excel sheet
According to the AMSTAR guidelines, at least two databases have to be searched in the SR/MA [ 8 ], but as you increase the number of searched databases, you get much yield and more accurate and comprehensive results. The ordering of the databases depends mostly on the review questions; being in a study of clinical trials, you will rely mostly on Cochrane, mRCTs, or International Clinical Trials Registry Platform (ICTRP). Here, we propose 12 databases (PubMed, Scopus, Web of Science, EMBASE, GHL, VHL, Cochrane, Google Scholar, Clinical trials.gov , mRCTs, POPLINE, and SIGLE), which help to cover almost all published articles in tropical medicine and other health-related fields. Among those databases, POPLINE focuses on reproductive health. Researchers should consider to choose relevant database according to the research topic. Some databases do not support the use of Boolean or quotation; otherwise, there are some databases that have special searching way. Therefore, we need to modify the initial search terms for each database to get appreciated results; therefore, manipulation guides for each online database searches are presented in Additional file 5 : Table S2. The detailed search strategy for each database is found in Additional file 5 : Table S3. The search term that we created in PubMed needs customization based on a specific characteristic of the database. An example for Google Scholar advanced search for our topic is as follows:
With all of the words: ebola virus
With at least one of the words: vaccine vaccination vaccinated immunization
Where my words occur: in the title of the article
With all of the words: EVD
Finally, all records are collected into one Endnote library in order to delete duplicates and then to it export into an excel sheet. Using remove duplicating function with two options is mandatory. All references which have (1) the same title and author, and published in the same year, and (2) the same title and author, and published in the same journal, would be deleted. References remaining after this step should be exported to an excel file with essential information for screening. These could be the authors’ names, publication year, journal, DOI, URL link, and abstract.
Protocol writing and registration
Protocol registration at an early stage guarantees transparency in the research process and protects from duplication problems. Besides, it is considered a documented proof of team plan of action, research question, eligibility criteria, intervention/exposure, quality assessment, and pre-analysis plan. It is recommended that researchers send it to the principal investigator (PI) to revise it, then upload it to registry sites. There are many registry sites available for SR/MA like those proposed by Cochrane and Campbell collaborations; however, we recommend registering the protocol into PROSPERO as it is easier. The layout of a protocol template, according to PROSPERO, can be found in Additional file 5 : File S1.
Title and abstract screening
Decisions to select retrieved articles for further assessment are based on eligibility criteria, to minimize the chance of including non-relevant articles. According to the Cochrane guidance, two reviewers are a must to do this step, but as for beginners and junior researchers, this might be tiresome; thus, we propose based on our experience that at least three reviewers should work independently to reduce the chance of error, particularly in teams with a large number of authors to add more scrutiny and ensure proper conduct. Mostly, the quality with three reviewers would be better than two, as two only would have different opinions from each other, so they cannot decide, while the third opinion is crucial. And here are some examples of systematic reviews which we conducted following the same strategy (by a different group of researchers in our research group) and published successfully, and they feature relevant ideas to tropical medicine and disease [ 9 , 10 , 11 ].
In this step, duplications will be removed manually whenever the reviewers find them out. When there is a doubt about an article decision, the team should be inclusive rather than exclusive, until the main leader or PI makes a decision after discussion and consensus. All excluded records should be given exclusion reasons.
Full text downloading and screening
Many search engines provide links for free to access full-text articles. In case not found, we can search in some research websites as ResearchGate, which offer an option of direct full-text request from authors. Additionally, exploring archives of wanted journals, or contacting PI to purchase it if available. Similarly, 2–3 reviewers work independently to decide about included full texts according to eligibility criteria, with reporting exclusion reasons of articles. In case any disagreement has occurred, the final decision has to be made by discussion.
Manual search
One has to exhaust all possibilities to reduce bias by performing an explicit hand-searching for retrieval of reports that may have been dropped from first search [ 12 ]. We apply five methods to make manual searching: searching references from included studies/reviews, contacting authors and experts, and looking at related articles/cited articles in PubMed and Google Scholar.
We describe here three consecutive methods to increase and refine the yield of manual searching: firstly, searching reference lists of included articles; secondly, performing what is known as citation tracking in which the reviewers track all the articles that cite each one of the included articles, and this might involve electronic searching of databases; and thirdly, similar to the citation tracking, we follow all “related to” or “similar” articles. Each of the abovementioned methods can be performed by 2–3 independent reviewers, and all the possible relevant article must undergo further scrutiny against the inclusion criteria, after following the same records yielded from electronic databases, i.e., title/abstract and full-text screening.
We propose an independent reviewing by assigning each member of the teams a “tag” and a distinct method, to compile all the results at the end for comparison of differences and discussion and to maximize the retrieval and minimize the bias. Similarly, the number of included articles has to be stated before addition to the overall included records.
Data extraction and quality assessment
This step entitles data collection from included full-texts in a structured extraction excel sheet, which is previously pilot-tested for extraction using some random studies. We recommend extracting both adjusted and non-adjusted data because it gives the most allowed confounding factor to be used in the analysis by pooling them later [ 13 ]. The process of extraction should be executed by 2–3 independent reviewers. Mostly, the sheet is classified into the study and patient characteristics, outcomes, and quality assessment (QA) tool.
Data presented in graphs should be extracted by software tools such as Web plot digitizer [ 14 ]. Most of the equations that can be used in extraction prior to analysis and estimation of standard deviation (SD) from other variables is found inside Additional file 5 : File S2 with their references as Hozo et al. [ 15 ], Xiang et al. [ 16 ], and Rijkom et al. [ 17 ]. A variety of tools are available for the QA, depending on the design: ROB-2 Cochrane tool for randomized controlled trials [ 18 ] which is presented as Additional file 1 : Figure S1 and Additional file 2 : Figure S2—from a previous published article data—[ 19 ], NIH tool for observational and cross-sectional studies [ 20 ], ROBINS-I tool for non-randomize trials [ 21 ], QUADAS-2 tool for diagnostic studies, QUIPS tool for prognostic studies, CARE tool for case reports, and ToxRtool for in vivo and in vitro studies. We recommend that 2–3 reviewers independently assess the quality of the studies and add to the data extraction form before the inclusion into the analysis to reduce the risk of bias. In the NIH tool for observational studies—cohort and cross-sectional—as in this EBOLA case, to evaluate the risk of bias, reviewers should rate each of the 14 items into dichotomous variables: yes, no, or not applicable. An overall score is calculated by adding all the items scores as yes equals one, while no and NA equals zero. A score will be given for every paper to classify them as poor, fair, or good conducted studies, where a score from 0–5 was considered poor, 6–9 as fair, and 10–14 as good.
In the EBOLA case example above, authors can extract the following information: name of authors, country of patients, year of publication, study design (case report, cohort study, or clinical trial or RCT), sample size, the infected point of time after EBOLA infection, follow-up interval after vaccination time, efficacy, safety, adverse effects after vaccinations, and QA sheet (Additional file 6 : Data S1).
Data checking
Due to the expected human error and bias, we recommend a data checking step, in which every included article is compared with its counterpart in an extraction sheet by evidence photos, to detect mistakes in data. We advise assigning articles to 2–3 independent reviewers, ideally not the ones who performed the extraction of those articles. When resources are limited, each reviewer is assigned a different article than the one he extracted in the previous stage.
Statistical analysis
Investigators use different methods for combining and summarizing findings of included studies. Before analysis, there is an important step called cleaning of data in the extraction sheet, where the analyst organizes extraction sheet data in a form that can be read by analytical software. The analysis consists of 2 types namely qualitative and quantitative analysis. Qualitative analysis mostly describes data in SR studies, while quantitative analysis consists of two main types: MA and network meta-analysis (NMA). Subgroup, sensitivity, cumulative analyses, and meta-regression are appropriate for testing whether the results are consistent or not and investigating the effect of certain confounders on the outcome and finding the best predictors. Publication bias should be assessed to investigate the presence of missing studies which can affect the summary.
To illustrate basic meta-analysis, we provide an imaginary data for the research question about Ebola vaccine safety (in terms of adverse events, 14 days after injection) and immunogenicity (Ebola virus antibodies rise in geometric mean titer, 6 months after injection). Assuming that from searching and data extraction, we decided to do an analysis to evaluate Ebola vaccine “A” safety and immunogenicity. Other Ebola vaccines were not meta-analyzed because of the limited number of studies (instead, it will be included for narrative review). The imaginary data for vaccine safety meta-analysis can be accessed in Additional file 7 : Data S2. To do the meta-analysis, we can use free software, such as RevMan [ 22 ] or R package meta [ 23 ]. In this example, we will use the R package meta. The tutorial of meta package can be accessed through “General Package for Meta-Analysis” tutorial pdf [ 23 ]. The R codes and its guidance for meta-analysis done can be found in Additional file 5 : File S3.
For the analysis, we assume that the study is heterogenous in nature; therefore, we choose a random effect model. We did an analysis on the safety of Ebola vaccine A. From the data table, we can see some adverse events occurring after intramuscular injection of vaccine A to the subject of the study. Suppose that we include six studies that fulfill our inclusion criteria. We can do a meta-analysis for each of the adverse events extracted from the studies, for example, arthralgia, from the results of random effect meta-analysis using the R meta package.
From the results shown in Additional file 3 : Figure S3, we can see that the odds ratio (OR) of arthralgia is 1.06 (0.79; 1.42), p value = 0.71, which means that there is no association between the intramuscular injection of Ebola vaccine A and arthralgia, as the OR is almost one, and besides, the P value is insignificant as it is > 0.05.
In the meta-analysis, we can also visualize the results in a forest plot. It is shown in Fig. 3 an example of a forest plot from the simulated analysis.
Random effect model forest plot for comparison of vaccine A versus placebo
From the forest plot, we can see six studies (A to F) and their respective OR (95% CI). The green box represents the effect size (in this case, OR) of each study. The bigger the box means the study weighted more (i.e., bigger sample size). The blue diamond shape represents the pooled OR of the six studies. We can see the blue diamond cross the vertical line OR = 1, which indicates no significance for the association as the diamond almost equalized in both sides. We can confirm this also from the 95% confidence interval that includes one and the p value > 0.05.
For heterogeneity, we see that I 2 = 0%, which means no heterogeneity is detected; the study is relatively homogenous (it is rare in the real study). To evaluate publication bias related to the meta-analysis of adverse events of arthralgia, we can use the metabias function from the R meta package (Additional file 4 : Figure S4) and visualization using a funnel plot. The results of publication bias are demonstrated in Fig. 4 . We see that the p value associated with this test is 0.74, indicating symmetry of the funnel plot. We can confirm it by looking at the funnel plot.
Publication bias funnel plot for comparison of vaccine A versus placebo
Looking at the funnel plot, the number of studies at the left and right side of the funnel plot is the same; therefore, the plot is symmetry, indicating no publication bias detected.
Sensitivity analysis is a procedure used to discover how different values of an independent variable will influence the significance of a particular dependent variable by removing one study from MA. If all included study p values are < 0.05, hence, removing any study will not change the significant association. It is only performed when there is a significant association, so if the p value of MA done is 0.7—more than one—the sensitivity analysis is not needed for this case study example. If there are 2 studies with p value > 0.05, removing any of the two studies will result in a loss of the significance.
Double data checking
For more assurance on the quality of results, the analyzed data should be rechecked from full-text data by evidence photos, to allow an obvious check for the PI of the study.
Manuscript writing, revision, and submission to a journal
Writing based on four scientific sections: introduction, methods, results, and discussion, mostly with a conclusion. Performing a characteristic table for study and patient characteristics is a mandatory step which can be found as a template in Additional file 5 : Table S3.
After finishing the manuscript writing, characteristics table, and PRISMA flow diagram, the team should send it to the PI to revise it well and reply to his comments and, finally, choose a suitable journal for the manuscript which fits with considerable impact factor and fitting field. We need to pay attention by reading the author guidelines of journals before submitting the manuscript.
The role of evidence-based medicine in biomedical research is rapidly growing. SR/MAs are also increasing in the medical literature. This paper has sought to provide a comprehensive approach to enable reviewers to produce high-quality SR/MAs. We hope that readers could gain general knowledge about how to conduct a SR/MA and have the confidence to perform one, although this kind of study requires complex steps compared to narrative reviews.
Having the basic steps for conduction of MA, there are many advanced steps that are applied for certain specific purposes. One of these steps is meta-regression which is performed to investigate the association of any confounder and the results of the MA. Furthermore, there are other types rather than the standard MA like NMA and MA. In NMA, we investigate the difference between several comparisons when there were not enough data to enable standard meta-analysis. It uses both direct and indirect comparisons to conclude what is the best between the competitors. On the other hand, mega MA or MA of patients tend to summarize the results of independent studies by using its individual subject data. As a more detailed analysis can be done, it is useful in conducting repeated measure analysis and time-to-event analysis. Moreover, it can perform analysis of variance and multiple regression analysis; however, it requires homogenous dataset and it is time-consuming in conduct [ 24 ].
Conclusions
Systematic review/meta-analysis steps include development of research question and its validation, forming criteria, search strategy, searching databases, importing all results to a library and exporting to an excel sheet, protocol writing and registration, title and abstract screening, full-text screening, manual searching, extracting data and assessing its quality, data checking, conducting statistical analysis, double data checking, manuscript writing, revising, and submitting to a journal.
Availability of data and materials
Not applicable.
Abbreviations
Network meta-analysis
Principal investigator
Population, Intervention, Comparison, Outcome
Preferred Reporting Items for Systematic Review and Meta-analysis statement
Quality assessment
Sample, Phenomenon of Interest, Design, Evaluation, Research type
Systematic review and meta-analyses
Bello A, Wiebe N, Garg A, Tonelli M. Evidence-based decision-making 2: systematic reviews and meta-analysis. Methods Mol Biol (Clifton, NJ). 2015;1281:397–416.
Article Google Scholar
Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.
Rys P, Wladysiuk M, Skrzekowska-Baran I, Malecki MT. Review articles, systematic reviews and meta-analyses: which can be trusted? Polskie Archiwum Medycyny Wewnetrznej. 2009;119(3):148–56.
PubMed Google Scholar
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.
Gross L, Lhomme E, Pasin C, Richert L, Thiebaut R. Ebola vaccine development: systematic review of pre-clinical and clinical studies, and meta-analysis of determinants of antibody response variability after vaccination. Int J Infect Dis. 2018;74:83–96.
Article CAS Google Scholar
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, ... Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.
Giang HTN, Banno K, Minh LHN, Trinh LT, Loc LT, Eltobgy A, et al. Dengue hemophagocytic syndrome: a systematic review and meta-analysis on epidemiology, clinical signs, outcomes, and risk factors. Rev Med Virol. 2018;28(6):e2005.
Morra ME, Altibi AMA, Iqtadar S, Minh LHN, Elawady SS, Hallab A, et al. Definitions for warning signs and signs of severe dengue according to the WHO 2009 classification: systematic review of literature. Rev Med Virol. 2018;28(4):e1979.
Morra ME, Van Thanh L, Kamel MG, Ghazy AA, Altibi AMA, Dat LM, et al. Clinical outcomes of current medical approaches for Middle East respiratory syndrome: a systematic review and meta-analysis. Rev Med Virol. 2018;28(3):e1977.
Vassar M, Atakpo P, Kash MJ. Manual search approaches used by systematic reviewers in dermatology. Journal of the Medical Library Association: JMLA. 2016;104(4):302.
Naunheim MR, Remenschneider AK, Scangas GA, Bunting GW, Deschler DG. The effect of initial tracheoesophageal voice prosthesis size on postoperative complications and voice outcomes. Ann Otol Rhinol Laryngol. 2016;125(6):478–84.
Rohatgi AJaiWa. Web Plot Digitizer. ht tp. 2014;2.
Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol. 2005;5(1):13.
Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14(1):135.
Van Rijkom HM, Truin GJ, Van’t Hof MA. A meta-analysis of clinical studies on the caries-inhibiting effect of fluoride gel treatment. Carries Res. 1998;32(2):83–92.
Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
Tawfik GM, Tieu TM, Ghozy S, Makram OM, Samuel P, Abdelaal A, et al. Speech efficacy, safety and factors affecting lifetime of voice prostheses in patients with laryngeal cancer: a systematic review and network meta-analysis of randomized controlled trials. J Clin Oncol. 2018;36(15_suppl):e18031-e.
Wannemuehler TJ, Lobo BC, Johnson JD, Deig CR, Ting JY, Gregory RL. Vibratory stimulus reduces in vitro biofilm formation on tracheoesophageal voice prostheses. Laryngoscope. 2016;126(12):2752–7.
Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355.
RevMan The Cochrane Collaboration %J Copenhagen TNCCTCC. Review Manager (RevMan). 5.0. 2008.
Schwarzer GJRn. meta: An R package for meta-analysis. 2007;7(3):40-45.
Google Scholar
Simms LLH. Meta-analysis versus mega-analysis: is there a difference? Oral budesonide for the maintenance of remission in Crohn’s disease: Faculty of Graduate Studies, University of Western Ontario; 1998.
Download references
Acknowledgements
This study was conducted (in part) at the Joint Usage/Research Center on Tropical Disease, Institute of Tropical Medicine, Nagasaki University, Japan.
Author information
Authors and affiliations.
Faculty of Medicine, Ain Shams University, Cairo, Egypt
Gehad Mohamed Tawfik
Online research Club http://www.onlineresearchclub.org/
Gehad Mohamed Tawfik, Kadek Agus Surya Dila, Muawia Yousif Fadlelmola Mohamed, Dao Ngoc Hien Tam, Nguyen Dang Kien & Ali Mahmoud Ahmed
Pratama Giri Emas Hospital, Singaraja-Amlapura street, Giri Emas village, Sawan subdistrict, Singaraja City, Buleleng, Bali, 81171, Indonesia
Kadek Agus Surya Dila
Faculty of Medicine, University of Khartoum, Khartoum, Sudan
Muawia Yousif Fadlelmola Mohamed
Nanogen Pharmaceutical Biotechnology Joint Stock Company, Ho Chi Minh City, Vietnam
Dao Ngoc Hien Tam
Department of Obstetrics and Gynecology, Thai Binh University of Medicine and Pharmacy, Thai Binh, Vietnam
Nguyen Dang Kien
Faculty of Medicine, Al-Azhar University, Cairo, Egypt
Ali Mahmoud Ahmed
Evidence Based Medicine Research Group & Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, 70000, Vietnam
Nguyen Tien Huy
Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, 70000, Vietnam
Department of Clinical Product Development, Institute of Tropical Medicine (NEKKEN), Leading Graduate School Program, and Graduate School of Biomedical Sciences, Nagasaki University, 1-12-4 Sakamoto, Nagasaki, 852-8523, Japan
You can also search for this author in PubMed Google Scholar
Contributions
NTH and GMT were responsible for the idea and its design. The figure was done by GMT. All authors contributed to the manuscript writing and approval of the final version.
Corresponding author
Correspondence to Nguyen Tien Huy .
Ethics declarations
Ethics approval and consent to participate, consent for publication, competing interests.
The authors declare that they have no competing interests.
Additional information
Publisher’s note.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional files
Additional file 1:.
Figure S1. Risk of bias assessment graph of included randomized controlled trials. (TIF 20 kb)
Additional file 2:
Figure S2. Risk of bias assessment summary. (TIF 69 kb)
Additional file 3:
Figure S3. Arthralgia results of random effect meta-analysis using R meta package. (TIF 20 kb)
Additional file 4:
Figure S4. Arthralgia linear regression test of funnel plot asymmetry using R meta package. (TIF 13 kb)
Additional file 5:
Table S1. PRISMA 2009 Checklist. Table S2. Manipulation guides for online database searches. Table S3. Detailed search strategy for twelve database searches. Table S4. Baseline characteristics of the patients in the included studies. File S1. PROSPERO protocol template file. File S2. Extraction equations that can be used prior to analysis to get missed variables. File S3. R codes and its guidance for meta-analysis done for comparison between EBOLA vaccine A and placebo. (DOCX 49 kb)
Additional file 6:
Data S1. Extraction and quality assessment data sheets for EBOLA case example. (XLSX 1368 kb)
Additional file 7:
Data S2. Imaginary data for EBOLA case example. (XLSX 10 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.
Reprints and Permissions
About this article
Cite this article.
Tawfik, G.M., Dila, K.A.S., Mohamed, M.Y.F. et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health 47 , 46 (2019). https://doi.org/10.1186/s41182-019-0165-6
Download citation
Received : 30 January 2019
Accepted : 24 May 2019
Published : 01 August 2019
DOI : https://doi.org/10.1186/s41182-019-0165-6
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Tropical Medicine and Health
ISSN: 1349-4147
- Submission enquiries: Access here and click Contact Us
- General enquiries: [email protected]
JEPS Bulletin
The Official Blog of the Journal of European Psychology Students
Writing a Systematic Literature Review
Investigating concepts associated with psychology requires an indefinite amount of reading. Hence, good literature reviews are an inevitably needed part of providing the modern scientists with a broad spectrum of knowledge. In order to help, this blog post will introduce you to the basics of literature reviews and explain a specific methodological approach towards writing one, known as the systematic literature review.
Literature review is a term associated with the process of collecting, checking and (re)analysing data from the existing literature with a particular search question in mind. The latter could be for example:
- What are the effects of yoga associated with individual’s subjective well-being?
- Does brief psychotherapy produce beneficial outcomes for individuals diagnosed with agoraphobia?
- What personality traits are most commonly associated with homelessness in the modern literature?
A literature review (a) defines a specific issue, concept, theory, phenomena; (b) compiles published literature on a topic; (c) summarises critical points of current knowledge about the problem and (d) suggests next steps in addressing it.
Literature reviews can be based on all sorts of information found in scientific journals, books, academic dissertations, electronic bibliographic databases and the rest of the Internet. Electronic databases such as PsycINFO , PubMed , Web of Science could be a good starting point. Some of them, like EBSCOhost , ScienceDirect , SciELO , and ProQuest , provide full-text information, while others provide the users mostly with the abstracts of the material. Besides scientific literature, literature reviews often include the so called gray literature . This refers to the material that is either unpublished or published in non-commercial form (e.g., theses, dissertations, government reports, fact sheets, pre-prints of articles). Excluding it completely from a literature review is inappropriate because the search should be always as complete as possible in order to reduce the risk of publication bias. However, when reviewing the material on for example Google Scholar , Science.gov , Social Science Research Network , or PsycEXTRA it should be kept in mind that such search engines also display the material without peer-review and have therefore less credibility regarding the information they are disclosing.
When performing literature reviews, the use of appropriately selected terminology is essential, since it allows the researchers much clearer communication. In psychology, without some commonly agreed lists of terms, we would all get lost in the variety of concepts and vocabularies that could be applied. A typical recommendation for where to look for such index terms would be ‘ Thesaurus of Psychological Index Terms (2007) ’, which includes nearly 9,000 most commonly cross-referenced terms in psychology. In addition, electronic databases mentioned before sometimes prompt the use of the so-called Boolean operators , simple words such as AND, OR, NOT, or AND NOT. These are used for combining and/or excluding specific terms in your search and sometimes allow to obtain more focused and productive results in the search. Other tools to make search strategy more comprehensive and focused are also truncations – a tool for searching terminologies that have same initial roots (e.g., anxiety and anxious) and wildcards for words with spelling deviations (e.g., man and men). It is worth noting that the databases slightly differ in how they label the index terms and utilize specific search tools in their systems.
Among authors, there is not much coherence about different types of literature reviews but in general, most recognize at least two: traditional and systematic. The main difference between them is situated in the process of collecting and selecting data and the material for the review. Systematic literature review, as the name implies, is the more structured of the two and is thought to be more credible. On the other hand, traditional is thought to heavily depend on the researcher’s decisions regarding the data selection and, consequently, evaluation and results. Systematic protocol of the systematic literature review can be therefore understood as an optional solution for controlling the incomplete and possibly biased reports of traditional reviews.
THE SYSTEMATIC LITERATURE REVIEW
The systematic literature review is a method/process/protocol in which a body of literature is aggregated, reviewed and assessed while utilizing pre-specified and standardized techniques. In other words, to reduce bias, the rationale, the hypothesis, and the methods of data collection are prepared before the review and are used as a guide for performing the process. Just like it is for the traditional literature reviews, the goal is to identify, critically appraise, and summarize the existing evidence concerning a clearly defined problem.
Systematic literature reviews allow us to examine conflicting and/or coincident findings, as well as to identify themes that require further investigation. Furthermore, they include the possibility of evaluating consistency and generalization of the evidence regarding specific scientific questions and are, therefore, also of great practical value within the psychological field. The method is particularly useful to integrate the information of a group of studies investigating the same phenomena and it typically focuses on a very specific empirical question, such as ‘Does the Rational Emotive Therapy intervention benefit the well-being of the patients diagnosed with depression?’.
Systematic literature reviews include all (or most) of the following characteristics:
- Objectives clearly defined a priori;
- Explicit pre-defined criteria for inclusion/exclusion of the literature;
- Predetermined search strategy in the collection of the information and systematic following of the process;
- Predefined characteristic criteria applied to all the sources utilized and clearly presented in the review;
- Systematic evaluation of the quality of the studies included in the review;
- Identification of the excluded sources of literature and justification for excluding them;
- Analysis/synthesis of the information (i.e., comparison of the results, qualitative synthesis of the results, meta-analysis);
- References to the incoherences and the errors found in the selected material.
The process of performing a systematic literature review consists of several stages and can be reported in a form of an original research article with the same name (i.e., systematic literature review):
1: Start by clearly defining the objective of the review or form a structured research question.
Place in the research article: Title, Abstract, Introduction.
Example of the objective: The objective of this literature revision is to systematically review and analyse the current research on the effects of music on the anxiety levels of children in hospital settings.
Example of a structured research question: What are the most important factors associated with the development of PTSD in soldiers?
Tip: In the title, identify that the report is a systematic literature review.
2: Clearly specify the methodology of the review and define eligibility criteria (i.e., study selection criteria that the published material must meet in order to be included or excluded from the study). The search should be extensive.
Place in the research article: Methods.
Examples of inclusion criteria: Publication was an academic and peer-reviewed study. Publication was a study that examined the effects of regular physical exercise intervention on depression and included a control group.
Examples of exclusion criteria: Publication was involving male adults. Studies that also examined non-physical activities as interventions. Studies that were only published in a language other than English.
Tips: The eligibility criteria sometimes fit to be presented in tables.
3: Retrieve eligible literature and thoroughly report your search strategy throughout the process. (Ideally, the selection process is performed by at least two independent investigators.)
Example: The EBSCOhost and PsychInfo electronic databases from 2010 to 2017 were searched. These were chosen because of the psychological focus that encompasses psychosocial effects of emotional abuse in childhood. Search terms were ‘emotional abuse’, ‘childhood’, ‘psychosocial effects’, and ‘psychosocial consequences’. The EBSCOhost produced 200 results from the search criteria, while PsychInfo produced 467, for a total of 667 articles. […] Articles were rejected if it was determined from the title and the abstract that the study failed to meet the inclusion criteria. Any ambiguities regarding the application of the selection criteria were resolved through discussions between all the researchers involved.
Tip: Sometimes it is nice to represent the selection process in a graphical representation; in the form of a decision tree or a flow diagram (check PRISMA ).
4: Assess the methodological quality of the selected literature whenever possible and exclude the articles with low methodological quality. Keep in mind that the quality of the systematic review depends on the validity and the quality of the studies included in the review.
Examples of the instruments available for evaluating the quality of the studies: PEDro, Jadad scale, the lists of Delphi, OTseeker, Maastricht criteria.
Tip: Present the excluded articles as a part of the selection process mentioned in step 3.
5: Proceed with the so-called characterization of the studies. Decide which data to look for in all the selected studies and present it in a summarized way. If the information is missing in some specific paper, always register it in your reports. (Ideally, the characterization of the studies is performed by at least two independent investigators.)
Place in the research article: Results.
Examples of the information that should and/or could be collected for characterization of the literature: authors, year, sample size, study design, aims and objectives, findings/results, limitations.
Tip: Sometimes results can be presented nicely in a form of a table depicting the main characteristics.
6: Write a synthesis of the results – integrate the results of different studies and interpret them in a narrative form.
Place in the research article: Interpretation, Conclusions.
Patterns discovered as results should be summarized in a qualitative, narrative form. Modulate one (or more) general arguments for organizing the review. Some trick to help you do this is to choose two or three main information sources (e.g., articles, books, other literature reviews) to explain the results of other studies through a similar way of organization. Connect the information reported by different sources and do not just summarize the results. Find patterns in the results of different studies, identify them, address the theoretical and/or methodological conflicts and try to interpret them. Summarize the principal conclusions and evaluate the current state on the subject by pointing out possible further directions.
CONCLUSIONS
The results emerging from the data that were included in such retrospective studies can lead to a certain level of credibility regarding their conclusions. Actually, systematic literature reviews are thought to be one of our best methods to summarize and synthesize evidence about some specific research question and are often used as the main ‘practice making guidelines’ in many health care disciplines. Therefore, it is no wonder why systematic reviews are gaining popularity among researchers and why journals are moving in this direction as well. This also shows in the development of more and more specific guidelines and checklists for writing systematic literature reviews (see for example PRISMA or Cochrane Handbook for Systematic Reviews of Interventions ). To find examples of systematic literature review articles you can check Cochrane Database of Systematic Reviews , BioMed Central’s Systematic Reviews Journal , and PROSPERO . If you are aware of the concept of ‘registered reports’, it is worth mentioning that submitting with PROSPERO provides you with the option of publishing the latter as well. I suggest that you go through the list of useful resources provided below and hopefully, you can get enough information about anything related that remained unanswered. Now, I encourage you to try to be a little more to be systematic whenever researching some topic, to try to write a systematic literature review yourself and to maybe even consider submitting it to JEPS .
USEFUL RESOURCES
- Cochrane Database of Systematic Reviews : http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/
EBSCOhost : https://search.ebscohost.com/
Google Scholar : https://scholar.google.com/
PRISMA : http://www.prisma-statement.org/
PROSPERO : https://www.crd.york.ac.uk/prospero/
ProQuest : http://www.proquest.com/
PsycEXTRA : http://www.apa.org/pubs/databases/psycextra/index.aspx :
PsycINFO : http://www.apa.org/pubs/databases/psycinfo/index.aspx
PubMed : https://www.ncbi.nlm.nih.gov/pubmed/
SciELO : http://www.scielo.org/php/index.php?lang=en
Science.gov : https://www.science.gov/
ScienceDirect : http://www.sciencedirect.com/
Scorpus : http://www.scopus.com/freelookup/form/author.uri
Social Science Research Network : https://www.ssrn.com/en/
Systematic Reviews Journal (BIOMED) : https://systematicreviewsjournal.biomedcentral.com/
Web of Science : https://webofknowledge.com/
Other sources
- Sampaio, R. F., & Mancini, M. C. (2007). Systematic review studies: A guide for a careful synthesis of scientific evidence. Brasilian Journal of Physical Therapy, 11 (1), 77-82. doi:http://dx.doi.org/10.1590/S1413-35552
- Tuleya, L. G. (2007). Thesaurus of psychological index terms . Washington, DC: American Psychological Association.
Eva Štrukelj
Eva Štrukelj is currently studying Clinical and Health Psychology at the University of Algarve in Portugal. Her main areas of interest are social psychology and health psychology. Regarding research, she is particularly curious about stigma and with it related topics.
Share this:
Related posts:.
- How to write a good literature review article?
- How to search for literature?
- Editorial perspective on scientific writing: An interview with Dr. Renata Franc
- Peer Review Process – Tips for Early Career Scientists
IMAGES
VIDEO
COMMENTS
The PRISMA statement is essential reading before starting a systematic literature review. Editors increasingly expect authors of systematic reviews to use
Systematic Review Protocol & Support Template ... Already had training from library on literature searching and RefWorks and also did literature review for
Kinship care for the safety, permanency, and well-being of children removed from the home for maltreatment. Campbell Systematic Reviews 20xx:x. DOI: 10.4073/csr
times in the USA (Kuhfeld 2009), which included a literature review. Researchers in. Kentucky, USA have conducted other non-systematic literature reviews
Key messages. Templates for reporting major incident medical management exist in different formats, but none have been tested for feasibility. A template for
When writing up your systematic review keep in mind the specific guidelines for structuring your review. Systematic review standards are elements that
Systematic literature reviews concentrate most of the writing in describing the methods used to collect, analyse and present the literature, so
Writing based on four scientific sections: introduction, methods, results, and discussion, mostly with a conclusion. Performing a characteristic
Funding for this systematic review was provided by 3ie's donors, ... example from the forensic economic literature is Olken's study, (2007), where he.
A literature review (a) defines a specific issue, concept, theory, phenomena; (b) compiles published literature on a topic; (c) summarises