appraisal theory case study

• Open access
• Published: 14 January 2019

Applying quantitative appraisal analysis to the study of institutional discourse: the case of EU migration documents

• Mira Tupala   ORCID: orcid.org/0000-0001-9513-7023 1  

Functional Linguistics volume  6 , Article number:  2 ( 2019 ) Cite this article

7278 Accesses

4 Citations

Metrics details

This article addresses the methodological issues related to applying the appraisal framework quantitatively with a corpus of institutional texts. The aim is to explore how to best annotate evaluative expressions in official institutional documents which are usually considered as factual and formal rather than attitudinal and affectual. Using a corpus of migration documents of the European Union as a point of reference, I outline a way of analysis in which each case of appraisal is systematically accounted for and annotated accurately with regard to the socio-cultural context of the EU, serving to deepen and widen the scope of the appraisal analysis. Ultimately, the study of evaluative patterns used in institutional discourse can reveal the ways in which evaluation is integrated into the structures of formal institutional language and how evaluation towards different phenomena (e.g. migration) is constructed.

Introduction

This article outlines the ways in which the appraisal framework can be applied quantitatively with a corpus of institutional texts. Several authors have approached the subject of conducting quantitative appraisal analysis (e.g. Aloy and Taboada 2017 , Fuoli 2015 , Macken-Horarik and Isaac 2014 , Carretero and Taboada 2014 , Kaltenbacher 2006 ), and this paper condenses those ideas that are the most suitable for analysing institutional discourse and connects them to my own insights in order to model an operational hands-on analysis based on the scrutiny of a corpus of migration documents of the European Union (EU). With the help of the appraisal framework, it is possible to unearth the attitudes integrated into the EU’s migration texts that demonstrate both continuity and change in the ways the EU views migration. As a conclusion, I will present a checklist of the main factors that are needed for a systematic analysis of evaluation that takes into account the specific socio-cultural context of EU migration documents. These documents involve a mixed set of, sometimes conflicting, ideologies, which complicates the appraisal coding process as deciphering all the nuances in the evaluations requires customised annotation practices.

The purpose of this research venture is to explore and encourage the usage of the appraisal framework in quantitative, corpus-based studies of institutional discourse which is an area where, despite its increasing popularity, it has not yet gained an established ground. Some corpus-based quantitative appraisal studies have been conducted; however, they have either used quite a small amount of data (e.g. corporate social reports (Fuoli 2012 )), focused on certain keywords/expressions (e.g. concordance lines (Miller 2006 )), have not systematically taken into account each instance of appraisal (e.g. quotations and reported voice (Jullian 2011 )), or used a limited range of appraisal types/sub-systems (e.g. focus on appreciation (Kaltenbacher 2006 )). Thus, more work is needed, and this article offers my contribution to bridging the gap between systemic functional linguistics and corpus linguistics (for previous research, see e.g. Matthiessen 2006 ); I focus on one specific area, namely quantifying evaluative expressions in institutional discourse. This study endeavours to deepen and widen the appraisal analysis. To increase depth, I use a wide range of attitude sub-systems and include both graduation and engagement into the analysis (see next section for a definition of appraisal concepts). Width, on the other hand, is increased by dealing with longer units of appraisal instead of individual words and by connecting the texts to their institutional contexts.

The data referred to in this article consists of official migration documents of the European Union and was collected for the purposes of my doctoral dissertation project that studies the attitudes and ideologies of EU institutions towards migration (Tupala in preparation n.d. ). The corpus includes 56 EU documents with a total word count of about 209,000 words, and it encompasses statements, communications, opinions, directives and reports issued by the European Parliament, European Commission, European Council, Council of the European Union and European Economic and Social Committee between 1999 and 2015. The EU has been actively developing its common and comprehensive migration policy since 1999 in the spirit of freedom, security and justice (European Parliament 1999 ), and these principles still prevail. However, the refugee crisis of 2014/2015 acts as a watershed that led the EU to swiftly refocus its migration agenda, this time with a more robust approach on border management and stopping irregular migration (European Commission 2015 ). The 17-year time-span enables the study of how and if the EU’s attitudes towards migration and migrants have changed over the years. Although 209,000 words may seem a relatively modest number, it is the level of detail with which the data has been annotated that gives it its value and strength. Size and scope of the corpus are not enough instead what is required is the richness of the annotations and a program that is up to the task (Wu 2009 : 132). The EU corpus was manually annotated with the help of the UAM CorpusTool which has been designed for the linguistic annotation of texts. Footnote 1

Migration can essentially be defined as movement of people (e.g. economic migrants or refugees) which varies in length, composition and cause (European Commission 2018 ). Through its migration policy, the EU aspires to control this movement and its effects on EU member states, and the documents the EU institutions issue to discuss the changing situations contain not only practical but also attitudinal responses towards migration. The EU’s migration policy focuses on enhancing economic development and solidarity with a strong commitment to fundamental rights; migration documents are drafted with these goals in mind, encouraging openness and mobility. However, nowadays they are also increasingly affected by security concerns and challenges of irregular migration, marking a change towards “managing” migration to suit the EU’s needs. Thus, as the increased population movement has switched the EU’s political priorities, it is even more important to study the EU’s attitudes towards migrants who are subjected to its policies by investigating the ways in which evaluation unfolds across texts as the EU juggles its diverse goals. I have identified three main issues in the appraisal framework that need to be considered in order to accurately annotate each instance of appraisal. These relate to meaningful units, contextual knowledge and layers in the annotations (see section Issues and Proposed Solutions).

Studying appraisal in the EU’s migration documents gives insight into the extent to which such texts carry evaluative and attitudinal meanings and what these reveal about the EU’s ideological base. Institutions use discourses to legitimise their existence and interests in order to cement certain social practices (Mayr 2008 : 2). Therefore, institutional discourses do not just relay information but also promote agendas favourable to the institution’s cause which hints at the existence of attitudinal and evaluative positions; however, the language used in official documents is generally perceived as very rigid and factual. Any given text type has its own set of systemic probabilities, that is, the kinds of linguistic functions that are most likely to occur in certain types of texts (Halliday and Matthiessen 2014 : 29), for instance, institutional texts are characterised by formulaic expressions and formal language structures. Evaluative and emotive lexis, on the other hand, is not (usually) associated with official institutional discourse. However, for this very reason, it is fruitful to study different types of evaluative expressions and their frequencies in the institutional documents of the EU as it can be assumed that the values and attitudes expressed in them have to be deep-rooted in order to penetrate the seemingly neutral legalese of the texts.

• Appraisal framework

The appraisal framework is situated within systemic functional linguistics and relates to the interpersonal in language. It studies attitudinal and evaluative language, that is, how authors/speakers express their emotions or how they approve of or criticise the phenomena and people around them, and how they negotiate these positions with conversational partners (Martin and White 2005 : 1). Appraisal can be divided into three types that have sub-systems (see Fig.  1 ). It is worth noting that there are no linguistic structures that have a purely evaluative purpose, instead evaluation functions at the level of wording and is parasitic of other language functions or linguistic structures, such as modality, polarity and the choice of pronouns.

Produced with the UAM CorpusTool; examples from the EU corpus. For details see O’Donnell 2007 ; Martin and White 2005 : 137; White 2015 : 4.

Explicit and implicit appraisal

Within attitude, we can make a general distinction between explicit and implicit appraisal, and studying how they work together is an important part of decoding attitudes in official texts. Explicit appraisal can be roughly categorised as the “easy” case as it is realised by lexical items or wordings that directly indicate the positive or negative attitudes of the author (Hyatt 2007 , 2009: 130) as illustrated in examples 1 and 2.

Smugglers and human traffickers exploit irregular migration. Footnote 3

The positive effects of migration for host societies in Europe. Footnote 4

In examples 1 and 2, exploit is explicitly negative and positive effects is explicitly positive describing the negative side effects of irregular migration and the benefits of migration for European societies respectively.

It is the implicit cases of appraisal that are the “problem” as they are not confined to individual words (Kaltenbacher 2006 : 274). Implicit appraisal is conveyed by factual, superficially neutral expressions that can be interpreted as attitudinal in specific contexts by audiences that recognise the cultural or social reference (Hyatt 2007 , 2009: 130) as demonstrated in examples 3 and 4.

This contributes to making immigrants more likely to take undeclared work. Footnote 5

Europe’s population is ageing. Footnote 6

In examples 3 and 4, evaluation is not expressed directly but in a way that prompts the reader to interpret the message negatively. Example 3 refers to how immigrants often encounter difficulties in finding jobs which results in them having to resort to undeclared work. This has negative consequences for both immigrants and the host society as it feeds black market economy. Example 4 is even more subtle and could be seen as a mere statement of fact but connected to discussions about labour shortages, it can be interpreted as a negative evaluation of the situation that needs to be addressed and to which migration is sometimes offered as a solution.

Kaltenbacher ( 2006 : 274–275) contemplates ways of identifying implicit appraisal in corpora including keyword searches (evaluative items or targets of evaluation) and using explicit appraisal as a starting point as implicit cases often occur in the vicinity of explicit ones. Albeit both of these offer a viable place to start, neither is enough to enable finding all cases of implicit evaluation. With keyword searches, it is possible to overlook some (less common) items that might be important for the analysis. Relying on explicit appraisal does not take the analysis very far either as, with institutional texts, it is often the case that a large number of the evaluations are carried out implicitly due to the formal nature of the texts that does not encourage strong attitudinal viewpoints. These cases would be left outside the scope of the analysis if one only considered the immediate co-text of explicit appraisal. Nevertheless, explicit cases in earlier parts of the text guide the reader’s interpretation of the following implicit cases by creating a setting through which readers are conditioned to accept certain meanings or value positions (see e.g. ‘dynamic reader positioning’ in Coffin and O’Halloran 2005 , Coffin and O’Halloran 2006 ; White 2008 ). Once a meaning has been established, for instance labour migrants having a positive impact on the EU’s economy, it can be backed up later on in the text by deploying implicit means, such as statements of fact about the decrease in the domestic working-age population.

Due to the diverse nature of the evaluative items and targets of evaluation in the data, I opted for a completely manual search. However, although keyword searches are not efficient enough for finding all instances of appraisal, they are useful when analysing evaluative patterns. I conducted keyword searches after the first round of manual annotations in order to trace how certain targets of evaluation, such as (im)migrants , (im)migration , Europe and the EU , are appraised in the data in terms of polarity and explicitness. It is important to tie the evaluative item to its target, and Mary Macken-Horarik and Anne Isaac ( 2014 : 86) highlight the significance of co-text when coding appraisal as the textual environment shapes the readers’ perceptions. Studying the co-occurrence and alternation of explicit and implicit appraisal in texts can reveal how evaluative meanings are constructed.

Evaluative patterns

The detailed mapping of all cases of appraisal is significant when studying evaluative patterns in institutional texts. Once each instance of evaluation has been systematically charted and analysed, repeating patterns start to emerge making it possible to see how attitudinal stances are built throughout a text(s) and connect them to prevailing institutional value systems and ideologies. Evaluation is founded on earlier evaluations (see Coffin and O’Halloran 2005 ), and I argue that frequency strengthens the force of evaluation making it more solid. It is worth noting that isolated linguistic features or words alone do not reconstruct attitudes and ideologies, instead they come into being through the interplay between and the layeredness of linguistic resources that engender a higher meaning (Caffarel and Rechniewski 2009 : 33). This meaning is moulded by what is valued or not valued in the surrounding society and is bigger than the sum of individual words. In the case of the EU, the attitudes and ideological networks it applies to migration need to be considered with respect to the EU’s fundamental values while bearing in mind the prevalent western economy-oriented way of thinking that quarrels with the notion of solidarity both within the EU and in its relations with third countries.

The EU’s migration documents often function as responses to previous ones showing how the discussion around migration has accumulated over the years in response to changing situations and how the EU uses discoursal means to build its stance towards migration. EU institutions repeatedly use certain explicit evaluative patterns, such as references to “enhancing prosperity” and “Europe that protects,” which are meant to demonstrate the economic benefits of migration and the EU’s commitment to human rights and solidarity. Coffin and O’Halloran ( 2005 : 144) identify an ‘evaluative groove’ to study the ways in which meanings are piled up within a single text or through a string of subsequent texts in order to invite readers to interpret assumingly neutral expressions positively or negatively. Referring back to example 4, we can see how just mentioning the ageing population is enough to stir up an idea that this will cause problems in the long run, therefore justifying the need to attract workers from outside the EU.

Corpus tool

The EU corpus was annotated with the UAM CorpusTool. However, as it cannot code cases of appraisal automatically, its role is to assist the manual annotation process. This coincides with Geoffrey Leech’s ( 1991 : 20) Data Retrieval Model in which the main function of a corpus program is to search, retrieve, store and count data while human intervention is needed to conduct the annotations. The corpus tool thus has a passive role, and tracing instances of appraisal manually emphasises the role of the analyst in ensuring that each annotational choice is justified and allows making connections with the specific context.

Auto-coding appraisal is generally not possible as it is not tied to any specific grammatical structures or words. The only exception to this in the UAM CorpusTool are those segments/words that have already been assigned a certain appraisal category as the program can be ordered to repeat it in similar cases throughout the data set. However, this only works with those cases of (explicit) appraisal that are carried by individual words as implicit cases of appraisal usually consist of longer units that most likely occur only once in the exact same form. In addition, since the same words may convey different meanings in different contexts, the auto-code function must be used with caution and the results checked manually. For instance, intolerance and racism are explicitly negative concepts and can be annotated as such. However, these words can also appear in contexts that condemn intolerance and racism and on such occasions, the overall effect is positive rather than negative. Nevertheless, searching for certain predictable items of interest in this way makes conducting the analysis easier since, instead of having to find all these instances manually, the researcher can check the auto-code results and refine them if needed. This step is necessary as the program is not able to decode the co-text, the understanding of which is essential in order to categorise the cases of appraisal accurately.

Let us now turn to the annotation scheme used for coding appraisal in the EU corpus. The UAM CorpusTool has a pre-made appraisal annotation scheme which allows editing, making it possible to increase or decrease the level of detail. I created an edited annotation scheme to suit the purposes of the present study (see Fig.  2 ). The edited scheme is deliberately elaborate when it comes to attitude enabling the charting of all the nuances in the evaluation of people/institutions and phenomena in the institutional context of the EU. I added the sub-categories of affect as the original scheme only makes the distinction between authorial and non-authorial affect (i.e. whose feelings are discussed) but does not include un/happiness, dis/satisfaction, in/security and dis/inclination (i.e. the distinction between different types of feelings). I also included the distinction between positive and negative cases within each sub-system of attitude instead of only dealing with polarity on a more general level. This enables a more detailed study of how and if polarity is used differently within different sub-systems. Conversely, I left out the sub-systems of engagement and graduation is only divided into force and focus as the sub-systems missing from the scheme are only dealt with qualitatively since their function is to support the attitude analysis.

Produced with the UAM CorpusTool.

Issues and proposed solutions

In this section, I will discuss the three main issues that are worth considering when conducting quantitative appraisal analysis on a corpus of institutional texts. I will also propose some solutions that have proven to be useful as it was necessary to adapt some aspects of the appraisal framework to make it better suit the characteristics of institutional discourse.

There is a clear evaluative layer in the EU migration documents as their function is to negotiate how to best manage migration and to assert the EU’s power over migrants by laying down their rights and obligations. However, given the official nature of these texts, rather than being conveyed by transparently attitudinal words, the evaluative expressions used by the EU institutions are often interwoven within the structures of formal institutional discourse and require contextual knowledge to be interpreted correctly. Furthermore, it is common for the EU migration documents to impersonalise expressions of attitude as they entail layered evaluations where explicit appreciation of phenomena is used to veil implicit judgement towards people or their actions. Based on these characteristics, the discussion here revolves around identifying units of meaning, avoiding subjective interpretation and allowing layered codings.

One word vs. a phrase/sentence

The first issue is to determine whether to operate with individual words or with longer units (phrases/sentences) when annotating cases of appraisal. Appraisal analysis has to be connected to particular instances of language that are countable, otherwise there is a risk that the analysis relies on the general “feel” of the text instead of being grounded on concrete findings. This is also what Geoff Thompson ( 2014 ) is wary of and thus he puts much emphasis on the actual wording when categorising cases of appraisal. I agree with Thompson about tying the analysis to the level of wording, which does not mean that the annotations should be restricted to individual words or noun phrases. Prototypical examples of appraisal usually focus on depicting these isolated instances; however, more often than not an evaluation is carried by a longer unit within which it is not possible to identify only one evaluative lexical item but in which the whole sentence, for example, conveys an evaluative stance. My analysis thus deals with coherent units of meaning that connect cases of appraisal to their co-texts.

Example 5 demonstrates how the use of longer units in the annotations helps to capture the ideological context of evaluation.

Intolerance, racism and xenophobia are on the rise in Europe. Footnote 8

In example 5, it would be possible to annotate the words intolerance , racism and xenophobia as separate instances but I argue for treating the whole sentence as one appraisal unit that expresses negative appreciation towards the phenomenon. The focus on individual words would make the analysis too scattered especially with a large data set since it is the units of meaning that matter when tracking down evaluative patterns in texts. As the interest here lies not only in what kinds of evaluations there are but also in studying what/who is being evaluated and why, it is essential to consider the immediate co-text around the evaluative item as it reveals the target of the evaluation and ties the evaluation to context. For instance, in the case of EU documents, the focus is on how many cases of appraisal and of what type are related to the EU, EU policies, migration and migrants, and whether they are positive or negative, explicit or implicit. This can be used as basis for determining the EU’s value positions towards migration, and the emerging patterns can shed light to the ways in which the institutional attitudes bleed into their discourses.

Relevance of contextual knowledge

The second issue that needs to be addressed is how to determine what can be considered attitudinal or evaluative in a given context. The annotations are often based on manual work conducted by one person, which allows subjective biases and preconceptions to penetrate the analytical process to some degree. For instance, if researchers start from the presumption that a certain institution engages in discrimination, there is a risk that they will evaluate the texts more negatively than if they started from an objective standpoint. Martin and White ( 2005 : 162–163, 207) argue that subjective interpretation cannot be totally avoided as meanings are negotiated between the (authors of the) texts and the readers whose cultural backgrounds and personal experiences affect their reading position. The subjective bias can be mitigated by ensuring that the annotations are backed by valid and relevant background information on what is considered as positive/acceptable or negative/unacceptable in the society or institution under inspection.

As stated earlier, implicit evaluation might become clear only to those audiences that share the author’s understanding of a given society or culture. The evaluative expressions that utilise prior knowledge can be used to either persuade readers to accept the text’s position or alienate certain groups that do not belong into the target readership (see e.g. ‘ideal subject’ in Fairclough 1989 : 49, 54; ‘in-group texts’ in Don 2016 ). Hence, in addition to subjective over-interpretation, there is the possibility that researchers might overlook the issues of which they do not have all the necessary information. In the case of official EU documents, it should also be borne in mind that although they are freely available to the public, they form a chain of communication between EU institutions and as such are written from one bureaucratic institution to another. Thus, these texts expect certain knowledge of the previous texts, and the evaluations they contain are affected by the previous ones. For this reason, the EU corpus includes text-chains that consist of documents from different EU institutions focusing on the same issue.

When studying EU migration discourse, what need to be taken into account are the institutional confinements of the EU that are reflected against its core values and agendas. The EU frames all its policies around the principles of prosperity, solidarity and security, Footnote 9 and also its attitudinal responses towards migration are grounded on these three dimensions. The common denominator in the EU corpus is the legality of migration; legal migration is described in positive terms as it enhances economic development, partnership and security while irregular migration is viewed as having a negative effect on all three.

Ambiguity and overlap

The third issue is that the annotation of appraisal types, and especially the sub-systems of attitude, always involves certain ambiguity since wordings can be categorised into more than one system depending on the interpretation (Martin and White 2005 : 60–61; Thompson 2004 : 77). For instance, a sentence might be superficially interpreted as appreciation but the underlying evaluation entails judgement such as in examples 6 and 7.

The Council emphasises that the respect for the Charter of Fundamental rights and human rights lie at the basis of the EU migration policy and its relations with third countries. Footnote 10

Example 6 includes an explicit positive appreciation towards the EU migration policy for its respect for fundamental and human rights and an implicit positive judgement towards the EU itself for enforcing a policy that embraces human rights. Furthermore, the example entails an engagement the Council emphasises that which conveys the Council’s recognition of the EU’s commitment to enhancing human rights. In addition, emphasises can be labelled as graduation as it is a stronger expression of engagement than, for instance, the verb ‘state’. It is common for explicit and implicit cases of appraisal to be intertwined and carry double meanings as often, especially in the case of judgement, the underlying target of evaluation is left opaque and expressed through appreciation of the situation rather than a direct judgement of (human) behaviour.

The Committee […] is alarmed because in some Member States the protection of people’s fundamental rights is being eroded. Footnote 11

Example 7 contains an explicit negative appreciation towards the described phenomenon, that is, the erosion of fundamental rights in some EU member states. However, when studied in context, there is also a deeper meaning in the form of an implicit negative judgement towards those member states’ governments and public officials which are letting it happen. Moreover, affect often coincides with engagement as the Committee is alarmed is both an emotional response and a way of taking a stand. As examples 6 and 7 demonstrate, layered codings are necessary to capture all relevant meanings. Monika Bednarek ( 2007 ) indicates that lexical items often carry multiple voices (polyphony) and this also applies to systems of appraisal which have a tendency to co-occur within the same wording. Appraisive language is not one-dimensional: there is often more than one evaluative layer at play and the underlying meanings might get lost if we only focus on the superficial layer. Therefore, I assert that layered codings enrich the analysis and help in decoding the overall meaning of the message.

The categorisation of underlying targets of evaluation, however, is not that straightforward. Thompson ( 2014 : 57) contemplates underlying targets and the semantically-based view when discussing whether to label a case of appraisal as judgement or as appreciation. According to the semantically-based view, if the underlying target is human (or an institution whose actions are comparable to those of humans), the evaluation can be categorised as judgement. To clarify, I refer back to examples 6 and 7 which at the superficial level are appreciations towards the EU migration policy and (some) member states respectively. Nevertheless, they also contain implicit judgement which is targeted at the EU and the member states’ governments, hence revealing reverence towards the EU and condemnation towards the ill-behaved governments. However, Thompson ( 2014 : 58–59) rejects the semantically-based view as he indicates that the actual (literal) wording should be used as a starting point for analysis instead of attempting to decipher the underlying meaning. Here, I differ from Thompson as I use both judgement and appreciation in cases like these for the sake of increased depth of the analysis. This helps in uncovering how the EU builds its attitudinal stance towards migration and different groups of migrants since preferring the use of impersonalised appreciation instead of judgement is a value choice in itself.

In order to increase the transparency of the annotations, I nonetheless share Thompson’s ( 2014 : 64) notion of connecting the analysis to particular (strings of) words. Earlier, I defined a case of appraisal as a coherent unit of meaning that may consist of a phrase or a sentence; however, for the sake of clarity, it should not extend to multiple sentences or whole paragraphs. With too long units, there is an increased risk of ambiguity and over-interpretation; moreover some significant details might get ignored if the analysis only focused on the big picture. The overall tone of the text can be identified as generally positive or negative but determining how evaluation actually builds up within a text requires a much more focused approach. It is the level of wording where the evaluation is grounded, and the alternation and the dynamic between different types of appraisal is revealed by the systematic quantitative study of each instance.

Mixed methods

Even though the gist of this article is about operationalising quantitative appraisal analysis, it is beneficial to combine quantitative and qualitative methods. To gain a profound understanding of the ways in which the EU evaluates migration, the quantitative study of frequencies of appraisal types needs to be complemented with the qualitative study of the co−/contextual factors that tie the evaluative items to their targets and to the wider framework of the EU’s goals and value systems concerning the movement of people. In this section, I will discuss two approaches, one from each side, that can be used as reference points when modelling a way to conduct quantitative appraisal analysis on institutional discourse. The first one is Matteo Fuoli’s ( 2015 ) seven step method on systematic quantitative appraisal analysis that is intended to decrease subjectivity, and the second one is Macken-Horarik and Isaac’s ( 2014 ) cline of implicitness that focuses on detailed qualitative analysis.

Fuoli ( 2015 ) has proposed a step-by-step method for manual appraisal annotation to enhance replicability , reliability and transparency . The approach is based on finding the suitable annotation criteria for a specific data set and on the constant refining of the annotations to maximise accuracy. There are two main areas in Fuoli’s method that I make use of when analysing a specialised corpus of institutional texts: these are related to data processing and content . Firstly, when it comes to data processing, Fuoli ( 2015 : 17–18) emphasises context-specificity: analysts should draft an annotation manual that specifies which appraisal categories they are going to use and how they are defined for the purposes of a particular data set so that the analysis can be repeated. This task can be eased by the use of a corpus tool where the annotation scheme can be edited to suit the researcher’s purposes so that it is custom-made for the project in question which gives the researcher control over how elaborate the analysis is. The UAM CorpusTool is suitable for this purpose, and the edited annotation scheme was presented in the third section. Fuoli ( 2015 : 17) refers to the UAM’s auto-code function as potentially useful but, as I already pointed out, it does not work without strong analyst intervention. The EU corpus was subjected to three rounds of annotations: twice for appraisal and once for targets of evaluation. Secondly, data processing is connected to the level of detail, and the main factors I identify here are the depth (the number of sub-systems/categories accounted for in the analysis) and width (individual words vs. longer units) of the annotations.

The second area, content, relates to the specific features of the discourse type. The generic conventions of institutional discourse determine the means by which evaluation is conveyed (e.g. collective expressions of attitude Footnote 12 and impersonal attitude constructions Footnote 13 ) and this also affects the annotations. Therefore, after the annotation manual has been created, category definitions should be tested against the data and possibly further adjusted according to its needs (Fuoli 2015 : 17–18). My category definitions are mainly based on those of Martin and White ( 2005 ) as they offer clear general descriptions of the appraisal types and sub-systems. Nevertheless, they also leave room for manoeuvre, and I have expanded the definition of judgement as, in its strictest sense, it ought to be used when evaluating human behaviour. However, I use it to refer to the EU/EU institutions and their actions as well for it is ultimately people who are responsible for the EU’s decision-making and holding to its commitments in the name of the EU’s shared value-base. Therefore, I deem that the actions of the institutions can be judged along the same norms as those of individual people. Furthermore, an underlying actor can often be identified even if the message is relayed impersonally: for instance, attributing the respect for human rights to EU policies when it is actually the EU that commits the act of respecting (see example 6). The UAM CorpusTool is useful in managing the layered codings and providing the frequency statistics for the overlaps so that they can be examined. In addition to coding different types of appraisal, there is a need to consider the text surrounding them due to its significance for understanding how evaluation operates. Hence, Macken-Horarik and Isaac’s ( 2014 ) approach is helpful as they offer guidance on identifying (implicit) evaluation on different textual levels based rather on cultural understanding than on quantifiable patterns. They also accept the “fuzzyness” of categories, that is, the need for multiple codings to discover all additudinal layers.

Macken-Horarik and Isaac ( 2014 : 89) present a cline of implicitness starting from the close scrutiny of individual words and wordings ranging all the way to the level of culture that requires an understanding of the norms and rules of a society. This approach is suitable for elaborate qualitative analyses, and although it is more oriented towards the analysis of literary texts, it has value in the study of institutional texts as well since much of the EU’s migration discourse relies on the understanding of the cultural background, values and procedures on which the EU is built. Evaluation naturally manifests itself differently in different types of texts; in literary texts, it might be used to guide the reader towards a certain reading position such as feeling sympathy for the protagonist while in institutional texts, evaluation is more about spreading the general attitudes and beliefs of the institution. These are subject to change in response to changes in society; for instance, the security dimension and the urge to curb irregular migration have become even more pronounced in EU documents after the onset of the refugee crisis (European Commission 2015 ). There is also diversity between different types of EU texts, statements and communications being more formal and opinions more informal containing more explicitly evaluative expressions.

If the merits of quantitative analysis lie in revealing evaluative patters in texts that can then be connected to the wider construction of value positions, qualitative approach ties the analysis to the socio-cultural context. The lexical patterns and linguistic structures we use do not rely only on our personal preferences but are affected by our linguistic and cultural backgrounds (Kaltenbacher 2006 : 270). Hence, we can come to the conclusion that the study of lexical/evaluative patterns in corpora discloses things about the society in which the texts have been created (see e.g. Kaltenbacher 2006 ). Here, however, lies the complexity of conducting quantitative corpus-based appraisal analysis as decoding cases of appraisal automatically is practically impossible due to evaluation being influenced by interpersonal factors that are not visible in individual words (Miller 2006 : 249). This underlines the significance of researcher involvement. Evaluation gains meaning from outside the realm of the text which shows that quantitative approach alone cannot grasp all evaluative dimensions if one wants to study the underlying motives of evaluation in institutional discourse.

To deal with the dilemma of how to quantify units of appraisal and to account for extra-linguistic factors, I have.

Conducted the appraisal analysis manually, sentence by sentence, taking into account the co-text of the cases of appraisal and the target and source of the evaluation.

Considered the wider institutional context of the texts and their impact and effectiveness.

In addition to appraisal annotation, point number one includes the annotation of the targets of evaluation: each case of appraisal is coded for whether it refers to the EU institutions, EU as a whole, EU policies, EU member states, non-EU countries, migration or migrants. There is a division between EU and non-EU targets which makes it possible to explore the differences between the ways in which the EU evaluates itself and the ways in which it evaluates migration. As suggested in point number two, this study is also interested in how the EU migration documents are situated in the context of the EU framework. Thompson and Hunston ( 2006 : 4) argue that collecting texts into a corpus has the downside of separating texts from their contexts. However, in order to avoid this disconnection, I have studied the texts’ legal force and their impact on the EU’s decision-making and how the institutions communicate with each other through different types of texts when negotiating the migration policy (Tupala in prep. n.d. ).

Conclusion: Checklist

As a conclusion, I will collect the most important points of applying quantitative appraisal analysis on a corpus of institutional texts into a checklist. Due to the formal nature of the EU’s institutional documents, rather than being overtly evaluative, the attitudinal positions they project are often discreet and require careful monitoring from the analyst to be analysed accurately. Therefore, to conduct the analysis as profoundly as possible, I declared it effective to mix quantitative and qualitative methods to be able to study evaluative patterns throughout the data and to closely inspect particular instances of evaluation in their specific contexts. There are six building blocks on which the appraisal analysis is based:

Manual annotation

Corpus tool assistance

Not limiting the analysis to individual words

Allowing layered codings

Quantifiability

Context-boundness

The first and second points on the list are closely related. The annotation process is based on manual work conducted with the help of the UAM CorpusTool, the main purpose of which is to act as a storing and counting device. It offers an interface for the user to conduct the annotations but is rather a passive participant for it cannot recognise appraisal on its own. Furthermore, a corpus tool helps in staying consistent, enabling constant revision and making changes with a few clicks. Since the corpus consists of institutional texts in which evaluations are usually very subtle and implicit appraisal is favoured, manual search was the most viable option as otherwise it would have been practically impossible to trace all the cases of appraisal. Keyword searches and reliance on explicit appraisal can be of help at the initial stages but they only get the analysis to a certain point due to diversity of evaluative items and targets of evaluation and due to many implicit cases of appraisal appearing “alone,” that is, without explicit instances in the vicinity.

Thirdly, since I have argued for the relevance of including co-text (and context), I did not limit the annotations to individual words but dealt with meaningful units instead. Especially in the case of implicit appraisal, it is not always possible to separate the evaluative item from its immediate co-text as it is entwined into it and as the correct understanding depends on it. The fourth point on the checklist is that I allow layered codings in the analysis in order to catch both the surface level and the underlying evaluations. Evaluation is multidimensional; for example, explicit appreciations towards EU policies or migration often convey implicit judgements towards the EU or migrants respectively. These kinds of veiled judgements tell a story of their own as there is a reason behind evading direct judgement towards the EU’s shortcomings for instance. It is also important to pay attention to those instances where evaluation towards a group of migrants is concealed by seemingly impersonal expressions which nonetheless entail an attitudinal stance towards them.

The fifth point revolves around countability. Every instance of appraisal has to be accounted for, and the corpus tool provides the statistics automatically (i.e. displays the exact numbers for each appraisal type) and enables making comparisons between targets of evaluation. Finally, point six is that all instances of appraisal have to be context-bound so that it is possible to determine who is evaluating what or who and how and eventually why. Even if official texts are often seen as plain and neutral, they are still tightly connected to certain culturally and socially bound ideas shared between the institution and its members (for details see e.g. ‘groups’ and ‘institutions’ in van Dijk 1998 : 141, 186). The EU’s core values and the ideological beliefs it promotes largely determine what kinds of evaluative and attitudinal expressions appear in its discourses. However, it is the evaluations and attitudes that run counter to these that are of special interest, and therefore, their frequency and quality in different EU documents can reveal discrepancies between the EU’s projected values and its (discursive) actions.

Fundamental values and economic growth lie at the centre of the EU migration policy, and in its strategy, the EU envisions an open and secure Europe that recognises both the opportunities and challenges of international mobility (European Commission 2014 ). Increased flows of refugees in recent years combined with the EU member states suffering from the after-effects of the economic crisis have caused the EU to re-evaluate its views on migration, and this is also reflected in the migration documents which put an even stronger emphasis on economic values, security and protection than before. The ultimate goal of the appraisal analysis here is to map out the ways in which the evaluation of migration and migrants mirrors the EU’s ideological networks and the changes therein in order to infer the motives that drive the migration policy.

The checklist was originally created for the purposes of my doctoral dissertation project (Tupala in prep. n.d. ), and the intention in this article was to clarify the annotation process and to propose ways to conduct quantitative appraisal analysis in a manner that connects it to the institutional and ideological context where the EU’s migration texts were produced. As I already indicated in the first section, previous quantitative appraisal studies have mainly focused on specific instances in rather restricted sets of data. My aim, however, is to promote the use of the appraisal framework in larger scale quantitative studies that not only account for each instance of appraisal and make use of a broad range of appraisal types (with their sub-systems/categories) and their layeredness but also ultimately chart the co−/context of evaluative expressions. Following the points on the checklist can hence lead to deeper and more grounded quantitative appraisal analyses. Detailed manual annotation, although very time-consuming with a large set of data, allows the analyst to find the rare, unpredictable and implicit instances of evaluation. The use of a corpus tool enables the coding of the co-text of cases of appraisal which, when combined with the appraisal annotation, binds the evaluative items to their targets of evaluation.

With the systematic account of all the cases of appraisal in a corpus, it is possible to see the ways in which evaluation towards different phenomena is constructed and how it is integrated into the structures of formal institutional discourse. It is common for EU institutions to use expressions of engagement, as meta-text before discussing the issue at hand, to demonstrate their level of commitment or stance towards it (see example 6). The scrutiny of implicit cases in particular can reveal underlying attitudes that have become the norm in institutional texts (see example 4: the EU referring to population ageing in order to rationalise the urgency to increase labour migration). The study of frequencies and tying the findings to the specific institutional context opens up new avenues for applying the appraisal framework. For instance, it allows the investigation of the extent to which a political institution, such as the EU, makes use of evaluative language in its official discourses and how (and if) evaluative choices reflect the ideological base of the institution. Furthermore, when combined with methods extracted, for example, from critical discourse analysis and political sociology, my method offers the possibility of examining the ways in which institutional constraints and social and economic forces shape attitudes in institutional discourses. Footnote 14

Universidad Autónoma de Madrid; Mick O’Donnell 2007 . UAM CorpusTool, Version 2.8.14.

European Parliament 2013 . Integration of Migrants, Its Effects on the Labour Market and the External Dimension of Social Security Coordination.

European Economic and Social Committee 2009 . A Common Immigration Policy for Europe.

European Commission 2008 . A Common Immigration Policy for Europe: Principles, Actions and Tools.

European Economic and Social Committee 2011 . The Role of Legal Immigration in the Context of Demographic Challenges.

European Economic and Social Committee 2014 . An Open and Secure Europe.

In relation to migration, prosperity is mainly focused on the legal movement of people, for instance, putting measures in place to attract highly-qualified and seasonal workers to the EU to reap economic benefits. Solidarity deals with co-operation between the EU member states to share the burden and with partnership with non-EU countries, for example, to reduce push factors and to ensure ethical recruitment. Security is about fighting irregular migration by enhancing border control and by targeting criminal networks that make profit by smuggling and trafficking in human beings (European Commission 2008 ).

Council of the European Union 2012 . Council Conclusions on the Global Approach to Migration and Mobility.

see example 7: “is alarmed” is a collective expression of the Committee’s emotions.

see example 5: the example contains an impersonal attitude construction where an appreciation of the situation is used instead of direct judgement of people.

The results of the appraisal analysis on EU migration documents and the discussion connecting them to chosen methods of critical discourse analysis and political sociology will be presented in my doctoral dissertation (Tupala in prep. n.d. ).

Abbreviations

European Union

Universidad Autónoma de Madrid

Aloy, M., and M. Taboada. 2017. Evaluation in political discourse addressed to women: Appraisal analysis of Cosmopolitan’s online coverage of the 2014 US midterm elections. Discourse, Context & Media 18: 40–48.

Article   Google Scholar  

Bednarek, M. 2007. Polyphony in appraisal: Typological and topological perspectives. Linguistics and Human Sciences 3 (2): 107–136.

Google Scholar  

Caffarel, A., and E. Rechniewski. 2009. A systemic functional approach to analysing and interpreting ideology: An illustration from French editorials. Revista Alicantina de Estudios Ingleses (22): 27–43.

Carretero, M., and M. Taboada. 2014. Graduation within the scope of attitude in English and Spanish consumer reviews of books and movies. In Evaluation in context , ed. Geoff Thompson and Laura Alba-Juez, 221–239. Amsterdam: Benjamins.

Chapter   Google Scholar  

Coffin, C., and K. O’Halloran. 2005. Finding the global groove: Theorising and analysing dynamic reader positioning using appraisal, corpus, and a concordancer. Critical Discourse Studies 2 (2): 143–163.

Coffin, C., and K. O’Halloran. 2006. The role of appraisal and corpora in detecting covert evaluation. Functions of Language 13 (1): 77–110.

Council of the European Union. 2012. Council Conclusions on the Global Approach to Migration and Mobility. General Secretariat of the Council. http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%209417%202012%20INIT . Accessed 9 Sept 2018.

Don, A. 2016. “It is hard to mesh all this”: Invoking attitude, persona and argument organization. Functional. Linguistics 3 (9): 1–26.

European Commission. 2008. A Common Immigration Policy for Europe: Principles, Actions and Tools. EUR-Lex. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52008DC0359 . Accessed 9 Sept 2018.

European Commission. 2014. An Open and Secure Europe: Making It Happen. EUR-Lex. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52014DC0154&from=EN . Accessed 10 September 2018.

European Commission. 2015. An European Agenda on Migration. EUR-Lex. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52015DC0240 . Accessed 10 Sept 2018.

European Commission. 2018. Migration. Migration and Home Affairs: Glossary. https://ec.europa.eu/home-affairs/content/migration_en . Accessed 4 Sept 2018.

European Economic and Social Committee. 2009. A common immigration policy for Europe. Official Journal of the European Union. https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ%3AC%3A2009%3A218%3A0069%3A0077%3AEN%3APDF . Accessed 11 Sept 2018.

European Economic and Social Committee. 2011. The role of legal immigration in the context of demographic challenges. Official Journal of the European Union. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52010AE1172 . Accessed 12 Sept 2018.

European Economic and Social Committee. 2014. An open and secure Europe. Official Journal of the European Union. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.C_.2014.451.01.0096.01.ENG . Accessed 9 Sept 2018.

European Parliament. 1999. Presidency Conclusions: Tampere European Council 15 and 16 October 1999. http://www.europarl.europa.eu/summits/tam_en.htm . Accessed 1 Oct 2018.

European Parliament. 2013. Integration of Migrants, Its Effects on the Labour Market and the External Dimension of Social Security Coordination. http://www.europarl.europa.eu/sides/getDoc.do?pubRef=-%2f%2fEP%2f%2fTEXT%2bTA%2bP7-TA-2013-0092%2b0%2bDOC%2bXML%2bV0%2f%2fEN&language=EN . Accessed 5 Sept 2018.

Fairclough, N. 1989. Language and power . London: Longman.

Fuoli, M. 2012. Assessing social responsibility: A quantitative analysis of appraisal in BP’s and IKEA’s social reports. Discourse & Communication 6 (1): 55–81.

Fuoli, M. 2015. A step-wise method for annotating appraisal. Functions of Language 25 (2): 1–26 Available: http://lup.lub.lu.se/search/ws/files/23007580/Article_3_a_stepwise_model.pdf . Accessed 19 Sept 2017.

Halliday, M.A.K., and Christian M.I.M. Matthiessen. 2014. Halliday’s introduction to functional grammar . 4th ed. London: Routledge.

Book   Google Scholar  

Hyatt, DF (2007) 2009. Applying a critical systemic-functional literacy frame in a UK secondary education context. In Advances in language and education, ed. Anne McCabe, Mick O’Donnell and Rachel Whittaker, 125–141. London: Continuum.

Jullian, P.M. 2011. Appraising through someone else’s words: The evaluative power of quotations in news reports. Discourse & Society 22 (6): 766–780.

Kaltenbacher, M. 2006. Culture related linguistic differences in tourist websites: The emotive and the factual. A corpus analysis within the framework of appraisal. In System and Corpus: Exploring connections , ed. Geoff Thompson and Susan Hunston, 269–292. Bristol: Equinox.

Leech, G. 1991. The state of the art in corpus linguistics. In English Corpus linguistics , ed. K. Aijmer and B. Altenberg, 8–39. London: Longman.

Macken-Horarik, M., and A. Isaac. 2014. Appraising appraisal. In Evaluation in context , ed. Geoff Thompson and Laura Alba-Juez, 67–92. Amsterdam: Benjamins.

Martin, J.R., and P.R.R. White. 2005. The language of evaluation: Appraisal in English . Basingstoke: Palgrave Macmillan.

Matthiessen, C.M.I.M. 2006. Frequency profiles of some basic grammatical systems: An interim report. In System and Corpus: Exploring connections , ed. Geoff Thompson and Susan Hunston, 103–142. Bristol: Equinox.

Mayr, A. 2008. Introduction: Power, discourse and institutions. In Language and power: An introduction to institutional discourse , ed. Andrea Mayr, 1–25. London: Continuum.

Miller, D.R. 2006. From concordance to text: Appraising ‘giving’ in Alma mater donation requests. In System and Corpus: Exploring connections , ed. Geoff Thompson and Susan Hunston, 248–268. Bristol: Equinox.

O'Donnell, M. 2007. UAM CorpusTool. Version 2.8.14. http://www.corpustool.com/ .

Thompson, G. 2004. Introducing functional grammar . London: Arnold.

Thompson, G. 2014. Affect and emotion, target-value mismatches and Russian dolls: Refining the appraisal model. In Evaluation in context , ed. Geoff Thompson and Laura Alba-Juez, 47–66. Amsterdam: Benjamins.

Thompson, G., and S. Hunston. 2006. System and corpus: Two traditions with a common ground. In System and Corpus: Exploring connections , ed. Geoff Thompson and Susan Hunston, 1–14. Bristol: Equinox.

van Dijk, T.A. 1998. Ideology: A multidisciplinary approach . London: Sage.

White, P.R.R. 2008. Praising and blaming, applauding and disparaging – Solidarity, audience positioning, and the linguistics of evaluative disposition. In Handbook of interpersonal communication , ed. Gerd Antos and Eija Ventola, 542–567. New York, NY: Mouton de Gruyter.

White, P.R.R. 2015. Appraisal theory. In International encyclopedia of language and social interaction , ed. Karen Tracy, Cornelia Ilie, and Todd Sandel, 1–7. Hoboken, NJ: Wiley-Blackwell.

Wu, C. 2009. Corpus-based research. In Continuum companion to systemic functional linguistics , ed. M.A.K. Halliday and Jonathan J. Webster, 128–142. London: Continuum.

Tupala, Mira. In preparation (n.d.). Balancing between economic gain and solidarity: Underlying attitudes and ideologies in official EU migration documents. University of Turku.

Download references

Acknowledgements

Not applicable.

The PhD project that this article is a part of received a one-year working grant from Turku University Foundation in 2017.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

School of Languages and Translation Studies, University of Turku, Turku, Finland

Mira Tupala

You can also search for this author in PubMed   Google Scholar

Contributions

MT analysed and interpreted the corpus data and wrote the whole manuscript and is therefore responsible for its content. The author read and approved the final manuscript.

Corresponding author

Correspondence to Mira Tupala .

Ethics declarations

Authors’ information.

MT has an MA in English Philology and is currently a PhD candidate at the Department of English at the University of Turku.

Ethics approval and consent to participate

Not applicable

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Tupala, M. Applying quantitative appraisal analysis to the study of institutional discourse: the case of EU migration documents. Functional Linguist. 6 , 2 (2019). https://doi.org/10.1186/s40554-018-0067-7

Download citation

Received : 31 October 2018

Accepted : 19 December 2018

Published : 14 January 2019

DOI : https://doi.org/10.1186/s40554-018-0067-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Discourse analysis
• Corpus analysis
• EU discourse

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Appraisal Theory in Translation Studies—A Systematic Literature Review

Pertanika Journal of Social Sciences and Humanities

Appraisal theory (AT) is developed from Systemic Functional Linguistics. It focuses on the interpersonal meaning of the text, which expresses the speaker’s attitude and constructs ideological space. However, AT has developed quite lately in translation studies with certain achievements as well as limitations in the previous studies. Thus, this study conducted a literature review to generalize the status quo of studies applying AT in translation from 2011 to 2021. The method of this study is a systematic literature review that selected 27 articles from three databases based on the screening procedure of PRISMA 2020. It also used qualitative synthesis to analyze the contents of the included studies in terms of genres, research foci, application of AT as well as methodologies. This study finds that AT is widely used in analyzing translation shifts, especially in the text types of politics, news and literature with the foci of appraisal shifts, ideology and positioning. However, the num...

Related Papers

Research in Applied Linguistics

Gholam-Reza Tajvidi

Translation studies (TS), as a young (inter)discipline, has partly relied on metadiscussions, critical assessments of its literature, and compilation of bibliographies to deal with certain problems of its youth, such as polarity and fragmentation. While the current status of TS shows general maturity, there are still young areas of inquiry in the discipline that are faced with similar issues. The current study is an attempt to introduce and bring together an area of research in TS that draws on appraisal theory (AT; Martin & White, 2005) to functionally and systematically study the expression of feelings, emotions, viewpoints, and intersubjective positioning in translation. It is argued that this body of research has not received enough attention, partly due to its fragmentation and diversity. Several appraisal theory-informed studies of translations in different languages, genres, and mediums are introduced and reviewed, and certain points of similarity and differences are highligh...

Misagh Shahbazi

Using Critical Discourse Analysis (CDA) as a framework for translation criticism, this study attempted to examine the reflection of ideology on translation. To this end, 'Uncle Tom's Cabin'and its translation were critically analyzed within the framework of the CDA based on Fairclough's (1989) Model. Thus, lexical items and grammatical choices which had ideological load were extracted, compared and contrasted with a qualitative approach at the micro level, focusing on the context in which the text was produced at the meso level and then, interpretation and explanation were applied at the macro level. In this respect, Fairclough (1989) expresses that formal features have three types of value: experiential, relational, and expressive that were applied in the present study hierarchically from high to low as follows: the relational, the experiential, and the expressive values. The findings showed that although the translator tried to maintain the author's ideology an...

Mir Saeed Mousavi Razavi

The present corpus-driven study addresses the current situation of translation criticisms published in print or online in the Iranian media. A sample of 17 criticisms (roughly 68,000 words altogether) from a variety of valid media outlets was compiled. Having been categorized into those with, and those without an explicit theoretical framework, the criticisms were examined on two levels: firstly, on a micro-textual level to ascertain their degree of subjectivity as well as the general features and secondly, on a macro-textual level, the aim of which was to find out the overall organizational pattern(s). The results showed that only 17 percent of the criticisms had been carried out within an explicit theoretical framework. The micro-textual analysis indicated that, despite being unsystematic, the criticisms' degree of subjectivity is relatively low: 0.81 percent in the first and 2.8 percent in the second category. The macro-structural analysis revealed interesting similarities and differences within and between the two categories, most strikingly the resemblance between the macro-textual structure of criticisms with an explicit framework and that of academic papers. The findings of this study might contribute to the literature on translation criticism and review in terms of both theory and practice.

Different Perspectives in Translation Studies

Pelin Korkmaz

Translation is a very old and significant phenomenon which is essential for communication, interaction and mutual understanding between different cultures around the world. It has been academically discussed and studied for years, and its scientific methodology and understanding have undergone many changes which could be called as ‘turns’ (linguistic turn, cultural turn and ‘social’ turn) until today, in accordance with the changing dynamics in the world. However, Translation Studies (TS) became an independent discipline just in 1970s. Within today’s academic understanding and perspective, TS, particularly Descriptive-Explanatory Translation Studies (DETS), evaluates and analyzes translations (both as products and processes) and all agents involved in translation process ranging from translators to editors, clients, publishing houses, to receptors in a socio-cultural framework, which takes power relations, ideologies, patronage and norms in the source and/or target language/culture into consideration. On the other hand, Discourse Analysis (DA) studies the reciprocal and dialectic relationship between discourse, namely text and talk, and social structures, such as power relations and ideologies. It has adopted a critical perspective in time and became Critical Discourse Analysis (CDA). Therefore, in this paper, it is demonstrated that DETS and CDA are related with one another regarding the socio-cultural/political context governing the text production and reception processes. Key words: Translation Studies, Descriptive Translation Studies, Discourse Analysis, Critical Discourse Analysis, norms, ideology, power relations, socio-cultural framework

Exploring dalogism in newspaper articles by male and female writers: An analysis of Saudi women’s empowerment from the perspective of the Engagement system of the Appraisal theory

Prof. Hesham Suleiman Alyousef

The present study focuses on the uses of heteroglossic formulations in the Arab News Journal articles about women’s empowerment in Saudi Arabia. More specifically, this study explains how journalists show their position regarding women’s empowerment in Saudi Arabia and how male/female journalists engage with Saudi women’s empowerment by using the Engagement system features. Appraisal theory was employed in the present study because it focuses on interpersonal meanings that provide writers and speakers with the means to be critical, and value, reject, accept, and challenge other positions mentioned in the articles. The findings revealed that the highest number of engagement instances are specifically acknowledgment, endorsement, and contractions resources which shows that the journal admits and appreciates the other sources included in the articles. Moreover, female writers utilized more acknowledgment resources while male journalists used more endorsement and contraction resources. Thus, the authors mostly employed three engagement features to attribute other stances in their articles supporting women’s empowerment in Saudi Arabia. Consequently, they are recognizing and appreciating Saudi women’s empowerment. To the best of our knowledge, this study is the first of its kind to investigate Saudi women’s empowerment in the Saud media. Studies on the same topic using other journals or different columns are recommended. The study contributes some understanding of interpersonal meaning in the news discourse in Saudi Arabia from the functional perspective of appraisal theory.

Yeong-Houn Yi

International Journal of English Language & Translation Studies

International Journal of English Language and Translation Studies

This research attempted to investigate the relationship between the attitudes of translators towards politics and the quality of their political translations. To this end, 30 students, majoring in English translation, were selected as the participants of the study. An attitude questionnaire was devised and administered to estimate the attitudes of the translator trainees towards politics. They were then asked to translate extracts of three political texts taken from “BBC News”. The translations were rated and the results were compared with their responses to the attitude questionnaire. The study followed an ex post facto design with no treatment. Analysis revealed a significant positive relationship between translator trainees’ attitudes towards politics and the quality of their renderings of political texts. Based upon the findings of the study, the paper also aims at making certain useful suggestions to promote a better instruction on political translation. It briefs on how families, mass-media, most importantly, translation teachers, attempt to implant positive attitudes towards politics in students and, at the same time, how they can play a very crucial role in forming, shaping and affecting attitudes towards politics.

Reza Jelveh

Sudrutai Arunsirot

RELATED PAPERS

IIUM Medical Journal Malaysia

Nazirah Ab Mumin

Expert review of proteomics

Judit Ovádi

Brian Teaman

Zeitschrift für Naturforschung B

Tamas Komives

Revista Eletrônica do TCE-RS, Edição Especial, 30 anos da Constituição Estadual

Ana Warpechowski , Letícia Ayres Ramos

Jeannette Rodgers

Informatică economică

Constantin-Gelu Apostol

The Journal of Clinical Endocrinology & Metabolism

Rose Webster

Eric A Dunipace

Anales de Pediatría

Carlos Rodrigo

Kevin S Powell

Lecture Notes in Computer Science

Emmanuel Coquery

Environmental Modeling & Assessment

Mantu Kumar

Eva Zámečníková

Journal of Geophysical Research: Atmospheres

Paul Pettré

Frontiers in Genetics

Inna Inashkina

Earth and Planetary Science Letters

omar torres

Mounira Chelbi-Alix

Christophe Morhange

Erick R Torrico

Journal of Vision

Daniel Arias Osorio

Caroline DECK

rocha ROCHA

Frontiers in Physiology

See More Documents Like This

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Encyclopedia of Personality and Individual Differences pp 232–240 Cite as

Appraisal Theory of Emotion

• Agnes Moors 3 , 4  
• Reference work entry
• First Online: 01 January 2020

1421 Accesses

11 Citations

Cognitive emotion theory ; Evaluation theory of emotion ; Information processing theory of emotion

Appraisal theory of emotion proposes that emotions or emotional components are caused and differentiated by an appraisal of the stimulus as mis/matching with goals and expectations, as easy/difficult to control, and as caused by others, themselves or impersonal circumstances.

Introduction

Appraisal theories (e.g., Arnold 1960 ; Ellsworth 2013 ; Frijda 1986 ; Lazarus 1991 ; Ortony et al. 1988 ; Roseman 2013 ; Scherer 2009 ) are a species of emotion theory. The basic premise of appraisal theories is that emotions are caused and differentiated by appraisal, a process in which values are determined for a number of appraisal factors such as goal relevance, goal in/congruence, un/expectedness, control, and agency. To situate appraisal theories in the theoretical landscape, the current chapter starts by clarifying how scientific theories often develop, and then applies it to the case...

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Aarts, H. (2007). Unconscious authorship ascription: The effects of success and effect-specific information priming on experienced authorship. Journal of Experimental Social Psychology, 43 , 119–126.

Article   Google Scholar  

Arnold, M. B. (1960). Emotion and personality . New York: Columbia University Press.

Google Scholar  

Aue, T., & Scherer, K. R. (2008). Appraisal-driven somatovisceral response patterning: Effects of intrinsic pleasantness and goal conduciveness. Biological Psychology, 79 , 158–164.

Article   PubMed   Google Scholar  

Berns, G. S., Cohen, J. D., & Mintun, M. A. (1997). Brain regions responsive to novelty in the absence of awareness. Science, 276 , 1272–1275.

Bossuyt, E., Moors, A., & De Houwer, J. (2014). Unexpected and just missed: The separate influence of the appraisals of expectancy and proximity on negative emotions. Emotion, 14 , 284–300.

Brosch, T., & Sander, D. (2013). Comment: The appraising brain: Towards a neuro-cognitive model of appraisal processes in emotion. Emotion Review, 5 , 163–168.

Clore, G. L., & Ortony, A. (2013). Psychological construction in the OCC model of emotion. Emotion Review, 5 , 335–343.

Article   PubMed   PubMed Central   Google Scholar  

Ellsworth, P. C. (1994). Levels of thought and levels of emotion. In P. Ekman & R. J. Davidson (Eds.), The nature of emotion: Fundamental questions (pp. 192–196). Oxford: Oxford University Press.

Ellsworth, P. C. (2013). Appraisal theory: Old and new questions. Emotion Review, 5 , 119–124.

Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., & Kardes, F. R. (1986). On the automatic activation of attitudes. Journal of Personality and Social Psychology, 50 , 229–238.

Frijda, N. H. (1986). The emotions . New York: Cambridge University Press.

Frijda, N. H. (1988). The laws of emotion. American Psychologist, 43 , 349–358.

Frijda, N. H. (2007). The laws of emotion . Mahwah, NJ: Erlbaum.

Frijda, N. H. (2009). Emotions, individual differences and time course: Reflections. Cognition and Emotion, 23 , 1444–1461.

Frijda, N. H., Kuipers, P., & ter Schure, L. (1989). Relations between emotion, appraisal and emotional action readiness. Journal of Personality and Social Psychology, 57 , 212–228.

Frijda, N. H., & Zeelenberg, M. (2001). What is the dependent? In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 141–155). New York: Oxford University Press.

Gati, I., & Ben-Shakar, G. (1990). Novelty and significance in orientation and habituation: A feature-matching approach. Journal of Experimental Psychology: General, 119 , 251–263.

Grandjean, D., & Scherer, K. R. (2008). Unpacking the cognitive architecture of emotion processes. Emotion, 8 , 341–351.

Laird, J. D., & Bresler, C. (1992). The process of emotional experience: A self-perception theory. In M. S. Clark (Ed.), Review of personality and social psychology: Emotion (Vol. 13, pp. 213–234). Newbury Park: Sage.

Lazarus, R. S. (1991). Emotion and adaptation . New York: Oxford University Press.

Leventhal, H., & Scherer, K. R. (1987). The relationship of emotion to cognition: A functional approach to a semantic controversy. Cognition and Emotion, 1 , 3–28.

Mesquita, B., & Ellsworth, P. (2001). The role of culture in appraisal. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 233–248). New York: Oxford University Press.

Moors, A. (2007). Can cognitive methods be used to study the unique aspect of emotion: An appraisal theorist’s answer. Cognition and Emotion, 21 , 1238–1269.

Moors, A. (2010). Automatic constructive appraisal as a candidate cause of emotion. Emotion Review, 2 , 139–156.

Moors, A. (2013). On the causal role of appraisal in emotion. Emotion Review, 5 , 132–140.

Moors, A. (2014). Flavors of appraisal theories of emotion. Emotion Review, 4 , 303–307.

Moors, A., & De Houwer, J. (2005). Automatic processing of dominance and submissiveness. Experimental Psychology, 52 (4), 296–302.

Moors, A., De Houwer, J., & Eelen, P. (2004). Automatic stimulus-goal comparisons: Support from motivational affective priming studies. Cognition and Emotion, 18 , 29–54.

Moors, A., & Scherer, K. R. (2013). The role of appraisal in emotion. In M. Robinson, E. Watkins, & E. Harmon-Jones (Eds.), Handbook of cognition and emotion (pp. 135–155). New York: Guilford Press.

Oatley, K., & Johnson-Laird, P. N. (1987). Towards a cognitive theory of emotions. Cognition and Emotion, 1 , 29–50.

Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions . New York: Cambridge University Press.

Book   Google Scholar  

Kuppens, P., Van Mechelen, I., Smits, D. J. M., De Boeck, P., & Ceulemans, E. (2007). Individual differences in patterns of appraisal and anger experience. Cognition & Emotion, 21 , 689–713.

Parkinson, B. (1997). Untangling the appraisal-emotion connection. Personality and Social Psychology Review, 1 , 62–79.

Robinson, M. D., & Clore, G. L. (2002). Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychological Bulletin, 128 , 934–960.

Roseman, I. J. (2013). Appraisal in the emotion system: Coherence in strategies for coping. Emotion Review, 5 , 141–149.

Roseman, I. J., & Smith, C. A. (2001). Appraisal theory: Overview, assumptions, varieties, controversies. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 3–34). New York: Oxford University Press.

Sander, D., Grafman, J., & Zalla, T. (2003). The human amygdala: An evolved system for relevance detection. Reviews in the Neurosciences, 14 , 303–316.

Scherer, K. R. (1993). Neuroscience projections to current debates in emotion psychology. Cognition and Emotion, 7 , 1–41.

Scherer, K. R. (2001). Appraisal considered as a process of multilevel sequential checking. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion (pp. 92–120). New York: Oxford University Press.

Scherer, K. R. (2009). The dynamic architecture of emotion: Evidence for the component process model. Cognition and Emotion, 23 , 1307–1351.

Scherer, K. R., & Brosch, T. (2009). Culture-specific appraisal biases contribute to emotion dispositions. European Journal of Personality, 23 , 265–288.

Scherer, K. R., & Ellgring, H. (2007). Are facial expressions of emotion produced by categorical affect programs or dynamically driven by appraisal? Emotion, 7 , 113–130.

Scholl, B. J., & Tremoulet, P. D. (2000). Perceptual causality and animacy. Trends in cognitive science, 4 , 299–309.

Smith, C. A. (1989). Dimensions of appraisal and physiological response in emotion. Journal of Personality and Social Psychology, 56 , 339–353.

Smith, C. A., & Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology, 48 , 813–838.

Smith, C. A., & Ellsworth, P. C. (1987). Patterns of appraisal and emotion related to taking an exam. Journal of Personality and Social Psychology, 52 , 475–488.

Smith, C. A., & Kirby, L. D. (2001). Toward delivering on the promise of appraisal theory. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 121–138). New York: Oxford University Press.

van Reekum, C., Banse, R., Johnstone, T., Etter, A., Wehrle, T., & Scherer, K. R. (2004). Psychophysiological responses to appraisal responses in a computer game. Cognition and Emotion, 18 , 663–688.

Download references

Author information

Authors and affiliations.

Research Group of Quantitative Psychology and Individual Differences; Centre for Social and Cultural Psychology, KU Leuven, Leuven, Belgium

Agnes Moors

Department of Experimental-Clinical and Health Psychology, Ghent University, Ghent, Belgium

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Agnes Moors .

Editor information

Editors and affiliations.

Oakland University, Rochester, MI, USA

Virgil Zeigler-Hill

Todd K. Shackelford

Section Editor information

Institute of Psychology, University of Lodz, Lodz, Poland

Monika Wróbel

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this entry

Cite this entry.

Moors, A. (2020). Appraisal Theory of Emotion. In: Zeigler-Hill, V., Shackelford, T.K. (eds) Encyclopedia of Personality and Individual Differences. Springer, Cham. https://doi.org/10.1007/978-3-319-24612-3_493

Download citation

DOI : https://doi.org/10.1007/978-3-319-24612-3_493

Published : 22 April 2020

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-24610-9

Online ISBN : 978-3-319-24612-3

eBook Packages : Behavioral Science and Psychology Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Advanced Search
• Journal List
• Psicol Reflex Crit
• v.34; 2021 Dec

Appraising psychotherapy case studies in practice-based evidence: introducing Case Study Evaluation-tool (CaSE)

Greta kaluzeviciute.

Department of Psychosocial and Psychoanalytic Studies, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ UK

Associated Data

Not applicable.

Systematic case studies are often placed at the low end of evidence-based practice (EBP) due to lack of critical appraisal. This paper seeks to attend to this research gap by introducing a novel Case Study Evaluation-tool (CaSE). First, issues around knowledge generation and validity are assessed in both EBP and practice-based evidence (PBE) paradigms. Although systematic case studies are more aligned with PBE paradigm, the paper argues for a complimentary, third way approach between the two paradigms and their ‘exemplary’ methodologies: case studies and randomised controlled trials (RCTs). Second, the paper argues that all forms of research can produce ‘valid evidence’ but the validity itself needs to be assessed against each specific research method and purpose. Existing appraisal tools for qualitative research (JBI, CASP, ETQS) are shown to have limited relevance for the appraisal of systematic case studies through a comparative tool assessment. Third, the paper develops purpose-oriented evaluation criteria for systematic case studies through CaSE Checklist for Essential Components in Systematic Case Studies and CaSE Purpose-based Evaluative Framework for Systematic Case Studies. The checklist approach aids reviewers in assessing the presence or absence of essential case study components (internal validity). The framework approach aims to assess the effectiveness of each case against its set out research objectives and aims (external validity), based on different systematic case study purposes in psychotherapy. Finally, the paper demonstrates the application of the tool with a case example and notes further research trajectories for the development of CaSE tool.

Introduction

Due to growing demands of evidence-based practice, standardised research assessment and appraisal tools have become common in healthcare and clinical treatment (Hannes, Lockwood, & Pearson, 2010 ; Hartling, Chisholm, Thomson, & Dryden, 2012 ; Katrak, Bialocerkowski, Massy-Westropp, Kumar, & Grimmer, 2004 ). This allows researchers to critically appraise research findings on the basis of their validity, results, and usefulness (Hill & Spittlehouse, 2003 ). Despite the upsurge of critical appraisal in qualitative research (Williams, Boylan, & Nunan, 2019 ), there are no assessment or appraisal tools designed for psychotherapy case studies.

Although not without controversies (Michels, 2000 ), case studies remain central to the investigation of psychotherapy processes (Midgley, 2006 ; Willemsen, Della Rosa, & Kegerreis, 2017 ). This is particularly true of systematic case studies, the most common form of case study in contemporary psychotherapy research (Davison & Lazarus, 2007 ; McLeod & Elliott, 2011 ).

Unlike the classic clinical case study, systematic cases usually involve a team of researchers, who gather data from multiple different sources (e.g., questionnaires, observations by the therapist, interviews, statistical findings, clinical assessment, etc.), and involve a rigorous data triangulation process to assess whether the data from different sources converge (McLeod, 2010 ). Since systematic case studies are methodologically pluralistic, they have a greater interest in situating patients within the study of a broader population than clinical case studies (Iwakabe & Gazzola, 2009 ). Systematic case studies are considered to be an accessible method for developing research evidence-base in psychotherapy (Widdowson, 2011 ), especially since they correct some of the methodological limitations (e.g. lack of ‘third party’ perspectives and bias in data analysis) inherent to classic clinical case studies (Iwakabe & Gazzola, 2009 ). They have been used for the purposes of clinical training (Tuckett, 2008 ), outcome assessment (Hilliard, 1993 ), development of clinical techniques (Almond, 2004 ) and meta-analysis of qualitative findings (Timulak, 2009 ). All these developments signal a revived interest in the case study method, but also point to the obvious lack of a research assessment tool suitable for case studies in psychotherapy (Table ​ (Table1 1 ).

Key concept: systematic case study

To attend to this research gap, this paper first reviews issues around the conceptualisation of validity within the paradigms of evidence-based practice (EBP) and practice-based evidence (PBE). Although case studies are often positioned at the low end of EBP (Aveline, 2005 ), the paper suggests that systematic cases are a valuable form of evidence, capable of complimenting large-scale studies such as randomised controlled trials (RCTs). However, there remains a difficulty in assessing the quality and relevance of case study findings to broader psychotherapy research.

As a way forward, the paper introduces a novel Case Study Evaluation-tool (CaSE) in the form of CaSE Purpose - based Evaluative Framework for Systematic Case Studies and CaSE Checklist for Essential Components in Systematic Case Studies . The long-term development of CaSE would contribute to psychotherapy research and practice in three ways.

Given the significance of methodological pluralism and diverse research aims in systematic case studies, CaSE will not seek to prescribe explicit case study writing guidelines, which has already been done by numerous authors (McLeod, 2010 ; Meganck, Inslegers, Krivzov, & Notaerts, 2017 ; Willemsen et al., 2017 ). Instead, CaSE will enable the retrospective assessment of systematic case study findings and their relevance (or lack thereof) to broader psychotherapy research and practice. However, there is no reason to assume that CaSE cannot be used prospectively (i.e. producing systematic case studies in accordance to CaSE evaluative framework, as per point 3 in Table ​ Table2 2 ).

How can Case Study Evaluation-tool (CaSE) be used in psychotherapy research and practice?

The development of a research assessment or appraisal tool is a lengthy, ongoing process (Long & Godfrey, 2004 ). It is particularly challenging to develop a comprehensive purpose - oriented evaluative framework, suitable for the assessment of diverse methodologies, aims and outcomes. As such, this paper should be treated as an introduction to the broader development of CaSE tool. It will introduce the rationale behind CaSE and lay out its main approach to evidence and evaluation, with further development in mind. A case example from the Single Case Archive (SCA) ( https://singlecasearchive.com ) will be used to demonstrate the application of the tool ‘in action’. The paper notes further research trajectories and discusses some of the limitations around the use of the tool.

Separating the wheat from the chaff: what is and is not evidence in psychotherapy (and who gets to decide?)

The common approach: evidence-based practice.

In the last two decades, psychotherapy has become increasingly centred around the idea of an evidence-based practice (EBP). Initially introduced in medicine, EBP has been defined as ‘conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients’ (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996 ). EBP revolves around efficacy research: it seeks to examine whether a specific intervention has a causal (in this case, measurable) effect on clinical populations (Barkham & Mellor-Clark, 2003 ). From a conceptual standpoint, Sackett and colleagues defined EBP as a paradigm that is inclusive of many methodologies, so long as they contribute towards clinical decision-making process and accumulation of best currently available evidence in any given set of circumstances (Gabbay & le May, 2011 ). Similarly, the American Psychological Association (APA, 2010 ) has recently issued calls for evidence-based systematic case studies in order to produce standardised measures for evaluating process and outcome data across different therapeutic modalities.

However, given EBP’s focus on establishing cause-and-effect relationships (Rosqvist, Thomas, & Truax, 2011 ), it is unsurprising that qualitative research is generally not considered to be ‘gold standard’ or ‘efficacious’ within this paradigm (Aveline, 2005 ; Cartwright & Hardie, 2012 ; Edwards, 2013 ; Edwards, Dattilio, & Bromley, 2004 ; Longhofer, Floersch, & Hartmann, 2017 ). Qualitative methods like systematic case studies maintain an appreciation for context, complexity and meaning making. Therefore, instead of measuring regularly occurring causal relations (as in quantitative studies), the focus is on studying complex social phenomena (e.g. relationships, events, experiences, feelings, etc.) (Erickson, 2012 ; Maxwell, 2004 ). Edwards ( 2013 ) points out that, although context-based research in systematic case studies is the bedrock of psychotherapy theory and practice, it has also become shrouded by an unfortunate ideological description: ‘anecdotal’ case studies (i.e. unscientific narratives lacking evidence, as opposed to ‘gold standard’ evidence, a term often used to describe the RCT method and the therapeutic modalities supported by it), leading to a further need for advocacy in and defence of the unique epistemic process involved in case study research (Fishman, Messer, Edwards, & Dattilio, 2017 ).

The EBP paradigm prioritises the quantitative approach to causality, most notably through its focus on high generalisability and the ability to deal with bias through randomisation process. These conditions are associated with randomised controlled trials (RCTs) but are limited (or, as some argue, impossible) in qualitative research methods such as the case study (Margison et al., 2000 ) (Table ​ (Table3 3 ).

Key concept: evidence-based practice (EBP)

‘Evidence’ from an EBP standpoint hovers over the epistemological assumption of procedural objectivity : knowledge can be generated in a standardised, non-erroneous way, thus producing objective (i.e. with minimised bias) data. This can be achieved by anyone, as long as they are able to perform the methodological procedure (e.g. RCT) appropriately, in a ‘clearly defined and accepted process that assists with knowledge production’ (Douglas, 2004 , p. 131). If there is a well-outlined quantitative form for knowledge production, the same outcome should be achieved regardless of who processes or interprets the information. For example, researchers using Cochrane Review assess the strength of evidence using meticulously controlled and scrupulous techniques; in turn, this minimises individual judgment and creates unanimity of outcomes across different groups of people (Gabbay & le May, 2011 ). The typical process of knowledge generation (through employing RCTs and procedural objectivity) in EBP is demonstrated in Fig. ​ Fig.1 1 .

Typical knowledge generation process in evidence–based practice (EBP)

In EBP, the concept of validity remains somewhat controversial, with many critics stating that it limits rather than strengthens knowledge generation (Berg, 2019 ; Berg & Slaattelid, 2017 ; Lilienfeld, Ritschel, Lynn, Cautin, & Latzman, 2013 ). This is because efficacy research relies on internal validity . At a general level, this concept refers to the congruence between the research study and the research findings (i.e. the research findings were not influenced by anything external to the study, such as confounding variables, methodological errors and bias); at a more specific level, internal validity determines the extent to which a study establishes a reliable causal relationship between an independent variable (e.g. treatment) and independent variable (outcome or effect) (Margison et al., 2000 ). This approach to validity is demonstrated in Fig. ​ Fig.2 2 .

Internal validity

Social scientists have argued that there is a trade-off between research rigour and generalisability: the more specific the sample and the more rigorously defined the intervention, the outcome is likely to be less applicable to everyday, routine practice. As such, there remains a tension between employing procedural objectivity which increases the rigour of research outcomes and applying such outcomes to routine psychotherapy practice where scientific standards of evidence are not uniform.

According to McLeod ( 2002 ), inability to address questions that are most relevant for practitioners contributed to a deepening research–practice divide in psychotherapy. Studies investigating how practitioners make clinical decisions and the kinds of evidence they refer to show that there is a strong preference for knowledge that is not generated procedurally, i.e. knowledge that encompasses concrete clinical situations, experiences and techniques. A study by Stewart and Chambless ( 2007 ) sought to assess how a larger population of clinicians (under APA, from varying clinical schools of thought and independent practices, sample size 591) make treatment decisions in private practice. The study found that large-scale statistical data was not the primary source of information sought by clinicians. The most important influences were identified as past clinical experiences and clinical expertise ( M = 5.62). Treatment materials based on clinical case observations and theory ( M = 4.72) were used almost as frequently as psychotherapy outcome research findings ( M = 4.80) (i.e. evidence-based research). These numbers are likely to fluctuate across different forms of psychotherapy; however, they are indicative of the need for research about routine clinical settings that does not isolate or generalise the effect of an intervention but examines the variations in psychotherapy processes.

The alternative approach: practice-based evidence

In an attempt to dissolve or lessen the research–practice divide, an alternative paradigm of practice-based evidence (PBE) has been suggested (Barkham & Mellor-Clark, 2003 ; Fox, 2003 ; Green & Latchford, 2012 ; Iwakabe & Gazzola, 2009 ; Laska, Motulsky, Wertz, Morrow, & Ponterotto, 2014 ; Margison et al., 2000 ). PBE represents a shift in how we think about evidence and knowledge generation in psychotherapy. PBE treats research as a local and contingent process (at least initially), which means it focuses on variations (e.g. in patient symptoms) and complexities (e.g. of clinical setting) in the studied phenomena (Fox, 2003 ). Moreover, research and theory-building are seen as complementary rather than detached activities from clinical practice. That is to say, PBE seeks to examine how and which treatments can be improved in everyday clinical practice by flagging up clinically salient issues and developing clinical techniques (Barkham & Mellor-Clark, 2003 ). For this reason, PBE is concerned with the effectiveness of research findings: it evaluates how well interventions work in real-world settings (Rosqvist et al., 2011 ). Therefore, although it is not unlikely for RCTs to be used in order to generate practice-informed evidence (Horn & Gassaway, 2007 ), qualitative methods like the systematic case study are seen as ideal for demonstrating the effectiveness of therapeutic interventions with individual patients (van Hennik, 2020 ) (Table ​ (Table4 4 ).

Key concept: practice-based evidence (PBE)

PBE’s epistemological approach to ‘evidence’ may be understood through the process of concordant objectivity (Douglas, 2004 ): ‘Instead of seeking to eliminate individual judgment, … [concordant objectivity] checks to see whether the individual judgments of people in fact do agree’ (p. 462). This does not mean that anyone can contribute to the evaluation process like in procedural objectivity, where the main criterion is following a set quantitative protocol or knowing how to operate a specific research design. Concordant objectivity requires that there is a set of competent observers who are closely familiar with the studied phenomenon (e.g. researchers and practitioners who are familiar with depression from a variety of therapeutic approaches).

Systematic case studies are a good example of PBE ‘in action’: they allow for the examination of detailed unfolding of events in psychotherapy practice, making it the most pragmatic and practice-oriented form of psychotherapy research (Fishman, 1999 , 2005 ). Furthermore, systematic case studies approach evidence and results through concordant objectivity (Douglas, 2004 ) by involving a team of researchers and rigorous data triangulation processes (McLeod, 2010 ). This means that, although systematic case studies remain focused on particular clinical situations and detailed subjective experiences (similar to classic clinical case studies; see Iwakabe & Gazzola, 2009 ), they still involve a series of validity checks and considerations on how findings from a single systematic case pertain to broader psychotherapy research (Fishman, 2005 ). The typical process of knowledge generation (through employing systematic case studies and concordant objectivity) in PBE is demonstrated in Fig. ​ Fig.3. 3 . The figure exemplifies a bidirectional approach to research and practice, which includes the development of research-supported psychological treatments (through systematic reviews of existing evidence) as well as the perspectives of clinical practitioners in the research process (through the study of local and contingent patient and/or treatment processes) (Teachman et al., 2012 ; Westen, Novotny, & Thompson-Brenner, 2004 ).

Typical knowledge generation process in practice-based evidence (PBE)

From a PBE standpoint, external validity is a desirable research condition: it measures extent to which the impact of interventions apply to real patients and therapists in everyday clinical settings. As such, external validity is not based on the strength of causal relationships between treatment interventions and outcomes (as in internal validity); instead, the use of specific therapeutic techniques and problem-solving decisions are considered to be important for generalising findings onto routine clinical practice (even if the findings are explicated from a single case study; see Aveline, 2005 ). This approach to validity is demonstrated in Fig. ​ Fig.4 4 .

External validity

Since effectiveness research is less focused on limiting the context of the studied phenomenon (indeed, explicating the context is often one of the research aims), there is more potential for confounding factors (e.g. bias and uncontrolled variables) which in turn can reduce the study’s internal validity (Barkham & Mellor-Clark, 2003 ). This is also an important challenge for research appraisal. Douglas ( 2004 ) argues that appraising research in terms of its effectiveness may produce significant disagreements or group illusions, since what might work for some practitioners may not work for others: ‘It cannot guarantee that values are not influencing or supplanting reasoning; the observers may have shared values that cause them to all disregard important aspects of an event’ (Douglas, 2004 , p. 462). Douglas further proposes that an interactive approach to objectivity may be employed as a more complex process in debating the evidential quality of a research study: it requires a discussion among observers and evaluators in the form of peer-review, scientific discourse, as well as research appraisal tools and instruments. While these processes of rigour are also applied in EBP, there appears to be much more space for debate, disagreement and interpretation in PBE’s approach to research evaluation, partly because the evaluation criteria themselves are subject of methodological debate and are often employed in different ways by researchers (Williams et al., 2019 ). This issue will be addressed more explicitly again in relation to CaSE development (‘Developing purpose-oriented evaluation criteria for systematic case studies’ section).

A third way approach to validity and evidence

The research–practice divide shows us that there may be something significant in establishing complementarity between EBP and PBE rather than treating them as mutually exclusive forms of research (Fishman et al., 2017 ). For one, EBP is not a sufficient condition for delivering research relevant to practice settings (Bower, 2003 ). While RCTs can demonstrate that an intervention works on average in a group, clinicians who are facing individual patients need to answer a different question: how can I make therapy work with this particular case ? (Cartwright & Hardie, 2012 ). Systematic case studies are ideal for filling this gap: they contain descriptions of microprocesses (e.g. patient symptoms, therapeutic relationships, therapist attitudes) in psychotherapy practice that are often overlooked in large-scale RCTs (Iwakabe & Gazzola, 2009 ). In particular, systematic case studies describing the use of specific interventions with less researched psychological conditions (e.g. childhood depression or complex post-traumatic stress disorder) can deepen practitioners’ understanding of effective clinical techniques before the results of large-scale outcome studies are disseminated.

Secondly, establishing a working relationship between systematic case studies and RCTs will contribute towards a more pragmatic understanding of validity in psychotherapy research. Indeed, the very tension and so-called trade-off between internal and external validity is based on the assumption that research methods are designed on an either/or basis; either they provide a sufficiently rigorous study design or they produce findings that can be applied to real-life practice. Jimenez-Buedo and Miller ( 2010 ) call this assumption into question: in their view, if a study is not internally valid, then ‘little, or rather nothing, can be said of the outside world’ (p. 302). In this sense, internal validity may be seen as a pre-requisite for any form of applied research and its external validity, but it need not be constrained to the quantitative approach of causality. For example, Levitt, Motulsky, Wertz, Morrow, and Ponterotto ( 2017 ) argue that, what is typically conceptualised as internal validity, is, in fact, a much broader construct, involving the assessment of how the research method (whether qualitative or quantitative) is best suited for the research goal, and whether it obtains the relevant conclusions. Similarly, Truijens, Cornelis, Desmet, and De Smet ( 2019 ) suggest that we should think about validity in a broader epistemic sense—not just in terms of psychometric measures, but also in terms of the research design, procedure, goals (research questions), approaches to inquiry (paradigms, epistemological assumptions), etc.

The overarching argument from research cited above is that all forms of research—qualitative and quantitative—can produce ‘valid evidence’ but the validity itself needs to be assessed against each specific research method and purpose. For example, RCTs are accompanied with a variety of clearly outlined appraisal tools and instruments such as CASP (Critical Appraisal Skills Programme) that are well suited for the assessment of RCT validity and their implications for EBP. Systematic case studies (or case studies more generally) currently have no appraisal tools in any discipline. The next section evaluates whether existing qualitative research appraisal tools are relevant for systematic case studies in psychotherapy and specifies the missing evaluative criteria.

The relevance of existing appraisal tools for qualitative research to systematic case studies in psychotherapy

What is a research tool.

Currently, there are several research appraisal tools, checklists and frameworks for qualitative studies. It is important to note that tools, checklists and frameworks are not equivalent to one another but actually refer to different approaches to appraising the validity of a research study. As such, it is erroneous to assume that all forms of qualitative appraisal feature the same aims and methods (Hannes et al., 2010 ; Williams et al., 2019 ).

Generally, research assessment falls into two categories: checklists and frameworks . Checklist approaches are often contrasted with quantitative research, since the focus is on assessing the internal validity of research (i.e. researcher’s independence from the study). This involves the assessment of bias in sampling, participant recruitment, data collection and analysis. Framework approaches to research appraisal, on the other hand, revolve around traditional qualitative concepts such as transparency, reflexivity, dependability and transferability (Williams et al., 2019 ). Framework approaches to appraisal are often challenging to use because they depend on the reviewer’s familiarisation and interpretation of the qualitative concepts.

Because of these different approaches, there is some ambiguity in terminology, particularly between research appraisal instruments and research appraisal tools . These terms are often used interchangeably in appraisal literature (Williams et al., 2019 ). In this paper, research appraisal tool is defined as a method-specific (i.e. it identifies a specific research method or component) form of appraisal that draws from both checklist and framework approaches. Furthermore, a research appraisal tool seeks to inform decision making in EBP or PBE paradigms and provides explicit definitions of the tool’s evaluative framework (thus minimising—but by no means eliminating—the reviewers’ interpretation of the tool). This definition will be applied to CaSE (Table ​ (Table5 5 ).

Key concept: research appraisal tool

In contrast, research appraisal instruments are generally seen as a broader form of appraisal in the sense that they may evaluate a variety of methods (i.e. they are non-method specific or they do not target a particular research component), and are aimed at checking whether the research findings and/or the study design contain specific elements (e.g. the aims of research, the rationale behind design methodology, participant recruitment strategies, etc.).

There is often an implicit difference in audience between appraisal tools and instruments. Research appraisal instruments are often aimed at researchers who want to assess the strength of their study; however, the process of appraisal may not be made explicit in the study itself (besides mentioning that the tool was used to appraise the study). Research appraisal tools are aimed at researchers who wish to explicitly demonstrate the evidential quality of the study to the readers (which is particularly common in RCTs). All forms of appraisal used in the comparative exercise below are defined as ‘tools’, even though they have different appraisal approaches and aims.

Comparing different qualitative tools

Hannes et al. ( 2010 ) identified CASP (Critical Appraisal Skills Programme-tool), JBI (Joanna Briggs Institute-tool) and ETQS (Evaluation Tool for Qualitative Studies) as the most frequently used critical appraisal tools by qualitative researchers. All three instruments are available online and are free of charge, which means that any researcher or reviewer can readily utilise CASP, JBI or ETQS evaluative frameworks to their research. Furthermore, all three instruments were developed within the context of organisational, institutional or consortium support (Tables ​ (Tables6, 6 , ​ ,7 7 and ​ and8 8 ).

CASP (Critical Appraisal Skills Programme-tool)

JBI (Joanna Briggs Institute-tool)

ETQS (Evaluation Tool for Qualitative Studies)

It is important to note that neither of the three tools is specific to systematic case studies or psychotherapy case studies (which would include not only systematic but also experimental and clinical cases). This means that using CASP, JBI or ETQS for case study appraisal may come at a cost of overlooking elements and components specific to the systematic case study method.

Based on Hannes et al. ( 2010 ) comparative study of qualitative appraisal tools as well as the different evaluation criteria explicated in CASP, JBI and ETQS evaluative frameworks, I assessed how well each of the three tools is attuned to the methodological , clinical and theoretical aspects of systematic case studies in psychotherapy. The latter components were based on case study guidelines featured in the journal of Pragmatic Case Studies in Psychotherapy as well as components commonly used by published systematic case studies across a variety of other psychotherapy journals (e.g. Psychotherapy Research , Research In Psychotherapy : Psychopathology Process And Outcome , etc.) (see Table ​ Table9 9 for detailed descriptions of each component).

Comparing the relevance of JBI (Joanna Briggs Institute), CASP (Critical Appraisal Skills Program) and ETQS (Evaluation Tool for Qualitative Studies) for appraising components specific to systematic case studies

The evaluation criteria for each tool in Table ​ Table9 9 follows Joanna Briggs Institute (JBI) ( 2017a , 2017b ); Critical Appraisal Skills Programme (CASP) ( 2018 ); and ETQS Questionnaire (first published in 2004 but revised continuously since). Table ​ Table10 10 demonstrates how each tool should be used (i.e. recommended reviewer responses to checklists and questionnaires).

Recommended reviewer responses to JBI (Joanna Briggs Institute), CASP (Critical Appraisal Skills Program) and ETQS (Evaluation Tool for Qualitative Studies)

Using CASP, JBI and ETQS for systematic case study appraisal

Although JBI, CASP and ETQS were all developed to appraise qualitative research, it is evident from the above comparison that there are significant differences between the three tools. For example, JBI and ETQS are well suited to assess researcher’s interpretations (Hannes et al. ( 2010 ) defined this as interpretive validity , a subcategory of internal validity ): the researcher’s ability to portray, understand and reflect on the research participants’ experiences, thoughts, viewpoints and intentions. JBI has an explicit requirement for participant voices to be clearly represented, whereas ETQS involves a set of questions about key characteristics of events, persons, times and settings that are relevant to the study. Furthermore, both JBI and ETQS seek to assess the researcher’s influence on the research, with ETQS particularly focusing on the evaluation of reflexivity (the researcher’s personal influence on the interpretation and collection of data). These elements are absent or addressed to a lesser extent in the CASP tool.

The appraisal of transferability of findings (what this paper previously referred to as external validity ) is addressed only by ETQS and CASP. Both tools have detailed questions about the value of research to practice and policy as well as its transferability to other populations and settings. Methodological research aspects are also extensively addressed by CASP and ETQS, but less so by JBI (which relies predominantly on congruity between research methodology and objectives without any particular assessment criteria for other data sources and/or data collection methods). Finally, the evaluation of theoretical aspects (referred to by Hannes et al. ( 2010 ) as theoretical validity ) is addressed only by JBI and ETQS; there are no assessment criteria for theoretical framework in CASP.

Given these differences, it is unsurprising that CASP, JBI and ETQS have limited relevance for systematic case studies in psychotherapy. First, it is evident that neither of the three tools has specific evaluative criteria for the clinical component of systematic case studies. Although JBI and ETQS feature some relevant questions about participants and their context, the conceptualisation of patients (and/or clients) in psychotherapy involves other kinds of data elements (e.g. diagnostic tools and questionnaires as well as therapist observations) that go beyond the usual participant data. Furthermore, much of the clinical data is intertwined with the therapist’s clinical decision-making and thinking style (Kaluzeviciute & Willemsen, 2020 ). As such, there is a need to appraise patient data and therapist interpretations not only on a separate basis, but also as two forms of knowledge that are deeply intertwined in the case narrative.

Secondly, since systematic case studies involve various forms of data, there is a need to appraise how these data converge (or how different methods complement one another in the case context) and how they can be transferred or applied in broader psychotherapy research and practice. These systematic case study components are attended to a degree by CASP (which is particularly attentive of methodological components) and ETQS (particularly specific criteria for research transferability onto policy and practice). These components are not addressed or less explicitly addressed by JBI. Overall, neither of the tools is attuned to all methodological, theoretical and clinical components of the systematic case study. Specifically, there are no clear evaluation criteria for the description of research teams (i.e. different data analysts and/or clinicians); the suitability of the systematic case study method; the description of patient’s clinical assessment; the use of other methods or data sources; the general data about therapeutic progress.

Finally, there is something to be said about the recommended reviewer responses (Table ​ (Table10). 10 ). Systematic case studies can vary significantly in their formulation and purpose. The methodological, theoretical and clinical components outlined in Table ​ Table9 9 follow guidelines made by case study journals; however, these are recommendations, not ‘set in stone’ case templates. For this reason, the straightforward checklist approaches adopted by JBI and CASP may be difficult to use for case study researchers and those reviewing case study research. The ETQS open-ended questionnaire approach suggested by Long and Godfrey ( 2004 ) enables a comprehensive, detailed and purpose-oriented assessment, suitable for the evaluation of systematic case studies. That said, there remains a challenge of ensuring that there is less space for the interpretation of evaluative criteria (Williams et al., 2019 ). The combination of checklist and framework approaches would, therefore, provide a more stable appraisal process across different reviewers.

Developing purpose-oriented evaluation criteria for systematic case studies

The starting point in developing evaluation criteria for Case Study Evaluation-tool (CaSE) is addressing the significance of pluralism in systematic case studies. Unlike RCTs, systematic case studies are pluralistic in the sense that they employ divergent practices in methodological procedures ( research process ), and they may include significantly different research aims and purpose ( the end - goal ) (Kaluzeviciute & Willemsen, 2020 ). While some systematic case studies will have an explicit intention to conceptualise and situate a single patient’s experiences and symptoms within a broader clinical population, others will focus on the exploration of phenomena as they emerge from the data. It is therefore important that CaSE is positioned within a purpose - oriented evaluative framework , suitable for the assessment of what each systematic case is good for (rather than determining an absolute measure of ‘good’ and ‘bad’ systematic case studies). This approach to evidence and appraisal is in line with the PBE paradigm. PBE emphasises the study of clinical complexities and variations through local and contingent settings (e.g. single case studies) and promotes methodological pluralism (Barkham & Mellor-Clark, 2003 ).

CaSE checklist for essential components in systematic case studies

In order to conceptualise purpose-oriented appraisal questions, we must first look at what unites and differentiates systematic case studies in psychotherapy. The commonly used theoretical, clinical and methodological systematic case study components were identified earlier in Table ​ Table9. 9 . These components will be seen as essential and common to most systematic case studies in CaSE evaluative criteria. If these essential components are missing in a systematic case study, then it may be implied there is a lack of information, which in turn diminishes the evidential quality of the case. As such, the checklist serves as a tool for checking whether a case study is, indeed, systematic (as opposed to experimental or clinical; see Iwakabe & Gazzola, 2009 for further differentiation between methodologically distinct case study types) and should be used before CaSE Purpose - based Evaluative Framework for Systematic Case Studie s (which is designed for the appraisal of different purposes common to systematic case studies).

As noted earlier in the paper, checklist approaches to appraisal are useful when evaluating the presence or absence of specific information in a research study. This approach can be used to appraise essential components in systematic case studies, as shown below. From a pragmatic point view (Levitt et al., 2017 ; Truijens et al., 2019 ), CaSE Checklist for Essential Components in Systematic Case Studies can be seen as a way to ensure the internal validity of systematic case study: the reviewer is assessing whether sufficient information is provided about the case design, procedure, approaches to inquiry, etc., and whether they are relevant to the researcher’s objectives and conclusions (Table ​ (Table11 11 ).

Case Study Evaluation-tool (CaSE) checklist for essential components in systematic case studies. Recommended responses: Yes, No, unclear or not applicable

CaSE purpose-based evaluative framework for systematic case studies

Identifying differences between systematic case studies means identifying the different purposes systematic case studies have in psychotherapy. Based on the earlier work by social scientist Yin ( 1984 , 1993 ), we can differentiate between exploratory (hypothesis generating, indicating a beginning phase of research), descriptive (particularising case data as it emerges) and representative (a case that is typical of a broader clinical population, referred to as the ‘explanatory case’ by Yin) cases.

Another increasingly significant strand of systematic case studies is transferable (aggregating and transferring case study findings) cases. These cases are based on the process of meta-synthesis (Iwakabe & Gazzola, 2009 ): by examining processes and outcomes in many different case studies dealing with similar clinical issues, researchers can identify common themes and inferences. In this way, single case studies that have relatively little impact on clinical practice, research or health care policy (in the sense that they capture psychotherapy processes rather than produce generalisable claims as in Yin’s representative case studies) can contribute to the generation of a wider knowledge base in psychotherapy (Iwakabe, 2003 , 2005 ). However, there is an ongoing issue of assessing the evidential quality of such transferable cases. According to Duncan and Sparks ( 2020 ), although meta-synthesis and meta-analysis are considered to be ‘gold standard’ for assessing interventions across disparate studies in psychotherapy, they often contain case studies with significant research limitations, inappropriate interpretations and insufficient information. It is therefore important to have a research appraisal process in place for selecting transferable case studies.

Two other types of systematic case study research include: critical (testing and/or confirming existing theories) cases, which are described as an excellent method for falsifying existing theoretical concepts and testing whether therapeutic interventions work in practice with concrete patients (Kaluzeviciute, 2021 ), and unique (going beyond the ‘typical’ cases and demonstrating deviations) cases (Merriam, 1998 ). These two systematic case study types are often seen as less valuable for psychotherapy research given that unique/falsificatory findings are difficult to generalise. But it is clear that practitioners and researchers in our field seek out context-specific data, as well as detailed information on the effectiveness of therapeutic techniques in single cases (Stiles, 2007 ) (Table ​ (Table12 12 ).

Key concept: purpose–based systematic case studies

Each purpose-based case study contributes to PBE in different ways. Representative cases provide qualitatively rich, in-depth data about a clinical phenomenon within its particular context. This offers other clinicians and researchers access to a ‘closed world’ (Mackrill & Iwakabe, 2013 ) containing a wide range of attributes about a conceptual type (e.g. clinical condition or therapeutic technique). Descriptive cases generally seek to demonstrate a realistic snapshot of therapeutic processes, including complex dynamics in therapeutic relationships, and instances of therapeutic failure (Maggio, Molgora, & Oasi, 2019 ). Data in descriptive cases should be presented in a transparent manner (e.g. if there are issues in standardising patient responses to a self-report questionnaire, this should be made explicit). Descriptive cases are commonly used in psychotherapy training and supervision. Unique cases are relevant for both clinicians and researchers: they often contain novel treatment approaches and/or introduce new diagnostic considerations about patients who deviate from the clinical population. Critical cases demonstrate the application of psychological theories ‘in action’ with particular patients; as such, they are relevant to clinicians, researchers and policymakers (Mackrill & Iwakabe, 2013 ). Exploratory cases bring new insight and observations into clinical practice and research. This is particularly useful when comparing (or introducing) different clinical approaches and techniques (Trad & Raine, 1994 ). Findings from exploratory cases often include future research suggestions. Finally, transferable cases provide one solution to the generalisation issue in psychotherapy research through the previously mentioned process of meta-synthesis. Grouped together, transferable cases can contribute to theory building and development, as well as higher levels of abstraction about a chosen area of psychotherapy research (Iwakabe & Gazzola, 2009 ).

With this plurality in mind, it is evident that CaSE has a challenging task of appraising research components that are distinct across six different types of purpose-based systematic case studies. The purpose-specific evaluative criteria in Table ​ Table13 13 was developed in close consultation with epistemological literature associated with each type of case study, including: Yin’s ( 1984 , 1993 ) work on establishing the typicality of representative cases; Duncan and Sparks’ ( 2020 ) and Iwakabe and Gazzola’s ( 2009 ) case selection criteria for meta-synthesis and meta-analysis; Stake’s ( 1995 , 2010 ) research on particularising case narratives; Merriam’s ( 1998 ) guidelines on distinctive attributes of unique case studies; Kennedy’s ( 1979 ) epistemological rules for generalising from case studies; Mahrer’s ( 1988 ) discovery oriented case study approach; and Edelson’s ( 1986 ) guidelines for rigorous hypothesis generation in case studies.

Case Study Evaluation-tool (CaSE) purpose-based evaluative framework for systematic case studies. Recommended responses: open-ended questionnaire

Research on epistemic issues in case writing (Kaluzeviciute, 2021 ) and different forms of scientific thinking in psychoanalytic case studies (Kaluzeviciute & Willemsen, 2020 ) was also utilised to identify case study components that would help improve therapist clinical decision-making and reflexivity.

For the analysis of more complex research components (e.g. the degree of therapist reflexivity), the purpose-based evaluation will utilise a framework approach, in line with comprehensive and open-ended reviewer responses in ETQS (Evaluation Tool for Qualitative Studies) (Long & Godfrey, 2004 ) (Table ​ (Table13). 13 ). That is to say, the evaluation here is not so much about the presence or absence of information (as in the checklist approach) but the degree to which the information helps the case with its unique purpose, whether it is generalisability or typicality. Therefore, although the purpose-oriented evaluation criteria below encompasses comprehensive questions at a considerable level of generality (in the sense that not all components may be required or relevant for each case study), it nevertheless seeks to engage with each type of purpose-based systematic case study on an individual basis (attending to research or clinical components that are unique to each of type of case study).

It is important to note that, as this is an introductory paper to CaSE, the evaluative framework is still preliminary: it involves some of the core questions that pertain to the nature of all six purpose-based systematic case studies. However, there is a need to develop a more comprehensive and detailed CaSE appraisal framework for each purpose-based systematic case study in the future.

Using CaSE on published systematic case studies in psychotherapy: an example

To illustrate the use of CaSE Purpose - based Evaluative Framework for Systematic Case Studies , a case study by Lunn, Daniel, and Poulsen ( 2016 ) titled ‘ Psychoanalytic Psychotherapy With a Client With Bulimia Nervosa ’ was selected from the Single Case Archive (SCA) and analysed in Table ​ Table14. 14 . Based on the core questions associated with the six purpose-based systematic case study types in Table ​ Table13(1 13 (1 to 6), the purpose of Lunn et al.’s ( 2016 ) case was identified as critical (testing an existing theoretical suggestion).

Using Case Study Evaluation-tool (CaSE): Lunn et al. ( 2016 )’s case ‘ Psychoanalytic psychotherapy with a client with bulimia nervosa ’

Sometimes, case study authors will explicitly define the purpose of their case in the form of research objectives (as was the case in Lunn et al.’s study); this helps identifying which purpose-based questions are most relevant for the evaluation of the case. However, some case studies will require comprehensive analysis in order to identify their purpose (or multiple purposes). As such, it is recommended that CaSE reviewers first assess the degree and manner in which information about the studied phenomenon, patient data, clinical discourse and research are presented before deciding on the case purpose.

Although each purpose-based systematic case study will contribute to different strands of psychotherapy (theory, practice, training, etc.) and focus on different forms of data (e.g. theory testing vs extensive clinical descriptions), the overarching aim across all systematic case studies in psychotherapy is to study local and contingent processes, such as variations in patient symptoms and complexities of the clinical setting. The comprehensive framework approach will therefore allow reviewers to assess the degree of external validity in systematic case studies (Barkham & Mellor-Clark, 2003 ). Furthermore, assessing the case against its purpose will let reviewers determine whether the case achieves its set goals (research objectives and aims). The example below shows that Lunn et al.’s ( 2016 ) case is successful in functioning as a critical case as the authors provide relevant, high-quality information about their tested therapeutic conditions.

Finally, it is also possible to use CaSE to gather specific type of systematic case studies for one’s research, practice, training, etc. For example, a CaSE reviewer might want to identify as many descriptive case studies focusing on negative therapeutic relationships as possible for their clinical supervision. The reviewer will therefore only need to refer to CaSE questions in Table ​ Table13(2) 13 (2) on descriptive cases. If the reviewed cases do not align with the questions in Table ​ Table13(2), 13 (2), then they are not suitable for the CaSE reviewer who is looking for “know-how” knowledge and detailed clinical narratives.

Concluding comments

This paper introduces a novel Case Study Evaluation-tool (CaSE) for systematic case studies in psychotherapy. Unlike most appraisal tools in EBP, CaSE is positioned within purpose-oriented evaluation criteria, in line with the PBE paradigm. CaSE enables reviewers to assess what each systematic case is good for (rather than determining an absolute measure of ‘good’ and ‘bad’ systematic case studies). In order to explicate a purpose-based evaluative framework, six different systematic case study purposes in psychotherapy have been identified: representative cases (purpose: typicality), descriptive cases (purpose: particularity), unique cases (purpose: deviation), critical cases (purpose: falsification/confirmation), exploratory cases (purpose: hypothesis generation) and transferable cases (purpose: generalisability). Each case was linked with an existing epistemological network, such as Iwakabe and Gazzola’s ( 2009 ) work on case selection criteria for meta-synthesis. The framework approach includes core questions specific to each purpose-based case study (Table 13 (1–6)). The aim is to assess the external validity and effectiveness of each case study against its set out research objectives and aims. Reviewers are required to perform a comprehensive and open-ended data analysis, as shown in the example in Table ​ Table14 14 .

Along with CaSE Purpose - based Evaluative Framework (Table ​ (Table13), 13 ), the paper also developed CaSE Checklist for Essential Components in Systematic Case Studies (Table ​ (Table12). 12 ). The checklist approach is meant to aid reviewers in assessing the presence or absence of essential case study components, such as the rationale behind choosing the case study method and description of patient’s history. If essential components are missing in a systematic case study, then it may be implied that there is a lack of information, which in turn diminishes the evidential quality of the case. Following broader definitions of validity set out by Levitt et al. ( 2017 ) and Truijens et al. ( 2019 ), it could be argued that the checklist approach allows for the assessment of (non-quantitative) internal validity in systematic case studies: does the researcher provide sufficient information about the case study design, rationale, research objectives, epistemological/philosophical paradigms, assessment procedures, data analysis, etc., to account for their research conclusions?

It is important to note that this paper is set as an introduction to CaSE; by extension, it is also set as an introduction to research evaluation and appraisal processes for case study researchers in psychotherapy. As such, it was important to provide a step-by-step epistemological rationale and process behind the development of CaSE evaluative framework and checklist. However, this also means that further research needs to be conducted in order to develop the tool. While CaSE Purpose - based Evaluative Framework involves some of the core questions that pertain to the nature of all six purpose-based systematic case studies, there is a need to develop individual and comprehensive CaSE evaluative frameworks for each of the purpose-based systematic case studies in the future. This line of research is likely to enhance CaSE target audience: clinicians interested in reviewing highly particular clinical narratives will attend to descriptive case study appraisal frameworks; researchers working with qualitative meta-synthesis will find transferable case study appraisal frameworks most relevant to their work; while teachers on psychotherapy and counselling modules may seek out unique case study appraisal frameworks.

Furthermore, although CaSE Checklist for Essential Components in Systematic Case Studies and CaSE Purpose - based Evaluative Framework for Systematic Case Studies are presented in a comprehensive, detailed manner, with definitions and examples that would enable reviewers to have a good grasp of the appraisal process, it is likely that different reviewers may have different interpretations or ideas of what might be ‘substantial’ case study data. This, in part, is due to the methodologically pluralistic nature of the case study genre itself; what is relevant for one case study may not be relevant for another, and vice-versa. To aid with the review process, future research on CaSE should include a comprehensive paper on using the tool. This paper should involve evaluation examples with all six purpose-based systematic case studies, as well as a ‘search’ exercise (using CaSE to assess the relevance of case studies for one’s research, practice, training, etc.).

Finally, further research needs to be developed on how (and, indeed, whether) systematic case studies should be reviewed with specific ‘grades’ or ‘assessments’ that go beyond the qualitative examination in Table ​ Table14. 14 . This would be particularly significant for the processes of qualitative meta-synthesis and meta-analysis. These research developments will further enhance CaSE tool, and, in turn, enable psychotherapy researchers to appraise their findings within clear, purpose-based evaluative criteria appropriate for systematic case studies.

Acknowledgments

I would like to thank Prof Jochem Willemsen (Faculty of Psychology and Educational Sciences, Université catholique de Louvain-la-Neuve), Prof Wayne Martin (School of Philosophy and Art History, University of Essex), Dr Femke Truijens (Institute of Psychology, Erasmus University Rotterdam) and the reviewers of Psicologia: Reflexão e Crítica / Psychology : Research and Review for their feedback, insight and contributions to the manuscript.

Author’s contributions

GK is the sole author of the manuscript. The author(s) read and approved the final manuscript.

Arts and Humanities Research Council (AHRC) and Consortium for Humanities and the Arts South-East England (CHASE) Doctoral Training Partnership, Award Number [AH/L50 3861/1].

Availability of data and materials

Declarations.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Almond R. “I Can Do It (All) Myself”: Clinical technique with defensive narcissistic self–sufficiency. Psychoanalytic Psychology. 2004; 21 (3):371–384. doi: 10.1037/0736-9735.21.3.371. [ CrossRef ] [ Google Scholar ]
• American Psychological Association . Evidence–based case study. 2010. [ Google Scholar ]
• Aveline M. Clinical case studies: Their place in evidence–based practice. Psychodynamic Practice: Individuals, Groups and Organisations. 2005; 11 (2):133–152. doi: 10.1080/14753630500108174. [ CrossRef ] [ Google Scholar ]
• Barkham M, Mellor-Clark J. Bridging evidence-based practice and practice-based evidence: Developing a rigorous and relevant knowledge for the psychological therapies. Clinical Psychology & Psychotherapy. 2003; 10 (6):319–327. doi: 10.1002/cpp.379. [ CrossRef ] [ Google Scholar ]
• Berg H. How does evidence–based practice in psychology work? – As an ethical demarcation. Philosophical Psychology. 2019; 32 (6):853–873. doi: 10.1080/09515089.2019.1632424. [ CrossRef ] [ Google Scholar ]
• Berg H, Slaattelid R. Facts and values in psychotherapy—A critique of the empirical reduction of psychotherapy within evidence-based practice. Journal of Evaluation in Clinical Practice. 2017; 23 (5):1075–1080. doi: 10.1111/jep.12739. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bower P. Efficacy in evidence-based practice. Clinical Psychology and Psychotherapy. 2003; 10 (6):328–336. doi: 10.1002/cpp.380. [ CrossRef ] [ Google Scholar ]
• Cartwright, N., & Hardie, J. (2012). What are RCTs good for? In N. Cartwright, & J. Hardie (Eds.), Evidence–based policy: A practical guide to doing it better . Oxford University Press. 10.1093/acprof:osobl/9780199841608.003.0008.
• Critical Appraisal Skills Programme (CASP) Qualitative checklist. 2018. [ Google Scholar ]
• Davison, G. C., & Lazarus, A. A. (2007). Clinical case studies are important in the science and practice of psychotherapy. In S. O. Lilienfeld, & W. T. O’Donohue (Eds.), The great ideas of clinical science: 17 principles that every mental health professional should understand , (pp. 149–162). Routledge/Taylor & Francis Group.
• Douglas H. The irreducible complexity of objectivity. Synthese. 2004; 138 (3):453–473. doi: 10.1023/B:SYNT.0000016451.18182.91. [ CrossRef ] [ Google Scholar ]
• Duncan BL, Sparks JA. When meta–analysis misleads: A critical case study of a meta–analysis of client feedback. Psychological Services. 2020; 17 (4):487–496. doi: 10.1037/ser0000398. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Edelson, M. (1986). Causal explanation in science and in psychoanalysis—Implications for writing a case study. Psychoanalytic Study of Child , 41 (1), 89–127. 10.1080/00797308.1986.11823452. [ PubMed ]
• Edwards DJA. Collaborative versus adversarial stances in scientific discourse: Implications for the role of systematic case studies in the development of evidence–based practice in psychotherapy. Pragmatic Case Studies in Psychotherapy. 2013; 3 (1):6–34. [ Google Scholar ]
• Edwards DJA, Dattilio FM, Bromley DB. Developing evidence–based practice: The role of case–based research. Professional Psychology: Research and Practice. 2004; 35 (6):589–597. doi: 10.1037/0735-7028.35.6.589. [ CrossRef ] [ Google Scholar ]
• Erickson F. Comments on causality in qualitative inquiry. Qualitative Inquiry. 2012; 18 (8):686–688. doi: 10.1177/1077800412454834. [ CrossRef ] [ Google Scholar ]
• Fishman, D. B. (1999). The case for pragmatic psychology . New York University Press.
• Fishman DB. Editor’s introduction to PCSP––From single case to database: A new method for enhancing psychotherapy practice. Pragmatic Case Studies in Psychotherapy. 2005; 1 (1):1–50. [ Google Scholar ]
• Fishman, D. B., Messer, S. B., Edwards, D. J. A., & Dattilio, F. M. (Eds.) (2017). Case studies within psychotherapy trials: Integrating qualitative and quantitative methods . Oxford University Press.
• Fox, N. J. (2003). Practice–based evidence: Towards collaborative and transgressive research. Sociology , 37 (1), 81–102. 10.1177/0038038503037001388.
• Gabbay, J., & le May, A. (2011). Practice–based evidence for healthcare: Clinical mindlines . Routledge.
• Green, L. W., & Latchford, G. (2012). Maximising the benefits of psychotherapy: A practice–based evidence approach . Wiley–Blackwell. 10.1002/9781119967590.
• Hannes K, Lockwood C, Pearson A. A comparative analysis of three online appraisal instruments’ ability to assess validity in qualitative research. Qualitative Health Research. 2010; 20 (12):1736–1743. doi: 10.1177/1049732310378656. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012; 7 (11):e49667. doi: 10.1371/journal.pone.0049667. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hill A, Spittlehouse C. What is critical appraisal? Evidence–Based Medicine. 2003; 3 (2):1–8. [ Google Scholar ]
• Hilliard RB. Single–case methodology in psychotherapy process and outcome research. Journal of Consulting and Clinical Psychology. 1993; 61 (3):373–380. doi: 10.1037/0022-006X.61.3.373. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Horn SD, Gassaway J. Practice–based evidence study design for comparative effectiveness research. Medical Care. 2007; 45 (10):S50–S57. doi: 10.1097/MLR.0b013e318070c07b. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Iwakabe S. Paper presented at the 19th Annual Conference of the Society for Exploration of Psychotherapy Integration, New York. 2003. Common change events in stages of psychotherapy: A qualitative analysis of case reports. [ Google Scholar ]
• Iwakabe S. Pragmatic meta–analysis of case studies. Annual Progress of Family Psychology. 2005; 23 :154–169. [ Google Scholar ]
• Iwakabe S, Gazzola N. From single–case studies to practice–based knowledge: Aggregating and synthesizing case studies. Psychotherapy Research. 2009; 19 (4-5):601–611. doi: 10.1080/10503300802688494. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jimenez-Buedo M, Miller L. Why a Trade–Off? The relationship between the external and internal validity of experiments. THEORIA: An International Journal for Theory History and Foundations of Science. 2010; 25 (3):301–321. [ Google Scholar ]
• Joanna Briggs Institute (JBI) Critical appraisal checklist for qualitative research. 2017. [ Google Scholar ]
• Joanna Briggs Institute (JBI) Checklist for case reports. 2017. [ Google Scholar ]
• Kaluzeviciute, G. (2021). Validity, Evidence and Appraisal in Systematic Psychotherapy Case Studies . Paper presented at the Research Forum of Department of Psychosocial and Psychoanalytic Studies, University of Essex, Colchester, UK. 10.13140/RG.2.2.33502.15683 
• Kaluzeviciute G, Willemsen J. Scientific thinking styles: The different ways of thinking in psychoanalytic case studies. The International Journal of Psychoanalysis. 2020; 101 (5):900–922. doi: 10.1080/00207578.2020.1796491. [ CrossRef ] [ Google Scholar ]
• Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar SVS, Grimmer K. A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology. 2004; 4 (1):22. doi: 10.1186/1471-2288-4-22. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kennedy MM. Generalising from single case studies. Evaluation Quarterly. 1979; 3 (4):661–678. doi: 10.1177/0193841X7900300409. [ CrossRef ] [ Google Scholar ]
• Laska KM, Gurman AS, Wampold BE. Expanding the lens of evidence–based practice in psychotherapy: A common factors perspective. Psychotherapy. 2014; 51 (4):467–481. doi: 10.1037/a0034332. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Levitt HM, Motulsky SL, Wertz FJ, Morrow SL, Ponterotto JG. Recommendations for designing and reviewing qualitative research in psychology: Promoting methodological integrity. Qualitative Psychology. 2017; 4 (1):2–22. doi: 10.1037/qup0000082. [ CrossRef ] [ Google Scholar ]
• Lilienfeld SO, Ritschel LA, Lynn SJ, Cautin RL, Latzman RD. Why many clinical psychologists are resistant to evidence–based practice: root causes and constructive remedies. Clinical Psychology Review. 2013; 33 (7):883–900. doi: 10.1016/j.cpr.2012.09.008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Long AF, Godfrey M. An evaluation tool to assess the quality of qualitative research studies. International Journal of Social Research Methodology. 2004; 7 (2):181–196. doi: 10.1080/1364557032000045302. [ CrossRef ] [ Google Scholar ]
• Longhofer J, Floersch J, Hartmann EA. Case for the case study: How and why they matter. Clinical Social Work Journal. 2017; 45 (3):189–200. doi: 10.1007/s10615-017-0631-8. [ CrossRef ] [ Google Scholar ]
• Lunn S, Daniel SIF, Poulsen S. Psychoanalytic psychotherapy with a client with bulimia nervosa. Psychotherapy. 2016; 53 (2):206–215. doi: 10.1037/pst0000052. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mackrill T, Iwakabe S. Making a case for case studies in psychotherapy training: A small step towards establishing an empirical basis for psychotherapy training. Counselling Psychotherapy Quarterly. 2013; 26 (3–4):250–266. doi: 10.1080/09515070.2013.832148. [ CrossRef ] [ Google Scholar ]
• Maggio S, Molgora S, Oasi O. Analyzing psychotherapeutic failures: A research on the variables involved in the treatment with an individual setting of 29 cases. Frontiers in Psychology. 2019; 10 :1250. doi: 10.3389/fpsyg.2019.01250. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mahrer AR. Discovery–oriented psychotherapy research: Rationale, aims, and methods. American Psychologist. 1988; 43 (9):694–702. doi: 10.1037/0003-066X.43.9.694. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Margison FB, et al. Measurement and psychotherapy: Evidence–based practice and practice–based evidence. British Journal of Psychiatry. 2000; 177 (2):123–130. doi: 10.1192/bjp.177.2.123. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Maxwell JA. Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher. 2004; 33 (2):3–11. doi: 10.3102/0013189X033002003. [ CrossRef ] [ Google Scholar ]
• McLeod J. Case studies and practitioner research: Building knowledge through systematic inquiry into individual cases. Counselling and Psychotherapy Research: Linking research with practice. 2002; 2 (4):264–268. doi: 10.1080/14733140212331384755. [ CrossRef ] [ Google Scholar ]
• McLeod, J. (2010). Case study research in counselling and psychotherapy . SAGE Publications. 10.4135/9781446287897.
• McLeod J, Elliott R. Systematic case study research: A practice–oriented introduction to building an evidence base for counselling and psychotherapy. Counselling and Psychotherapy Research. 2011; 11 (1):1–10. doi: 10.1080/14733145.2011.548954. [ CrossRef ] [ Google Scholar ]
• Meganck R, Inslegers R, Krivzov J, Notaerts L. Beyond clinical case studies in psychoanalysis: A review of psychoanalytic empirical single case studies published in ISI–ranked journals. Frontiers in Psychology. 2017; 8 :1749. doi: 10.3389/fpsyg.2017.01749. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Merriam, S. B. (1998). Qualitative research and case study applications in education . Jossey–Bass Publishers.
• Michels R. The case history. Journal of the American Psychoanalytic Association. 2000; 48 (2):355–375. doi: 10.1177/00030651000480021201. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Midgley N. Re–reading “Little Hans”: Freud’s case study and the question of competing paradigms in psychoanalysis. Journal of the American Psychoanalytic Association. 2006; 54 (2):537–559. doi: 10.1177/00030651060540021601. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosqvist, J., Thomas, J. C., & Truax, P. (2011). Effectiveness versus efficacy studies. In J. C. Thomas, & M. Hersen (Eds.), Understanding research in clinical and counseling psychology , (pp. 319–354). Routledge/Taylor & Francis Group.
• Sackett DL, Rosenberg WM, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996; 312 (7023):71–72. doi: 10.1136/bmj.312.7023.71. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stake, R. E. (1995). The art of case study research . SAGE Publications.
• Stake, R. E. (2010). Qualitative research: Studying how things work . The Guilford Press.
• Stewart RE, Chambless DL. Does psychotherapy research inform treatment decisions in private practice? Journal of Clinical Psychology. 2007; 63 (3):267–281. doi: 10.1002/jclp.20347. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stiles WB. Theory–building case studies of counselling and psychotherapy. Counselling and Psychotherapy Research. 2007; 7 (2):122–127. doi: 10.1080/14733140701356742. [ CrossRef ] [ Google Scholar ]
• Teachman BA, Drabick DA, Hershenberg R, Vivian D, Wolfe BE, Goldfried MR. Bridging the gap between clinical research and clinical practice: introduction to the special section. Psychotherapy. 2012; 49 (2):97–100. doi: 10.1037/a0027346. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: Reflections on methodological orientation and ideological agenda. Qualitative Health Research. 2004; 14 (10):1342–1365. doi: 10.1177/1049732304269888. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Timulak L. Meta–analysis of qualitative studies: A tool for reviewing qualitative research findings in psychotherapy. Psychotherapy Research. 2009; 19 (4–5):591–600. doi: 10.1080/10503300802477989. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Trad PV, Raine MJ. A prospective interpretation of unconscious processes during psychoanalytic psychotherapy. Psychoanalytic Psychology. 1994; 11 (1):77–100. doi: 10.1037/h0079522. [ CrossRef ] [ Google Scholar ]
• Truijens, F., Cornelis, S., Desmet, M., & De Smet, M. (2019). Validity beyond measurement: Why psychometric validity is insufficient for valid psychotherapy research. Frontiers in Psychology , 10 . 10.3389/fpsyg.2019.00532. [ PMC free article ] [ PubMed ]
• Tuckett, D. (Ed.) (2008). The new library of psychoanalysis. Psychoanalysis comparable and incomparable: The evolution of a method to describe and compare psychoanalytic approaches . Routledge/Taylor & Francis Group. 10.4324/9780203932551.
• van Hennik R. Practice based evidence based practice, part II: Navigating complexity and validity from within. Journal of Family Therapy. 2020; 43 (1):27–45. doi: 10.1111/1467-6427.12291. [ CrossRef ] [ Google Scholar ]
• Westen D, Novotny CM, Thompson-Brenner H. The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin. 2004; 130 (4):631–663. doi: 10.1037/0033-2909.130.4.631. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Widdowson, M. (2011). Case study research methodology. International Journal of Transactional Analysis Research & Practice , 2 (1). 10.29044/v2i1p25.
• Willemsen, J., Della Rosa, E., & Kegerreis, S. (2017). Clinical case studies in psychoanalytic and psychodynamic treatment. Frontiers in Psychology , 8 (108). 10.3389/fpsyg.2017.00108. [ PMC free article ] [ PubMed ]
• Williams, V., Boylan, A., & Nunan, D. (2019). Critical appraisal of qualitative research: Necessity, partialities and the issue of bias. BMJ Evidence–Based Medicine . 10.1136/bmjebm-2018-111132. [ PubMed ]
• Yin, R. K. (1984). Case study research: Design and methods . SAGE Publications.
• Yin, R. K. (1993). Applications of case study research . SAGE Publications.

• Share full article

The Great Read

The lindbergh baby kidnapping: a grisly theory and a renewed debate.

New speculation about the toddler’s death and pressure to force DNA testing of evidence have revived scrutiny of what was known as the “crime of the century.”

Credit... Associated Press

Supported by

By Tracey Tully

• Published March 5, 2024 Updated March 6, 2024

Bruno Richard Hauptmann’s mug shot. The wooden electric chair where he was put to death. A sponge like the one that was dampened with salt water and placed on his head to conduct the deadly jolts of electricity.

Listen to this article with reporter commentary

Open this article in the New York Times Audio app on iOS.

This grim assortment of relics is housed in a small museum in New Jersey , about 20 miles away from where the decaying body of Charles A. Lindbergh Jr., the toddler Hauptmann was convicted of kidnapping and killing, was discovered face down in the dirt.

Nearly 100 years ago, the Lindbergh case was known as the crime of the century by virtue of its cinematic details and the boy’s high-profile parents, Anne Morrow Lindbergh, a diplomat’s daughter , and Charles A. Lindbergh, an aviator who had catapulted to fame by completing the world’s first nonstop solo flight from New York to Paris.

In the decades since, as the keepers of the Lindbergh kidnapping archives can attest, public interest in the case has never subsided — nor has skepticism about Hauptmann’s guilt. But a bizarre and grisly new theory about Lindbergh’s potential involvement in his son’s death, and renewed legal pressure to force DNA testing of evidence, have combined to thrust one of the country’s most enduring murder mysteries squarely back into public consciousness.

Hauptmann, a German immigrant who had worked as a carpenter and lived in the Bronx, was executed for the crime in April 1936. His great-great niece, Cezanne Love, and her aunt recently provided DNA samples in the hope that New Jersey’s courts would decide to clear the way for modern science to explore century-old doubts: Was an innocent man put to death? And, if not, did he act alone?

“I personally don’t think he did it,” Ms. Love said, noting that Hauptmann and his widow maintained his innocence , and his alibi, until the end. But if the evidence does link him to the case, “then so be it,” she said. “I want to uncover the truth.”

Charles Lindbergh Jr. was 20 months old when he disappeared from his bedroom in East Amwell, N.J., on March 1, 1932. A wooden ladder, a chisel and the first of more than a dozen ransom notes were found at the house after the kidnapping. The family arranged for a $50,000 payment, but the child was discovered dead 10 weeks later.

The investigation, led by the New Jersey State Police and splashed across front pages worldwide, raged on until September 1934, when a $10 gold certificate from the ransom payment was used to buy gasoline in New York.

Investigators traced the car at the gas station to Hauptmann and later discovered $13,760 in ransom bills in his garage — money he said he’d been asked to hold for a man who died in Germany before the trial.

There were no fingerprints, and there was no confession and no explanation for how a solitary kidnapper might have orchestrated a toddler’s abduction from a second-floor nursery on a rainy Tuesday evening, while five adults and a dog were in the home. Hauptmann was convicted after a six-week trial in Flemington, N.J., and sentenced to death.

At the time of the boy’s kidnapping his father was a national hero.

But history would come to view the renowned pilot who died 50 years ago far more critically. Lindbergh was fascinated with the study of eugenics and was vilified by the media after accepting a medal for his contributions to aviation from Hermann Goering on behalf of Hitler in 1938 — seen by many as a sign of his sympathies with the Nazi regime.

Lise Pearlman, a retired California judge, now speculates that Lindbergh was capable of something even more sinister: sacrificing his son for scientific experiments that led to the child’s death.

“I take advantage of the distance in time to treat the boy’s father as a potential suspect in his kidnap and murder; like all the others on the list, a fallible human being, not a demigod,” Ms. Pearlman wrote in a 2020 book, “The Lindbergh Kidnapping Suspect No. 1: The Man Who Got Away.”

It is a theory that other Lindbergh researchers view with deep skepticism.

Ms. Pearlman acknowledged that her findings were based on circumstantial, yet compelling, evidence about the crime scene, the condition of the boy’s remains and the research that Lindbergh was conducting at the time with Dr. Alexis Carrel, a Nobel Prize-winning surgeon and organ transplant pioneer.

Last February, she shared her theory at a conference held by the American Academy of Forensic Sciences, broadening the reach of her views and stoking renewed debate about the crime’s persistent riddles.

Writers who have researched the case just as extensively as Ms. Pearlman said it would be impossible to definitively prove a whole new theory 100 years later. But that has not stopped people from trying.

A former Rutgers University professor, Lloyd C. Gardner, founded a social group decades ago that called itself the State Street Irregulars, a homage to Arthur Conan Doyle’s Sherlock Holmes series. The group met at the bar of a restaurant in Lambertville, N.J., while Professor Gardner was writing his book on the subject, “The Case That Never Dies.”

“It will never be proven one way or another,” Professor Gardner said.

Push for DNA tests

Barry Scheck, a founder of the Innocence Project, which has represented roughly 200 clients exonerated by DNA evidence, was a classmate of Ms. Pearlman’s in college and in law school and wrote a supportive blurb for her book. The Innocence Project has not taken on the case, but Mr. Scheck said he had provided consultation to Ms. Pearlman.

In an interview, Mr. Scheck said his interest in the Lindbergh case was tied more to the possibility that an innocent man was executed than the identity of a culprit.

“If there is a DNA test that you can do that will shed light on the guilt or the innocence of somebody, particularly someone who was executed, I think the family has the right to get that information,” he said.

His sentiment is in line with many people’s lasting interest in the case and follows renewed efforts to determine Hauptmann’s guilt or innocence.

Lawrence S. Lustberg, a prominent criminal defense lawyer in New Jersey, said Monday that he had joined the effort on behalf of “opponents of wrongful conviction.” He said he was “carefully considering what procedural mechanisms” might be available to “redress what appears to be a terrible, historic injustice.”

There is already an unrelated legal challenge in New Jersey over a request to test the saliva on the stamped ransom envelopes for clues. Researchers have also argued that the ladder and a wooden board found in Hauptmann’s attic should be evaluated using modern methods to confirm that they match.

A Superior Court judge ruled last year against releasing the envelopes for DNA testing, and an appeals panel is expected to set a date to hear arguments on the matter in the coming months.

Kurt Perhach, who has cited the state’s open public records laws in his fight to win access to the envelopes, traces his career as a lawyer to a teenage fascination with the Lindbergh kidnapping case.

“I have felt compelled that there’s something wrong here since I was a boy,” said Mr. Perhach, 46, who grew up about 30 minutes away from the Lindberghs’ farmhouse in East Amwell. He represents a group of people, including Wayne McDaniel, a New York City-based filmmaker, hoping to use information gleaned from testing to explore a new documentary angle within the already well-plumbed Lindbergh oeuvre.

The New Jersey attorney general’s office has opposed the request, arguing that the “integrity of the historical items outweighs any interest in conducting DNA testing that will permanently alter and potentially damage the items.”

A trove of records

In 1981, a former governor of New Jersey, Brendan Byrne, issued an executive order that made all records pertaining to the Lindbergh investigation available to the public for research.

That material — roughly 225,000 documents — is housed in a large room adjacent to the Lindbergh museum gallery at the State Police’s headquarters near Trenton, N.J., and overseen by a full-time archivist, Greg Ferrara.

Visitors are free to open filing cabinets and leaf through original crime-scene photos, the trial transcript, index cards bearing names of potential witnesses, police reports and Hauptmann’s account of his life, which was typed onto translucent paper so brittle it crinkles when handled.

“Much has been said in the newspapers about my life to make me appear as a black sheep,” he wrote. “Why don’t they investigate the true course of events?”

Since then, few cases have generated more independent investigation.

Sleuths, some of them admittedly obsessed by the case, still flock to online discussion boards to trade hypotheses. The kidnapping has inspired dozens of researched books, movies, documentaries and even musicals .

“I got four calls this week,” Mr. Ferrara said, describing the level of continuing public interest.

Professor Gardner, 89, said he once suggested adding a sign above the New Jersey archive.

“Abandon all hope ye who enter here,” he said, borrowing a passage from Dante’s “Divine Comedy.”

“You get into this and you can’t get out,” he said. “It’s so intriguing.”

Del Quentin Wilber heard the same Dante line as he researched a book he is writing about the case for Simon & Schuster.

Mr. Wilber, the Washington investigations editor for The Associated Press, said he had mined hundreds of thousands of records held at the U.S. Treasury Department, the Federal Bureau of Investigation, the New Jersey archives and several universities.

“It’s still the ultimate American spectacle that draws people to it in this obsessive way,” said Mr. Wilber, who is convinced of Hauptmann’s guilt despite trial flaws that have opened the case to unceasing scrutiny.

“They railroaded a guilty man,” he said.

‘Complicated, flawed men’

One of the most grotesque theories to emerge since the boy’s death is detailed in the book by the retired judge in California.

Ms. Pearlman said she considered the apparent absence of blood in the woods where the body was discovered a smoking gun that proved the boy died somewhere else. Unexplored questions about the condition of the body and items found nearby have led her to speculate that Lindbergh colluded with Dr. Carrel, his friend who worked at the Rockefeller Institute for Medical Research in New York City, to experiment on his son.

She wrote that the boy, who had an unusually large head and took medicines associated with rickets, would have been seen as expendable to the men, who, as eugenicists , believed in improving the genetic quality of the population. She has concluded that there was a “horrendous probability” that the men removed his organs in hope of achieving a medical breakthrough that might help Lindbergh’s sister-in-law, who had a damaged heart valve.

To reach this conclusion, Ms. Pearlman worked with a New Jersey pathologist, Dr. Peter Speth, who evaluated records from the crime scene and autopsy, which showed that all the boy’s organs other than the heart and liver were missing. At the time, investigators deduced that foraging animals had mutilated the body as it lay in the woods.

Dr. Speth said the apparent absence of vermin on the boy’s remains — and clues suggesting the child’s face and one of his feet decayed more slowly than other parts of the body — indicated that the body had been dumped in the woods well after the abduction and that chemicals commonly found in laboratories had also probably been used.

“Bruno could not have conceived of it and could not have carried it out,” Dr. Speth said in an interview. He called Hauptmann’s execution “a terrible miscarriage of justice.”

The Academy of Forensic Sciences routinely includes disclaimers noting that the research presented at its conferences is unverified, and Ms. Pearlman’s findings have not been subject to peer review. But a screening team would have had to approve Ms. Pearlman’s participation in the conference, and the academy published an abstract summarizing her findings, as is routine, Jeri Ropero-Miller, a scientist and member of the academy, said.

Still, authors who have also studied the crime and its characters extensively question Ms. Pearlman’s book and its extreme conclusions.

David M. Friedman, who wrote “The Immortalists: Charles Lindbergh, Dr. Alexis Carrel and Their Daring Quest to Live Forever,” said he had not read Ms. Pearlman’s book but found the theory about Lindbergh and Carrel absurd.

“They were complicated, flawed men,” Mr. Friedman said. “But the idea that they would collaborate in an act of infanticide strikes me as malicious garbage.”

Ms. Love, Hauptmann’s relative, said she remained focused less on the whodunit than on helping to wipe away a stain that had haunted her family for nearly a century.

Her goal, she said, was to expose the limits of science and justice in the 1930s.

“With the hysteria and the horribleness of the crime, the police wanted to put a nice little bow,” she said.

“But in today’s court system it would have been a totally different outcome.”

Audio produced by Sarah Diamond .

Tracey Tully is a reporter for The Times who covers New Jersey, where she has lived for more than 20 years. More about Tracey Tully

Here are more fascinating tales you can’t help reading all the way to the end.

Saved by a Rescue Dog: He spent 13 years addicted to cocaine. Running a shelter for abused and neglected dogs in New York has kept him sober, but it hasn’t been easy .

An Art Mogul's Fall: After a dramatic rise in business and society, Louise Blouin finds herself unloading a Hamptons dream home in bankruptcy court .

America's Most Hated Man?: Tom Sandoval turned last year’s season of “Vanderpump Rules” into the best in reality TV’s history — and ruined his life in the process .

Inside an N.Y.P.D. Basement: In the office of the Manhattan Property Clerk, evidence and lost items arrive by the tens of thousands . A small band of officers and civilians has to manage never-ending pressure.

India’s Great Curry Clash: A court in Delhi has been asked to solve a bitter dispute between two families who have very different accounts of the origins of butter chicken , a dish beloved around the world.

Advertisement

share this!

March 14, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

Study finds workers misjudge wage markets

by Peter Dizikes, Massachusetts Institute of Technology

Many employees believe their counterparts at other firms make less in salary than is actually the case—an assumption that costs them money, according to a study co-authored by MIT scholars.

"Workers wrongly anchor their beliefs about outside options on their current wage," says MIT economist Simon Jäger, co-author of a newly published paper detailing the study's results.

As a top-line figure, the study indicates that workers who would experience a 10% wage increase by switching firms only expect a 1% wage increase instead, leading them to earn less than they otherwise might.

That is one of multiple related findings in the study, which also shows that workers in lower-paying firms are highly susceptible to underestimating wages at other companies; and that giving workers correct information about the salary structure in their industry makes them more likely to declare that they intend to leave their current jobs.

The study also has implications for further economics research, since economists' job-search models generally assume workers have accurate salary information about their industries. The study was performed using data from Germany, although it quite likely applies to other countries as well.

"Misperceptions about outside options have substantial consequences on wages," says Nina Roussille, an economist at MIT and also a co-author of the paper. "The intuition is simple: If low-wage workers do not know that they could make more elsewhere, then these workers stay put in low-wage firms. In turn, these low-wage firms do not feel the competitive pressure from the external labor market to raise their wages."

The paper , "Worker Beliefs about Outside Options," is published in advance online form in The Quarterly Journal of Economics .

Updating beliefs

To conduct the study, the researchers incorporated a survey module into the Innovation Sample of the German Socio-Economic Panel, an annual survey of a representative sample of the German population. They used their survey questions to find out the nature of worker beliefs about outside employment opportunities. The scholars then linked these findings to actual job and salary data collected from the German government's Institute for Employment Research (IAB), with the prior consent of 558 survey respondents.

Linking those two data sources allowed the scholars to quantify the mismatch between what workers believe about industry-wide salaries, and what wages are in reality. One good piece of evidence on the compression of those beliefs is that about 56% of respondents believe they have a salary in between the 40th and 60th percentiles among comparable workers.

The scholars then added another element to the research project. They conducted an online experiment with 2,448 participants, giving these workers correct information about salaries at other companies, and then measuring the employees' intention to find other job opportunities, among other things.

By adding this layer to the study, the scholars found that a 10 percentage point increase in the belief about salaries at other firms leads to a 2.6 percentage point increase in a worker intending to leave their present firm.

"This updating of beliefs causes workers to adjust their job search and wage negotiation intentions," Roussille observes.

While the exact circumstances in every job market may vary somewhat, the researchers think the basic research findings from Germany could well apply in many other places.

"We are confident the results are representative of the German labor market," Jäger says. "Of course, the German labor market may differ from, say, the U.S. labor market. Our intuition, though, is that, if anything, misperceptions would be even more consequential in a country like the U.S. where wages are more unequal than in Europe."

Moreover, he adds, the recent dynamics of the U.S. job market during the COVID-19 pandemic, when many workers searched for new work and ended up in higher-paying jobs, is "consistent with the idea that workers had been stuck in low-paying jobs for a long time without realizing that there may have been better opportunities elsewhere."

Data informing theory

The findings of Jäger, Roth, Roussille, Schoefer stand in contrast to established economic theory in this area, which has often worked from the expectation that employees have an accurate perception of industry wages and make decisions on that basis.

Roussille says the feedback the scholars have received from economics colleagues has been favorable, since other economists perceive "an opportunity to better tailor our models to reality," as she puts it. "This follows a broader trend in economics in the past 20 to 30 years: The combination of better data collection and access with greater computing power has allowed the field to challenge longstanding but untested assumptions, learn from new empirical evidence, and build more realistic models."

The findings have also encouraged the scholars to explore the topic further, especially by examining what the state of industry-wide wage knowledge is among employers.

"One natural follow-up to this project would be to better understand the firm side," Jäger says. "Are firms aware of these misperceptions? Do they also hold inaccurate beliefs about the wages at their competitors?"

To this end, the researchers have already conducted a survey of managers on this topic, and intend to pursue further related work.

Provided by Massachusetts Institute of Technology

This story is republished courtesy of MIT News ( web.mit.edu/newsoffice/ ), a popular site that covers news about MIT research, innovation and teaching.

Explore further

Feedback to editors

DNA origami-based vaccine platform enhances anti-tumor responses through nanometer-precise spacing of molecules

10 minutes ago

Oil resources should stay underground to meet the commitments of the Paris Agreement, study finds

16 minutes ago

Study finds lands used for grazing can worsen or help climate change

19 minutes ago

Study shows that antibiotics targeting the same enzyme elicit varied responses

35 minutes ago

Scientists find hundreds of unique species in Africa's newest and most threatened ecoregion

42 minutes ago

Global climate databases work with incorrect data for the tropics, study shows

48 minutes ago

New framework embraces uncertainty to make sense of history

50 minutes ago

Boosted exciton mobility approaching the Mott-Ioffe-Regel limit in a 2D Ruddlesden-Popper perovskite

52 minutes ago

GALILEO: Scientists propose a new method to search for light dark matter

53 minutes ago

Research team identifies three bacteria species in the human gut that can break down cellulose

55 minutes ago

Relevant PhysicsForums posts

Interesting anecdotes in the history of physics.

3 hours ago

Biographies, history, personal accounts

4 hours ago

Music to Lift Your Soul: 4 Genres & Honorable Mention

5 hours ago

Cover songs versus the original track, which ones are better?

6 hours ago

Two-tone, Ska rock

What was the true purpose of the german expedition to tibet in 1938.

23 hours ago

More from Art, Music, History, and Linguistics

Related Stories

Increasing minimum wage has positive effects on employment, says study

Oct 13, 2023

Workers in key government public health occupations earn salaries well below the private sector, reveals study

Feb 12, 2024

For some workers, low-paying jobs might be more of a dead end than a stepping stone

Apr 26, 2023

The costs of racial discrimination in the labor market

Dec 7, 2022

Women and younger workers more likely to be underemployed, UK report finds

Jan 4, 2024

What's it worth to work from home? For some, it's as much as one-third of their wage

Jan 16, 2024

Recommended for you

Energy price hikes hit older people hardest, study finds

Supply chain disruptions will further exacerbate economic losses from climate change, finds study

Mar 13, 2024

Research finds a college degree remains a sound investment despite rising tuition

Mar 12, 2024

Keep the change: Scientists analyze the attitudes of shop assistants

Mar 8, 2024

AI model trained with images can recognize visual indicators of gentrification

Mar 5, 2024

Researchers show Reddit users caused the famous GameStop 'short squeeze'

Feb 27, 2024

Let us know if there is a problem with our content

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Strategies to enhance the level of service and safety of rural roads: A case study

Roles Conceptualization, Data curation, Investigation, Writing – original draft

Affiliation School of Public Utility, Jiangsu Urban and Rural Construction College, Changzhou, Jiangsu Province, China

Roles Funding acquisition, Methodology, Visualization, Writing – review & editing

* E-mail: [email protected]

Affiliation Department of Transportation Engineering, College of Architectural Science and Engineering, Yangzhou University, Yangzhou, Jiangsu Province, China

Roles Formal analysis, Methodology, Validation, Writing – review & editing

Affiliations College of Transportation Engineering, Tongji University, Shanghai, China, Key Laboratory of Road and Traffic Engineering of the State Ministry of Education, Shanghai, China

• Qiannan Ai, 
• Jun Zhang, 

• Published: March 14, 2024
• https://doi.org/10.1371/journal.pone.0300525
• Peer Review
• Reader Comments

Faced with the contradiction between the increasing traffic volume and the aging road infrastructures in the rural area, this paper aims to propose feasible strategies to enhance the level of service and safety, by a case study of the rural area in the north Jintan district. In order to figure out current issues related to rural roads, a carefully designed investigation has been conducted, and the results of the two-week investigation include roads’ basic information, traffic signs and protective facilities, surrounding landscape, and etc. Based on the field driving tests, specific problems including signs category, signs installation and facility maintenance have been fully analyzed. Meanwhile, the problem of roadnet connectivity has also been pointed out through the theory of complex network, and results show that the average node clustering coefficient and shortest path length perform worse than the demonstration plot of other rural districts. For the sake of rural traffic safety and management efficiency, both quantified and qualified strategies have been put forward. The quantified strategies include the regular inspection indicators, the safety sight distance at T-type crossings, as well as the risk severity of sections and the crossings. The qualified strategies involve the management of trucks and roadworks, the setting of signalized intersections, and the timely updates of traffic signs and facilities. Finally, an intelligent management system framework has been established for rural road traffic, with highly interconnected modules of data acquisition, risk identification and information publishing.

Citation: Ai Q, Zhang J, Ye Y (2024) Strategies to enhance the level of service and safety of rural roads: A case study. PLoS ONE 19(3): e0300525. https://doi.org/10.1371/journal.pone.0300525

Editor: Ibrahim Badi, Libyan Academy, LIBYA

Received: August 12, 2023; Accepted: February 28, 2024; Published: March 14, 2024

Copyright: © 2024 Ai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The major data supporting the findings has already shown in the manuscript, and the other data including the GIS files of regional road network and the traffic characteristics data are available on request from the Chanzhou City Planning and Design Institute, email: [email protected] , the data are not publicly available due to privacy restrictions.

Funding: Local Innovation Talent Project of Yangzhou under Grant number 2022YXBS118. Funder: Yangzhou Government, Jiangsu Province, China. Recipient: Jun Zhang The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The increasing rural economy under the integration of urban and rural development has brought the increasing traffic demand on rural roads, which has become a challenge to the transportation safety and rural governance. In most rural areas of China, many factors contribute to the urgency of enhancing rural road traffic, including the rural roads construction oversight, the limitation of infrastructure maintenance, and increasing traffic conflicts or crashes between local residents and passing-by vehicles. Under the national strategy of all-around rural revitalization, the improvement of rural transportation functions as a key support, wherein establishing a lifecycle mechanism of coordinated construction, management, maintenance and operation has been put on the agenda of regional governors. The first step is to screen current rural road system for potential risks and deficiencies, and put forward possible enhancement strategies for daily managing and future planning. Generally, current literatures on the rural roads enhancement can be divided into three categories. The first is the basic infrastructure improvement, the second is the traffic safety facilities configuration, and the third is the advanced management strategies.

Studies on the improvement of basic infrastructures mainly include the road alignment and the road pavement. Road alignment improvement focuses on the interactions between road geometric parameters and driving behaviors. Based on the data of geometric alignment and surface performance of a two-lane rural road, Bella explored the corresponding influence on driving safety via theories of decision tree and Bayesian network [ 1 ]. Machiani et al. further studied drivers’ perception of curves upon an analysis of speed and lateral position on the rural roads [ 2 ]. It has also been revealed that lane width, curve length and curve radius are key factors affecting the traffic order on the rural road, using the safety performance functions [ 3 ]. By comparison, the road pavement improvement considers more on a sustainable design and maintenance. To realize the environment-friendly maintenance, Kumar and Ryntathiang studied the performance of the pavement condition index with the age of pavement, and introduced the technology of micro-surfacing [ 4 ]. Pasindu et al. performed an optimization analysis for the pavement management of the rural roadnet, upon the calculation of cumulative safety and international roughness [ 5 ]. Current literatures on the enhancement of basic infrastructure for rural roads are safety-based geometric design and pavement management, with common considerations on traffic volume, traffic composition and design speed [ 6 , 7 ], and the results can provide important references for enhancement strategies.

As to the traffic safety facilities configuration, the signs, markings, signals, and protective infrastructures are usually discussed. Griselda et al. pointed out that the completeness and correctness of markings or signs plays an important role in reducing crashes on rural highway [ 8 ]. Zhang et al. studied the visual recognizability of different traffic signs, and analyzed the impact of sight distance, lane location, occluded ratio, shape damage, and installation parameters (height, angle, size, and etc.), which can provide a solid base for regulations modification [ 9 ]. Reinolsmann et al. tested the warning distance of variable message signs on rural roads with low visibility, and designed a visualized sandstorm animation to warn drivers [ 10 ]. The method of setting traffic safety facilities for rural highway in different road sections (small curve, steep slope, rural tunnel, crossings and etc.) have also been proposed, including traffic signs, traffic markings and anti-collision facilities [ 11 ]. It can be concluded that the road alignment and traffic environment are a key basis for the configuration of safety facilities on rural roads. The configuration schemes can be improved to the level above standards by analyzing the interaction mechanisms among facility installation parameters, drivers’ perception attributes and driving behaviors.

In the aspect of advanced management strategies, a variety of updated technologies have been proposed or preliminarily applied. The safety evaluation and enhancement for road infrastructures are critical parts during daily management [ 12 , 13 ]. Vaiana et al. carried out a safety performance evaluation through the RSI approach for a high-risk rural road [ 14 ]. Faced with the deterioration of rural road infrastructure, Subedi et al. developed the priority list of road maintenance considering economic factors and technological factors [ 15 ]. Similarly, Gupta et al. further assessed the maintenance status by structural and functional parameters including pavement roughness, cracking, texture depth [ 16 ]. Meanwhile, the intelligent traffic management in the rural area has gained more attention recently. Targeting at providing effective information for travelers, Rasaizadi et al. proposed an ELP (ensemble learning process) based prediction method of traffic distribution on rural roads [ 17 ]. Considering the difference between driving perception and action, Tian et al. conduct an investigation on the Rural Intersection Conflict Warning System, and provided suggestions for delicate management [ 18 ]. David et al. proposed a model of measuring real-time carbon emissions for rural road, for the convenience of monitoring roadside air pollutions [ 19 ]. Generally, existing strategies have covered the lifecycle management of rural roads including planning, construction, operation and maintenance, but the strategies are independent from each other, the integration of different monitoring and measuring data sources appears to be the development trend for the smart management and decision.

From the foregoing literatures, it can be concluded that the enhancement of rural roads is a systematic work, while most existing literatures focus on presenting a specific method or strategy to improve the level of service or safety, it is undeniable that the proposed strategies are theoretically feasible and useful, but the methods of different studies are usually mutual separated to each other, causing the problem of global evaluation and enhancement when it comes a rural network. Most theories are not appropriate for use in rural areas due to the absence of financial support and professional engineers for rural roads. The current gap is in evaluating the status of rural roads effectively and adopting appropriate enhancement strategies correctly, taking into account their feasibility. This paper aims to illustrate the implementation method of field investigating and countermeasures analyzing, through a case study of rural roads in Jintan District, Jiangsu Province, China. Our paper is noteworthy for its presentation of a well-organized enhancement mechanism for rural roads, which consists of both practical suggestions and theoretical assessments.

The remainder of this paper is structured as follows. The Current situation investigation section describes the current situation of rural roads in the studied area, mainly introducing the road information and infrastructure characteristics. The section of Road infrastructure condition analysis performs a condition analysis of road infrastructures including traffic signs, traffic control, facility maintenance and network topology. An enhancement mechanism is presented in the Enhancement mechanism section, followed by an architecture framework of intelligent management system for rural roads. Finally, the Conclusions section ends the paper with major contributions and possible future work.

Current situation investigation

Data investigation.

Considering the peculiarity of rural roads, a general framework of field investigation is designed in Fig 1 , composed of three aspects, the basic road information, the traffic safety facility, and the roadside landscape. Among the three, the traffic safety facility has strong relationships with the other two, wherein the traffic signs and protective facilities are dependent on the road section attributes and the surrounding circumstances.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0300525.g001

Taking into account the demands of future analysis and studies, the data collection methods employed in field observations and measurement are:

• Photographing method. Taking photos of roadside environment, road pavement and traffic-related facilities. The photos are used as basic references to illustrate current condition of rural roads.
• Aerial video recording. Recording the traffic status of rural road sections and signal-control intersections with UAVs (unmanned aerial vehicle). The aerial videos are the basis of road traffic analysis, including traffic volume, traffic density and conflicts distribution.
• Positioning method. Using GIS map tools to mark the locations of inspected infrastructures, facilities, and signs in synchronization with the photographing method.
• Trajectory recording method. Recording the trajectory data during the repeated paths driving test, which is utilized for the speed distribution analysis and travel time estimation. Dynamic path management for trucks and traffic guidance for reconstruction sections can be aided by the trajectory data.
• Physical Measurement. Using tape measures, it is possible to measure the physical parameters of road infrastructures and signs, including lane width, sign size and location, guardrail length, and other parameters. Safety checks and evaluations of traffic facilities depend heavily on measurement data.
• History data investigation. Obtaining the data like area population distribution, road maintenance records, road accidents record from the published data of local government and transportation sectors. Indicators calculation and prediction can be utilized with these types of data, particularly for decision making in traffic control and infrastructure maintenance.

Roads information

The area of study is the northern rural zone of Jintan district, situated in the southwest of Changzhou, China. The field investigation was conducted from 1 st to 22 nd , Jun 2023, approved by the Science and Technology Bureau of Changzhou, where the major roads and key connecting branches are involved, excluding the ways accessing villages, as shown in Fig 2 . The major information of major investigated rural roads is indicated in Table 1 . All roads in the study area share a same climate condition, the average daily rainfall is about 2.91 mm, where days with rainfall over 10 mm only accounts for 2.85% of the whole year. Meanwhile, the average wind speed is about 3.3 m/s, and the average daily sunshine percentage is about 46%.

(Republished from TransCAD under a CC BY license, with permission from Caliper Corporation, original copyright 2024).

https://doi.org/10.1371/journal.pone.0300525.g002

https://doi.org/10.1371/journal.pone.0300525.t001

In Table 1 , the types of cross-section types are classified into four kinds according to the actual situation of the studied area, as indicated in Table 2 .

https://doi.org/10.1371/journal.pone.0300525.t002

The following three points summarize the general condition of rural roads within the study area based on the investigated data.

• 29% of the studied road sections are dual carriageways designed with bi-directional 4 motor lanes, where the central isolation forms are different. E.g., the Jinsha road adopts the central green belt to isolate contrary carriageways, while the North Huanyuan road uses the guardrail, as shown in Fig 3 .
• 84% of the investigated road sections are the asphalt pavement, which can provide a better driving comfortability. As shown in Fig 4 , the rest of the road sections are paved with cement, which presents serious problems with cracks and depressions.
• 5% of the road sections are triple carriageways, distributed along the Jinzhuang Road and the X203 Road, especially the sections passing through the town area, as indicated in Fig 5 . Due to the large demand of non-motorized traffic, barriers have been installed to isolate the motor lanes from the bilateral non-motorized lanes, in order to guarantee the safety of traffic participants.

https://doi.org/10.1371/journal.pone.0300525.g003

https://doi.org/10.1371/journal.pone.0300525.g004

https://doi.org/10.1371/journal.pone.0300525.g005

Traffic signs and safety facilities

Three representative roads, Baita Road, X302, and Jianshe Road, were chosen to reflect the traffic facility distribution on rural roads.

Baita Road.

The investigated road section spans from the BT-JC intersection to the BT-NJH intersection, with a total length of 4.4 km. Two motor lanes and two non-motorized lanes are located on a single carriageway in the investigated section. The cross section parameters together with traffic signs and facilities are indicated in Fig 6 , where the major facilities are warning signs, prohibition signs and protection infrastructures.

• Warning signs The current road section has major warning signs such as pedestrian crossings, hazardous locations, and crossings. Children’s caution signs are placed above the motor lanes in the area near schools.
• Prohibition signs Speed limit signs are distributed in the places along the town area and the factory area, and the weight limit signs are further installed before bridges to limit vehicles axle weight. No-parking signs are scattered to ensure the right-of-way for non-motorized traffic.
• Protection infrastructures Even though there are barriers between motor lanes and non-motor lanes, guardrails are installed along parts of slopes or rivers.

https://doi.org/10.1371/journal.pone.0300525.g006

The investigated road section spans from the X302-X203 intersection to the X302-JSH intersection, with a total length of 12.5 km. The investigated section is a 10 meters wide single carriageway without bilateral non-motorized lanes. The basic condition of cross section and related facilities are shown in Fig 7 .

• Warning signs. Because there exists large number of villages near current road, the warning signs of non-signalized T type crossing and village caution are distributed along X302. Meanwhile, in the section with poor visibility and alignment, sings like sharp turn and side road are also set accordingly.
• Prohibition signs. Most speed limits of X302 are 60 km/h, while in the section passing town area, the speed limits will decrease to 30km/h. Due to the branch ways connecting villages and current road, yield signs have been set on the corner of the branch way. No-parking signs are also laid along the busy town sections.
• Protection infrastructures. Fluorescent arrow signs and guardrails are not the only safety measures on dangerous roads. Vibration markings and red-white posts are also often distributed. The vibration markings are placed before the bends as a reminder of slowing down, and the red-white reflective posts are used to identify the crossings of branch ways.

https://doi.org/10.1371/journal.pone.0300525.g007

Jianshe Road.

The investigated road section spans from the JS-X302 intersection to the southern provincial road, with a total length of 9 km. The corresponding cross section is similar to the Baita Road. Major signs and facilities are indicated in Fig 8 .

• Warning signs. Compared with the foregoing sections, current road section is another equipped with signs of narrow roads.
• Prohibition signs. The major prohibition signs on current road include the signs of speed limits, weight limits and the no overtaking, where the no overtaking signs are placed along the curve sections.
• Protection infrastructures. The current road has a variety of traffic protection infrastructures. Besides the abovementioned side guardrails and surface vibration markings, the yellow flashing lights are deployed to remind drivers slowing down, and the warning lights and message boards are specially set before the adjacent branches to identify the non-signalized crossings.

https://doi.org/10.1371/journal.pone.0300525.g008

Road infrastructure condition analysis

According to the investigated data of rural road sections, the problems and enhancement strategies have been discussed from the perspectives of facility configuration, traffic management and maintenance, and network topology.

Adopted standards and guidelines.

Two standards are the primary basis for inspecting traffic signs. One is the Road Traffic Signs and Markings (GB5768.2–2022), the national standard applied in China. And the other is the Road Traffic Sign Panels (JT/T 279–2004), issued by the Ministry of Transport. Generally, the adopted standards are summarized from the following three perspectives according to our study focus. It should be pointed out that the adopted standards and guidelines for facility inspection are restricted to China, as rural road transportation conditions vary in different countries and regions.

Quantified standards for sign size and appearance.

• The size of sign panels should conform to the related standards in GB5768.2, where the acceptable size deviation is ± 5mm for normal panels, and ± 0.5% for panels longer than 1.2m.
• Sign panels should have a flat surface with less than 3mm/m unevenness.
• The following defects are not allowed on the traffic sign panels: cracks, wrinkles and peeling-off edges; obvious scratches, damages and the non-uniform color painting; blisters or bubbles larger than 10 mm2 on any surface of a 50cm×50cm size; the uneven retroreflective performance.
• The size of sign panels depends on the roads design speed. Table 3 shows the corresponding sizes of different prohibitions and size shapes for rural roads with a design speed between 40 km/h and 70 km/h.

https://doi.org/10.1371/journal.pone.0300525.t003

Requirements for sign angles.

• The installation angle should avoid the glare to drivers from the sign panel.
• It is important that the roadside traffic signs are either vertically aligned with the road centerline or at a specific angle from that direction. Specifically, the angle of prohibition signs and directional signs should be 0~10° or 30~45°, the angle of warning signs and guiding signs should be 0~10°. As indicated in the Fig 9 .

https://doi.org/10.1371/journal.pone.0300525.g009

Regulations for sign locations.

Due to the large number of T-type crossings or non-signalized crossings, special focuses are paid on the yielding signs and branch way posts.

The stop yielding signs should be installed on the mouth of branch way under the following conditions: the left-turn motor traffic needs control, or the conflict between motor traffic and non-motor traffic needs control; the crossing sight distance is limited or the intersection is a hazardous location; the signal intersection without all day 24-hour control.

As compared to the stop yielding signs, the speed reduction yielding signs can be installed at the mouth of branch way with a better sight distance or a lower accident risk, or at the entrances of non-signal roundabouts.

The branch way posts should be installed on the two sides of branch mouth, in order to warn the drivers along the main road against the potential conflict with vehicles from the branch way. The location and feature of branch way posts are illustrated in Fig 10 .

https://doi.org/10.1371/journal.pone.0300525.g010

Traffic facility configuration

Despite the fact that the investigated rural roads have equipped with a variety of warning signs and prohibition signs, there do exist obvious defects in the aspects of sign type, install location and timely update.

Traffic signs types.

The following traffic signs are missed along the road sections considering the surrounding landscapes and traffic characteristics, as shown in Fig 11 .

• Speed limit release signs. The top speed of rural roads is 60 km/h, and the speed will be limited to 30 km/h or 20km/h in the accident hazardous sections and the town area sections under the roadside warning signs, but the speed limit release signs are seldom set in the investigated road net. It is necessary to tell drivers the boundary of speed limit section, so that the drivers can decide when to accelerate to save the trip time.
• No tooting signs. In the road sections passing through the street community and town market, no tooting signs should be placed appropriately in order to facilitate the construction of a peaceful and harmonious rural community [ 20 ].
• Continuous curve signs. For those adjacent road sections with sharp or big curves, the single sharp turn signs should be updated to the continuous curve signs, in order to inform the drivers of poor alignments ahead.
• Livestock caution signs. During the investigation, it is found that some sheep or geese groups will cross the rural road, while the traffic speed is relatively higher. Therefore, it is indispensable to set the livestock caution signs in the related section, together with the signs of speed limit at 30 km/h.
• Other warning signs. Other warning signs including the Y-type crossings, staggered crossings and hump bridges should also be installed in the appropriate road section.

https://doi.org/10.1371/journal.pone.0300525.g011

Install location.

The problems of traffic related signs and facilities location are:

• Guardrail missing. The guardrails mainly refer to the roadside guardrails. Some road sections along the river have not installed the guardrail due to the existence of trees or barriers.
• Occluded signs. Along the rural roads, a great many of the signs are occluded by the trees, buildings and telegraph poles, and some sign columns are set to a wrong angle, which will affect the perception and action of drivers. Therefore, it is suggested to clear the surrounding area of current occluded signs, or set the signs in a more apparent location without occlusion.
• Overlapped signs. In some special area like bridges and streets, the phenomenon that several kinds of warning signs and prohibition signs are installed together, namely the information are displayed too intensive, which is adverse for drivers to perceive information effectively.

Facility updates and maintenance.

In the rural area, traffic safety related facilities lack timely updates and maintenance, mainly reflected in the following aspects.

• Different degrees of damage and fade of traffic signs. The fade of yellow warning signs will affect the night visibility, and the common damage status include the tilted column, the folded or bent sign and the worn surface upon our investigation. According to the observation data, the reflective performance of faded signs has decreased by 15% to 40%.
• Unclear road surface markings. Under the influence of road cracks, some motor lane lines and center lines are worn away to varying degrees, making it harder to distinguish the solid lines and the dotted lines. The aggregate length of road sections with unclear markings is about 6.1 km.
• Broken branch way posts. Some red-white posts used to identify branch ways are broken, and some posts lost the reflective coatings. According to the regulations for branch way posts, nearly 1/3 non-signal control T-type crossings are found disqualified both in the number of posts and in the height of posts.
• Bended guardrails, Along the section with adjacent rivers or lakes, some roadside guardrails have been hit by vehicles and become bended for a long while. It is suggested to inspect the status of guardrails and conduct timely maintenance.

Traffic management and maintenance

The following three kinds of traffic control risks are found during our investigation.

• Trucks traffic management The rural area being studied has several industry parks and factories that produce energy, chemical, and building materials, which leads to a significant demand for truck traffic. On one hand, the phenomenon of trucks illegal parking has occupied some road sections, making it difficult for vehicles to meet. On the other hand, the heavy trucks have caused severe problems of road surface cracks [ 21 ] and fugitive dusts, especially on the cement paved roads. Hence, it is significant to enhance the management of trucks by setting parking rules and planning driving routes.
• Non-signalized crossings control The non-signalized crossings along the rural roads mainly include the T type crossings and the Y type crossings under the priority control rules, by setting the yield signs on branch ways. Due to the presence of lush shrubs and trees along the roads, the sight triangle was unable to satisfy the driving visibility demand, resulting in merging conflicts at the intersection.
• Roadworks management During the investigation period, some road sections are being rebuilt or maintained, and the necessary management measures need to be enhanced. In the micro section traffic safety organization, the current measure is just place speed limit signs and warning signs before the work area, without signs of narrow road or lane blockage, nor the protection facilities like anti-collision buckets and cones. At the global network level, there lacks bulletin boards displaying roadwork information and path guidance suggestion on the adjacent road, which may increase the detour distance of some vehicles.
• Road surface maintenance Due to the repeated demand of heavy trucks, roads around the industry parks and factories have suffered from surface cracks, as indicated in Fig 4 . The cracks will affect the driving stability and comfortability, especially threatening the safety of non-motorized vehicles on the single carriageway section.

Rural roadnet connectivity

The relation between road accessibility and rural economics has been verified in current studies [ 22 ]. In order to analyze the roadnet connectivity of studied area, the theory of complex network is applied here to calculate the corresponding indicators, mainly including the degree centrality, the clustering coefficient and the average shortest path length [ 23 ]. The results of complex network analysis will support the evaluation of the level of service and safety. The undirected network is output upon the adjacency matrix of the actual roadnet, as indicated in Fig 12 .

https://doi.org/10.1371/journal.pone.0300525.g012

Degree centrality.

To describe the connectivity between the current node and it’s adjacent nodes, the indicator of degree centrality is used. The degree of node i is defined by the number of its surrounding edges ki . E.g., the degree of an intersection point is 4, the degree of a T-type crossing is 3, and the degree of a dead-end road section is 1.

The node degree distribution is shown in Fig 13(A) . The node degree in the current network ranges from 1 to 4, with an average node degree of 2.7, which approaches the degree of T-type or Y-type crossings, validating that there exist many T-type crossings in the area.

https://doi.org/10.1371/journal.pone.0300525.g013

Clustering coefficient.

Via self-coded batch processing algorithm, the clustering coefficient of 72 nodes are indicated in Fig 13(B) . It is obvious that most of the surrounding nodes are not connected to each other. Only 9 nodes’ clustering coefficient are above zero, and the average clustering coefficient is about 0.028. The connection levels of the overall network nodes are inferior to those of the nodes in the urban roadnet.

Average length of shortest paths.

https://doi.org/10.1371/journal.pone.0300525.g014

In order to analyze the connectivity level, we have performed a comparison analysis between the studied area and the rural areas with high-quality roadnet construction, as indicated in Table 4 . The average node degree of the studied area is slightly higher than the Feixi district, while lower than the Huzhou and Enshi district, because the latter two rural districts own more signalized intersections. Meanwhile, the studied area underperforms in the other two indicators, indicating that the local accessibility and connectivity need improvement.

https://doi.org/10.1371/journal.pone.0300525.t004

Enhancement mechanism for rural roads infrastructure

Enhancement rules..

Targeting at the high-quality development of rural roads, the following four rules have been put forward in order to achieve the status where drivers enjoy their trips and goods enjoy their circulations.

• Classified management. The management scheme should consider the differences among rural roads with different conditions and realize a problem-oriented enhancement.
• Symbiotic development. Since the rural roads major serve the demands of passenger and freight traffic in the rural districts, the enhancement or reconstruction should merge into the development of ecological country or tourist town, providing coordinated infrastructures and facilities for local development.
• Multi-pronged measures. In order to enhance globally, it is necessary to incorporate road traffic safety, network connectivity, and service sustainability measures, while also focusing on the integrated lifecycle development of planning, construction, management, and maintenance.
• Clear liabilities. It is of urgent need to improve the liability system of rural road management, where the duties of local government, transport management department, law-enforcing department and agricultural sector should be clarified hierarchically.

Enhancement strategies

Quantified traffic safety inspection..

The result of field observation and measurement shows that current configurations of traffic facilities on rural roads are not optimistic, lagging behind the national and industrial standards. In order to scientifically assess the level of road service and safety, an assessment framework is established considering the indicators from perspective of road alignment infrastructure, roadside landscape, traffic condition, traffic signs and protection facilities, as indicated in Tables 5 and 6 , where the road alignment, roadside landscape and traffic condition belong to the auxiliary indicators, and the others are categorized as the fundamental indicators.

https://doi.org/10.1371/journal.pone.0300525.t005

https://doi.org/10.1371/journal.pone.0300525.t006

Traffic control enhancement.

According to the investigation results, rural road traffic is more susceptible to uncertainty and illegal activity than urban road traffic. E.g., over-speeding behaviors occur frequently on low-volume sections according to aerial video analysis, which makes crossings dangerous. The traffic control of rural roads can be enhanced from the following perspectives.

• (1) Signalized crossings setting. With the development of rural economy, the traffic volume will arise accordingly. Based on the particular investigation on network traffic volumes distribution, some non-signalized intersections with higher volumes should be updated as signal control crossings to reduce the merging conflicts, where the decision of timing scheme should consider the dynamicity of volumes temporal distribution.
• (2) Visibility enhancement for non-signalized crossings. As shown in Fig 15 , the sight triangle should both consider the sight point, sight boundary and sight angle, where the sight boundary is the axis of the outer lane, and the sight angle is 60°. The sight distance S 1 and S 2 are calculated by Eq ( 3 ) and Eq ( 4 ) respectively.

https://doi.org/10.1371/journal.pone.0300525.g015

Taking the crossing of a two-carriageways road and a single-carriageway branch, the sight distance S 1 should be at least 11.75 meters, S 2 should be 20.4 m. Meanwhile, the shrubs or structures inside the clear area should be controlled under 1.2 m, according to the Specification for design of intersections (CJJ152-2010) published by the Ministry of Housing and Urban-Rural Development of China.

• (3) Truck management. Based on the location of industries and factories, it is necessary to plan the regular truck driving paths on the rural network. Guardrails or barriers are required on the road sections of trucks driving paths, and asphalt pavement is required. Meanwhile, the parking area and no-tooting sections should also be planned in advance.
• (4) Roadwork management. To minimize conflicts in the construction area, traffic flow organizations near the roadwork area should pay attention to the occupancy status of the lanes [ 25 ]. Taking the roadwork occupying half range of a 4-lane road as an example, the local distribution of traffic signs and facilities should be roadwork warning sign, speed limit sign, narrow road sign, anti-collision facilities and speed release signs in sequence. If the road section is full occupied, besides the warning signs of road blockage and no passing, regional paths guiding boards are suggested to install around the area to realize the microsimulation of network flows, as indicated in Fig 16(A) and 16(b).

https://doi.org/10.1371/journal.pone.0300525.g016

Intelligent management system for rural roads.

In order to effectively enhance the performance of rural road network and conduct real-time control strategies, a framework of intelligent management system is presented in this section, as indicated in Fig 17 .

(The two maps are republished from TransCAD under a CC BY license, with permission from Caliper Corporation, original copyright 2024).

https://doi.org/10.1371/journal.pone.0300525.g017

The system is composed of the following basic modules.

• (1) Data acquisition module. The data acquisition module is an integrated database of video-based data, the sensor-based data and the historical data. The video-based data include the traffic volume, vehicle velocity and traffic density, which can be extracted from the video upon mature algorithms and theories of traffic-related image recognition [ 26 ]. The video data can be provided by the fixed cameras and the unmanned aerial vehicles. The sensor-based data include the traffic noise, vehicles emission and truckloads. The historical data include traffic accidents, maintenance record, pavement roughness, GIS data and etc., where the pavement roughness data are collected by periodic laser scanning.
• (2) Risk identification module. Based on the data of historical traffic accident, flow characteristics and road infrastructures, current module is designed to identify the road sections risk distribution, and find out locations with poor safety status. The use of hierarchical analysis or fuzzy clustering method can help identify the safety status, allowing for better daily management and regular inspection. As indicated in Fig 17 , the global risk distribution of rural road network is reflected by a heat map, where a deeper color means a higher severity of road sections, with a higher possibility of traffic conflicts or a worse condition of road infrastructures. The risk severity of the section and the crossing are quantified by Eq ( 5 ) and Eq ( 6 ) respectively. Results show that over 35% of road sections and crossings in the town area suffer from higher risk perturbations.

In Eq ( 5 ), R se refers to the section risk severity; P se denotes the basic accident probability on current road section; V se is the daily traffic volume from 8:00 am to 9:00 pm, PCU/day; Cap se is the corresponding lane capacity, PCU/day, affected by the lane width, cross-section type and speed limit; V tr is the volume of trucks, veh/day, and PCE tr is the passenger car equivalent of trucks used to convert V tr to standard volume, usually takes the value of 4; L da is the length of damaged road surface including cracks and depressions, km; L se is the length of current road section, km.

• (3) Information publishing module. The purpose of this module is to quantitatively analyze and visualize important data, including real-time network accessibility, traffic flow, emission, noise distribution, illegal traffic behavior, and other relevant information. Meanwhile, the published information should be shared among transportation departments, village committees and local governors, for the convenience of coordinated administration.

Conclusions

This main contribution of this paper lies in improving the level of service and safety for rural roads through the following two aspects. The first is the evaluation methodology for rural roads infrastructures, composed of field investigation, traffic facility configuration, management and maintenance, and network connectivity. The field study reveals that the common problems of rural roads are safety infrastructure maintenance and traffic operation management, arriving from the outdated construction standards and the long-term absence of effective supervision, e.g. the number of effective warning signs only accounts for 69% according to the standards. The second is the enhancement mechanism, including the evaluation tables composed of quantifiable indicators, the specific strategies from local traffic enhancement to global improvement, and the necessary modules for the establishment of intelligent management system. The proposed framework of rural roads management has integrated technologies of multi-source data analysis, risk visualization and assistant decision. Although every strategy is feasible and reliable under current theory and technology, some strategy can only be implemented by the combined efforts of relevant engineers, administrators and managers.

Meanwhile, during our analysis and evaluation, it is found that there exists a strong relation between the traffic characteristics and the level of service and safety, e.g., the higher proportion of trucks and lorries usually corresponds to a worse pavement condition, a worse signs and markings visibility and a higher traffic conflict rate. Therefore, it is of great urgency to establish the intelligent management system for rural roads, with embedded technologies of multi-source data analysis, risk visualization and assistant decision.

Current research is our first step into the field of rural roads management. Our future research will be focused on the emergency management of rural roads under adverse weather conditions [ 27 , 28 ], such as the heavy rainfall and the snowstorm, which will cause the disruption or blockage of partial road sections. The specific studies include the road network resilience evaluation and the paths reorganization algorithm, in order to realize a safe and efficient traffic round-the-clock.

Acknowledgments

The authors deeply thank professorate senior engineer Wei Shen in Chanzhou City Planning and Design Institute for his assistance in field data acquisition.

• View Article
• Google Scholar
• 2. Machiani SG, Medina A, Gibbons R, Williams B. Driver behavior modeling on horizontal curves for two-lane rural roads Using Naturalistic Driving Data. Transportation Research Board 95th Annual Meeting, Washington DC, United States, 2016.
• 4. Kumar R, Ryntathiang TL. Rural road preventive maintenance with microsurfacing. IJCA Proceedings on International Conference on Emerging Frontiers in Technology for Rural Area (EFITRA-2012). Available from: https://www.ijcaonline.org/proceedings/efitra/number4/5954-1026
• PubMed/NCBI
• 13. Zhang M, Gao S. Evaluation of Rural Road Asphalt Pavement Performance Based on Matter Element Analysis. 2nd International Conference on Civil Engineering and Rock Engineering (ICCERE), 2017.
• 18. Tian D, Morris N, Libby D. Rural intersection conflict warning system evaluation and design investigation. Center for Transportation Studies, University of Minnesota, 2018. Available from: https://hdl.handle.net/11299/198534
• 26. Algiriyage N, Prasanna R, Doyle EEH, Stock K, Johnston DM, Punchihewa M, et al. Towards real-time traffic flow estimation using YOLO and SORT from surveillance video footage. ISCRAM 2021 Conference Proceedings– 18th International Conference on Information Systems for Crisis Response and Management. Blacksburg, USA, 2021.

Why do women go through menopause? Scientists find fascinating clues in a study of whales.

The existence of menopause in humans has long been a biological conundrum, but scientists are getting a better understanding from a surprising source: whales.

Findings of a new study suggest menopause gives an evolutionary advantage to grandmother whales’ grandchildren. It's a unique insight because very few groups of animals experience menopause.

A paper published Wednesday in the journal Nature looked at a total of 32 whale species, five of which undergo menopause. The findings could offer clues about why humans, the only land-based animals that also goes through menopause, evolved the trait.

“They’ve done a great job of compiling all the evidence,” said Michael Gurven, a professor of anthropology at the University of California, Santa Barbara who studies human evolution and societies. “This paper quite elegantly gets at these very difficult issues.”

Whales might seem very distant from humans, but they have important similarities. Both are mammals, both are long-lived, and both live in family and social groups that help each other.

How long does menopause last? Menopause questions and concerns, answered.

Studying these toothed whale species offers a way to think about human evolution, said Gurven, who was not involved in the study.

In five species of toothed whales – killer whales, beluga whales, narwhals, short-finned pilot whales and false killer whales – the researchers’ findings suggest menopause evolved so grandmothers could help their daughters' offspring, without competing with them for mates.

Only daughters' offspring are aided because in these whales, while the males stay with their family group, they mate with females in other groups. But mothers do tend to give more support to their male offspring than to their female offspring.

Post-reproductive-age females help their family group in many ways. Off the coast of Washington state and British Columbia in Canada, grandmother killer whales catch salmon and "break the fish in half and share that catch with their families. So they're actively feeding their families,” said Darren Croft, a professor of behavioral ecology at the University of Exeter in the United Kingdom and senior author on the paper.

The whale grandmothers also store ecological knowledge about when and where to find food in times of hardship by using the experience they have gained over the lifetime of their environments.

“We see just the same patterns in (human) hunter-gatherer societies,” Croft said. “In times of a drought or in during times of social conflict, the people would turn to the elders of that community. They would have the knowledge.”

The 'grandmother hypothesis'

The researchers’ findings support what’s known as “the grandmother hypothesis .” It states that menopause is evolutionarily useful because while older women are no longer able to have children, they can instead focus their efforts on supporting their children and grandchildren. This means their family lines are more likely to survive, which has the same effect as having more children.

“What we showed is that species with menopause have a much longer time spent to live with their grand offspring, giving them many more opportunities for intergenerational health due to their long life,” said Samuel Ellis, an expert in human social behavior at the University of Exeter and the paper’s first author.

The difference in humans, Gurven said, is that both grandmothers and grandfathers contribute to the well-being of their children and grandchildren.

“In the human story, I think it’s multigenerational cooperation on steroids,” he said.

Though the study doesn’t prove once and for all that the grandmother hypothesis is the reason for menopause in women, it does lay out the evidence, he said. “It’s part of the story, but no one would say it tells the whole story,” Gruven said.

Does menopause lead to a longer life in humans?

There are two proposed pathways for how menopause evolved in humans: the live-long hypothesis and the stop-early hypothesis.

The live-long hypothesis suggests menopause increased total life span, but not how long a woman could have children. That leads to a prediction that species with menopause would live longer but have the same reproductive life span as species without menopause.

In the stop-early hypothesis, the theory is that menopause evolved by shortening the reproductive life span while the total life span remained unchanged. For this to be true, it would be likely that similar species without menopause would have the same life span as those that have menopause, but a shorter reproductive life span.

In looking at species of toothed whales that don’t have menopause and five that do, the researchers' findings make the long-life hypothesis seem most likely.

“This comparative work we’ve been able to do shows that females minimize this competition over reproduction by not also lengthening their reproductive period. Instead, they've evolved a longer lifespan while keeping a shorter reproductive life span,” Croft said.

This appears to be exactly what humans did.

“One of the striking features of this work is the fact that we find this really incredible and rare life-history strategy that we see human societies and in the ocean, but not elsewhere in mammal societies,” he said.

Whale study doesn't reflect men's life spans

The similarities with humans are not across the board, which is good news for men.

No one knows why in humans only females undergo menopause even though both sexes live to be approximately the same ages.

That’s not the case in some of these whales species, where male life spans are typically much shorter than those of females.

“In the killer whale population, for example, females regularly live into their 60s and 70s," Croft said. "The males are all dead by 40.”