1. Introduction

The United Nations Office on Drugs and Crime [1] estimates that approximately 11.7 million people were detained globally in 2019, of which 93% are male. In England and Wales, the current prison population stands at 86,344, of which 96% are male [2] and 55% of all prison sentences are for less than 12 months [3]. Ministry of Justice research suggests that sentencing offenders to short term custody is associated with higher reoffending rates than if they had instead received community orders [4]. In addition to reoffending, men in prison have elevated levels of mental health problems in comparison to men in the general population [5] and there is evidence to suggest that they are less likely to engage in treatment and services than other prison populations [6]. Thus, a rationale for developing an intervention for male short term sentenced prisoners who were also struggling with their mental health, with the aim of facilitating engagement with community services on release.

We developed Engager, which is a complex collaborative care intervention for men serving short-term prison sentences and who have common mental health problems [7, 8, 9, 10]. It is a manualized, person-centered intervention underpinned by a mentalization based approach. Mentalization is the ability to think about thinking. It helps to make sense of our thoughts, beliefs and feelings and to link these to our actions and behaviors. Engager involves the development of a “shared understanding and action plan” of the participant’s thoughts, behaviors, needs and goals to create a transition plan for release into the community, working alongside and liaising with community services and the participant’s own social network. The intervention is delivered in prison between 4- and 16-weeks pre-release and up to 20 weeks’ post-release, by experienced support workers and a supervisor experienced in the delivery of psychological therapies. Engager upon release into the community is delivered via flexible face-to-face and telephone contact, with the aim of supporting and facilitating engagement with community services to meet the goals set out in the shared understanding and action plan.

We conducted a randomized controlled trial of Engager (plus usual care) compared to usual care alone [7]. A total of 280 participants were randomized. Results showed that there was no mean difference between the two groups for change in the primary outcome measure of psychological distress (Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM [11])); 1.1, 95% (Confidence Interval (CI) −1.1 to 3.2, p = 0.325) or any secondary outcomes. Embedded in the trial was a mixed method process evaluation. This was conducted to ensure vital information was collected concerning implementation, mechanisms of impact and context, to enhance the overall understanding of the trial findings. Through this we observed that delivery of Engager as intended was not achieved, with less than half of participants (48%) receiving the minimum dose of the intervention seen as likely to be required to have an impact (two prison sessions and eight community sessions). Despite this, we found evidence that a minority of Engager recipients had sustained some positive change, and that this seemed to be linked to session attendance. Twenty-four men were purposively sampled based on psychological distress (CORE-OM) variations in the dose of sessions pre- and post-release (Mean (M) = 6.9, Standard Deviation (SD) = 3.3; M = 12.4, SD = 11.7) and session focus of the intervention delivery were observed. The majority of the total sessions attended by the 24 participants (n = 165, 63%) were categorized as “practical” in nature (e.g., attempting to source housing, transport to/from appointments). Only 33 sessions (13%) were coded as being solely “therapeutic” (e.g., personal goals, development of confidence and self-belief in attainable goals) in nature and a further 63 (24%) contained elements of both practical and therapeutic support. Five of the 24 participants were observed to have sustained positive changes in response to intervention delivery. These participants received the greatest number of intervention sessions post-release (M = 27, SD = 16.9) and the greatest number of therapeutic focused sessions (M = 7.8, SD = 5.1). The content of the intervention delivery appeared to differentiate those who sustained change from those who did not; participants sustaining long-term engagement and sustained change reached a state best described as “crises but coping”, whereby there were still significant and ongoing challenges but were managing to cope with these and it had not resulted in a deterioration in terms of mental health or offending. We found evidence that there were several components of the intervention key to achieving this sustained engagement. This included, trusting relationships, therapeutic work delivered well and over time; and an in-depth shared understanding of needs, concerns and goals between the practitioner and participants [10]. Based on these findings, it was important that we revisited the quantitative data from the trial to explore which individual factors are associated with attendance, and this is what we present here in this paper.

Within the literature, there are relatively few studies of attendance and engagement in voluntary psychological therapy by those in contact with the justice system. Most focus on offending behavior or treatment requirement interventions, where attendance is mandated as part of sentencing or is required to be considered for parole or early-release. For example, a recent study [12] of treatment engagement in Mental Health Treatment Requirements (MHTR) found that a lack of insight, substance use, offending history and higher CORE-OM (psychological distress) significantly predicted non-engagement, whereas a lack of motivation, mental health diagnosis and housing needs did not show statistical significance. Substance use and previous offending had the strongest predictive potential. Those using illegal substances were almost four times less likely to engage than those who did not report substance use. However, the evidence in this field is somewhat mixed as Macinnes et al. [13] conducted a retrospective study of engagement with forensic mental health services for 264 patients detained in high and medium secure hospitals. They found that there was no relationship between CORE-OM (psychological distress) scores and therapy engagement.

In terms of factors previously identified in the literature, not specific to individuals in contact with the justice system, a frequent finding is that those with poorer mental health engage less. For example, Di Bona et al. [14] undertook an evaluation of two Improving Access to Psychological Therapy (IAPT) services in the North of England. IAPT is a service provided by the National Health Service in England to improve the delivery of, and access to, evidence-based, psychological therapies for depression and anxiety disorders. Of 363 patients they found that lower CORE-OM (psychological distress) and either a very recent onset of common mental health disorder (1 month or less) or a long-term condition (more than 2 years) was predictive of non-attendance. Sweetman et al. [15] conducted a much larger retrospective analysis of referral and attendance data at five IAPT services. There were 97,020 referrals received between 2010 and 2014. Those referred for treatment for phobic anxiety disorder, obsessive compulsive disorder or somatoform disorder were significantly more likely than those with depressive disorder to attend for treatment. Also, those who reported more severe anxiety symptoms using the Generalized Anxiety Disorder-7 scale were significantly more likely to attend for treatment than those with less severe anxiety symptoms. Similar results for depression—but contrasting results for anxiety—had previously been reported [16] where dropping out of psychological therapy was significantly associated with higher levels of depression, as measured by the Patient Health Questionnaire-9 (PHQ-9), higher levels of anxiety, measured by the Generalized Anxiety Disorder Questionnaire-7 (GAD-7), higher levels of clinical risk and higher levels of deprivation.

Another frequent finding is that poorer attendance and engagement is associated with substance use [12, 17, 18] with the main focus being on illegal substance use. However, a recent study of psychological treatment attendance focusing just on alcohol use [19], looked at the electronic health records for 7986 patients accessing psychological treatment for common mental disorders. They found that alcohol consumption was not significantly associated with attendance and that moderate drinkers may have some shared characteristics which favor treatment response.

The analyses reported in this paper revisit the evaluation data collected from the Engager intervention participants. We investigated a set of measures taken at baseline to see if they were predictive of session attendance. These included depression (PHQ-9), anxiety (GAD-7), post-traumatic stress disorder (PTSD), psychological distress (CORE-OM), personality disorder, past trauma, homelessness and problematic use of alcohol and other drugs. We predicted that higher scores on the measures and/or the presence of these issues would generally predict low engagement (i.e., inverse relationship with attendance). However, we anticipated no relationship with alcohol and psychological distress, and we did not have a clear prediction for anxiety because of previous mixed findings [15, 16]. We also made no predictions about homelessness because of a lack of evidence.

2. Materials and methods

The specification for the Engager intervention and trial are already reported in the literature [7, 8] and briefly summarized in the introduction. For this study, we conducted an exploratory analysis of the Engager data, to look for relationships between 11 predictor variables and one outcome variable (i.e., a measure of engagement). The predictor variables were all self-report mental health/psychological measures taken at baseline (prior to randomization), while the outcome variable was number of sessions attended over the course of the entire intervention. The predictor variables comprised:

⋅ PHQ-9 scale [20] (9-question Patient Health Questionnaire). A nine item self-report questionnaire measuring severity of depression. Items are rated as “0” (not at all) to “3” (nearly every day).

⋅ GAD-7 scale [21] (Generalized Anxiety Disorder 7 anxiety). A seven item self-report questionnaire measuring severity of anxiety. Items are rated as 0, 1, 2 and 3, to the response categories of “not at all”, “several days”, “more than half the days” and “nearly every day”, respectively, and adding together the scores for the seven questions.

⋅ PC-PTSD-5 scale [22] (Primary Care PTSD Screen). A five-question screening tool for measuring post-traumatic stress disorder. The tool begins with a question assessing lifetime exposure to traumatic events. If there is no exposure, the PC-PTSD-5 is complete with a score of 0. However, if there is exposure, then there are five additional yes/no questions about how that trauma exposure has affected them over the past month. Scores can range from 0–5.

⋅ CORE-OM scale [11] (Clinical Outcomes in Routine Evaluation-Outcome Measure). A self-report measure of psychological distress. A 34-item scale comprising four domains: subjective wellbeing; depression and anxiety related problems and symptoms; general, social and relationship functioning; and risk of harm to self or others. Items are rated on a five-point Likert scale ranging from “not at all” to “most or all of the time”. The mean across the items, i.e., between 0 and 4 was used for the current analyses.

⋅ SAPAS scale [23] (Standardized Assessment of Personality-Abbreviated Scale). Eight-item screening interview for personality disorder. Each question is scored 0 (No)/1 (Yes), except for question 3 which is inversely scored 1 (No)/0 (Yes). The scores on the eight items are added together to produce a total score ranging between 0 and 8.

⋅ Problem with alcohol. Self-reported Yes/No question for self-identifying as having a problem with alcohol.

⋅ Days used alcohol. Numeric count of number of days’ alcohol used in the month prior to coming into prison.

⋅ Problem with drugs. Self-reported Yes/No question for self-identifying as having a problem with drugs.

⋅ LDQ scale [24] (Leeds Dependence Questionnaire). 10-question self-report questionnaire measuring alcohol and substance dependence. Items are scored 0–1–2–3 to create a total score.

⋅ Trauma History Screen [25]. Nine different types of traumatic event historic events rated as Yes/No and total summed. Scores range from 0–9.

⋅ Homelessness. Self-reported Yes/No question for homelessness at point of entry into prison.

The outcome variable was the total number of Engager sessions attended, both remotely and face to face, and including sessions both in prison (before release) and in the community (after release). This data was extracted from session records kept by the Engager practitioners.

The data were processed and analyzed in R (version 4.2.3, R Core Team, Vienna, Austria.) [26]. We conducted frequentist versions of all significance tests (producing p-values), along with Bayesian analogues of each test (producing either Bayes Factors or Bayesian coefficients). All frequentist tests used an alpha level of 0.05 as the threshold for significance. In keeping with accepted practice [27], Bayes Factors of more than three were interpreted as evidence of an effect, while values less than one third were interpreted as evidence of no effect. Values between one third and three were interpreted as inconclusive. Bayesian coefficients were assessed using Bayesian credible intervals (CI’s), with CI’s discrete from zero interpreted as evidence of an effect. Any conflicting results between frequentist and Bayesian results were treated as a sensitivity analysis, with discrepancies noted as mixed findings and discussed accordingly.

The first phase of the analyses was to conduct simple bivariate tests between each predictor variable and the outcome variable. The majority of variables were numeric; either quantitative (e.g., number of sessions attended) or scale measures (e.g., PHQ-9). Therefore, the relationship between most variables was assessed using correlational tests of significance and effect size (Kendall’s Tau). However, some predictor variables had a binary categorization (i.e., problem with alcohol, problem with drugs and homelessness), and were assessed with t-tests of significance and Cohen’s d tests of effect size. This was followed by a second phase of analyses where we controlled for all variables together using multiple regressions. Effect sizes for the frequentist regressions were calculated using standardized Beta values, while the Bayesian regression coefficients were similarly standardized by converting the data ahead of running the Bayesian multiple regression models.

3. Results

A total of 140 men were randomized to receive Engager, the mean age was 34.3 (SD = 11.4; range 18–65) and the vast majority were white (n = 128; 93%).

Attendance data for 14 participants were missing, as session notes were not obtainable, and were omitted from the analysis. Overall, the mean number of Engager sessions received (in prison and in the community) was 17.50 (SD = 13.01; min 1; max 78). Whilst in prison the vast majority (n = 121; 96.1%) received at least one session, 30 (23.8%) received four to five sessions in prison and 17 (13.4%) received ten or more sessions in prison. In the community 108 (85.7%) received at least one session (either face-to-face or over the phone), 24 (19%) receiving between two and five sessions and 24 (19%) receiving over 18 sessions. Twenty-one people (16.6%) received no sessions in the community.

Descriptive statistics for each of the eleven predictor variables are presented in Table 1.

The results of the bivariate tests are presented in Table 2. For most of the variables, there was evidence of no relationship with number of sessions attended (i.e., Bayes Factors (BF) <0.33). However, the three measures related to alcohol use did show a significant positive relationship, on both the frequentist and Bayesian tests. One variable (GAD-7 anxiety) produced an inconclusive result (i.e., BF between 0.33 and 3).

The mean number of sessions attended was 22.52 (SD = 15.67) for participants self-identifying as having an alcohol problem, versus 14.41 (SD = 9.98) for those who did not. The correlations for LDQ (alcohol and substance dependence) and days used alcohol were both positive. Taken together, these results suggest that higher (and potentially more problematic) alcohol use is predictive of better engagement with the Engager intervention.

Regression analyses were conducted to test whether the significant bivariate effects remained when controlling for the other variables within a single model. The three significant bivariate effects were all measures of alcohol use, so there was a strong theoretical basis for not including all of these measures within a single model. We tested this point by first running a regression analysis (using both a frequentist and Bayesian model) containing only the three alcohol-related predictor variables (LDQ; alcohol and substance dependence), self-identified problem with alcohol, and days used alcohol. We found that none of these variables remained significant predictors of attendance (see Table 3).

To overcome this issue, we ran three separate regressions for each alcohol-related predictor variable, with each in turn controlling for all the other variables analyzed in this study. Each of the alcohol-related variables remained significant, when controlling for all other variables, in both the frequentist and Bayesian models (see Table 4). None of the other variables was significant in any model.

4. Discussion

Whilst the Engager intervention did not show effectiveness in the main trial analysis, the process evaluation data showed a positive change for some individuals that was potentially linked to session attendance [7, 8, 10], prompting this additional analysis. Our exploratory investigation of the Engager quantitative trial data revealed that self-identifying as having an alcohol problem, the number of days drinking (in the month prior to entry into prison) and dependance, all indicative of higher alcohol use, predict better engagement with the Engager intervention, in terms of the number of sessions attended. We did not find any relationship with common mental health problems, psychological distress, personality disorder, trauma or homelessness.

The evidence base on factors associated with attendance is mixed and so are our findings. We had anticipated that those with more severe common mental health issues, especially depression (PHQ-9 score) would tend to engage less, since studies within justice populations [12, 13] and general community samples [14, 15, 16] have reported this. However, for the men receiving Engager their level of depression did not appear to impact attendance rates. We did not have a clear prediction based on the literature in relation to psychological distress and anxiety because of the previously mixed findings [14, 15, 16] and our findings add to this mixed evidence. In the Engager study, measures used to assess mental health and psychological distress were collected at baseline while the men were in prison. While the aim was to randomize and commence the intervention as soon as possible, this was not always the case and for some there was a delay. The time between baseline assessment and first prison session ranged from one to 139 days, with a mean of 22 days (SD = 22.6) [8]. There is evidence of good temporal stability of the questionnaires used [28, 29, 30] but we cannot rule out fluctuation in scores over time, given that there are no studies of stability of these measures in prison populations. In our study, the PHQ-9 and GAD-7 were only collected at baseline, so we have no comparison, but the CORE-OM was collected at multiple time points. We did see that mean scores for the CORE-OM changed differentially over time depending on location. We observed that for some men, psychological distress was low while in prison as they felt safe and contained, while for others, being in prison itself was psychologically distressing [8]. Therefore, it may be that these measures are not a stable enough measure given the impact of location.

Our findings contrast with the previous evidence in relation to substance use [12, 17, 18]. We found no evidence to suggest that illegal substance use was linked to poorer attendance in Engager. We did find that self-identifying as having an alcohol problem, the number of days drinking (in the month prior to entry into prison) and dependance were all indicative of better engagement with the Engager intervention. Research has suggested that alcohol users are just as likely to engage in and benefit from psychological therapy, in fact those with moderate alcohol use may gain more from therapy [19]. This may explain the increased attendance for this group in the Engager intervention, why we also saw that participants attending more sessions were receiving more therapeutic sessions and that we have defined this group as “crisis but coping”. It is postulated that moderate levels of alcohol use are linked to a capacity to tolerate distress and so these participants were able to tolerate more anxiety provoking situations and not rely on safety and avoidant behaviors [19], thus possibly explaining engagement.

The Engager intervention differs in some important ways from more traditional psychological therapy which may also account for the findings. While Engager included elements of psychological therapy it was created to be deliberately flexible, designed to support whatever goals an individual needed and wanted. Therefore, Engager participants could guide the focus of sessions, this means for some they may have wanted to focus on only practical issues, while for others they may have chosen sessions to be more psychologically challenging. The variability of sessions does make it harder for us to draw firm conclusions, but it would suggest that Engager as an approach, at the very least, does not deter those with more severe common mental health problems or substance use from engaging and it is possible that those with more moderate alcohol use are able to engage more. Further research is needed to test the assumption that this could be linked to a person’s ability to tolerate distress.

We also note that there was a somewhat marginal non-significant result for trauma in two of the regression models (A and B). It is not possible to draw any inferences from these results, but further investigation of past trauma as a predictor of attendance in prisoner mental health populations may be a useful avenue for future research.

Further research is needed to unpick the complexities of engagement in interventions for men in prison and whether these also reflect more ubiquitous barriers and facilitators seen in the general population of those accessing mental health services, is worthy of further investigation. Research is needed to help us to understand why individuals with problem alcohol use seem to engage better, in at least some circumstances, and if this is due to distress tolerance or other factors. Also, if this is specific to the Engager intervention, specific to men in prison or a more global phenomenon. Further mixed-methods research is needed into what factors impact on engagement. In this analysis we were unable to look at therapeutic alliance as over the course of delivering Engager we had 11 different staff delivering the intervention [8] but there is evidence to suggest that alliance with the therapist is a significant predictor of treatment outcomes [31].

There are several limitations to this study that need to be highlighted. This is an exploratory analysis of existing data, rather than collecting new data according to a pre-developed protocol, although the existing data analyzed, was itself collected as part of a pre-developed evaluation/intervention protocol. We have assumed here that attendance in session is a measure of success and/or engagement and that may have not always been the case, it would have been beneficial to have other measures of engagement in addition to attendance. We cannot specifically explain why the inclusion of all three alcohol measures in a single regression model removes any significant effect. This needs further investigation, particularly understanding which aspect(s) of alcohol use drives engagement. For example, it is possible (and intuitive) that the aspect of problematic alcohol use that predicts attendance is captured by all three measures, even though all three measures also capture unique aspects that might not predict attendance. However, it is beyond the scope of the current study to unpick these potential nuances. Additionally, a follow up study could consider factors beyond mental health/psychological measures, for example demographic variables and protected characteristics.

5. Conclusions

The Engager intervention was not shown to be effective from a previous evaluation of standard outcome measures, although process evaluation data appeared to show positive change for some individuals that was linked to session attendance. Our exploratory investigation of the Engager quantitative evaluation data produced some unexpected findings, where higher alcohol use predicts better Engager session attendance. There is some speculation from other research that this could be due to distress tolerance in moderate alcohol users, and further research is warranted. Furthermore, all other variables showed no evidence of impacting attendance. These results suggest that aspects of the Engager intervention may have been effective, in terms of successfully engaging some individuals from the prison population, or at the very least not impacting on engagement. However, further research is needed to accurately understand the underlying causes.

Availability of data and materials

The Engager study protocol has been published and along with the original evaluation. Anonymized data may be made available by request to the corresponding author. An application for trial registration was made in December 2015 in the normal way through the NIHR portfolio registration system (before recruitment) but due to administrative delays the ISRCTN registration date was 04 February 2016 while recruitment started 14 January 2016. The discrepancies from the published protocol included provision of “top-up” training for new and existing practitioners during the trial and provision of “meta-supervision” to support Engager team leader supervisors to overcome ongoing operational problems in order to optimize (but not change) intervention delivery.

Author contributions

CL—drafted the manuscript and project managed the research. SGS—undertook the analysis. SL—collected and collated the attendance data. RB—was Chief Investigator for the project and designed the Engager research. SGS and SL—contributed to the drafting of the manuscript. All authors contributed to editorial changes in the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

This research was performed in accordance with the Declaration of Helsinki. All procedures involving patients were approved by the UK National Health Service, Wales Research Ethics Committee 3 (ref: 15/WA/0314), and the National Research Committee of Her Majesty’s Prison and Probation Service. The trial was registered as ISRCTN11707331 (04 February 2016). All participants consented to participate.

Acknowledgment

We would like to acknowledge the invaluable input and support of Tim Kirkpatrick, who is very sadly no longer with us, in both the Engager project, and the very early stages of this analysis.

Funding

This is independent research funded by the National Institute for Health Research Programme Grants for Applied Research (RP-PG-1210-12011) with further support from the Applied Research Collaboration Southwest Peninsula. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.

Conflict of interest

The authors declare no conflict of interest.