The organisation of interagency training to safeguard children in England: a case study using realistic evaluation

Aims

The overall aim of the research of which this report is a part was to develop an evidence base for interagency training to safeguard children. It was jointly commissioned by the (then) Department for Children Schools and Families and the Department of Health in England as part of a research programme on safeguarding children. In other papers we shall present a series of the evaluations of the learning outcomes of selected interagency training courses. Here we report findings of the first aim of the study, which was to investigate how the agency partners in Local Safeguarding Children Boards carried out their statutory responsibilities to organise interagency training. Couched in terms of Realistic Evaluation, this component of the study examined the mechanisms or processes by which a policy intervention (statutory guidance) produced particular outputs (interagency training for joint working) and outcomes (effective partnerships), within particular organisational contexts (local conditions in the Local Safeguarding Children Boards).

Case studies in eight Local Safeguarding Children Board sites

The study took place in eight Local Safeguarding Children Boards in four regions of England during 2007ߝ2008. A protocol was established for inclusion in the study and Local Safeguarding Children Boards in four regions of England, the North East, West Midlands, South West and London were invited to apply. Participation on the evaluation would meet the government’s expectation that Local Safeguarding Children Boards evaluate their training. Fifteen applications were received for the eight places and study sites selected to ensure representation and type of local authority (country council, unitary and London borough). The chosen Local Safeguarding Children Boards had all demonstrated a commitment to the evaluation by the Local Safeguarding Children Board Chair and were providing substantial programmes of interagency training. We undertook a detailed case study in each site between January and February 2008. The components of the case study, which was approved by the School for Policy Studies research Ethics Committee on behalf of the University of Bristol, are described below.

Analysis of documents

A range of relevant documents was reviewed. These included: annual reports and business plans; training strategies; training programmes; and minutes of the training sub-group (or equivalent) meetings. This documentary evidence provided information about the formal goals and aims of the training programmes, the financial contributions made by partner agencies and the number of courses provided.

Observation

Non-participant observation of meetings of the training sub-groups was used to collect information about the processes through which training needs were discussed and decisions were made about the content of the programme. The project researcher was able to attend meetings in six of the eight sites. Prior to the meetings, he reviewed minutes of the previous two meetings to acquaint himself with the recent issues. Permission was sought and received to take notes of the meeting, including personal observations of group dynamics.

Interviews with stakeholders

Personal and telephone interviews were held with key stakeholders: the agency managers/representatives on the training sub-groups and the training coordinators, who are appointed to organise and deliver training. The aim was to interview at least one member from each partner agency. The topics are summarised in Table 1.

Forty-five agency representatives were approached for an interview and 39 agreed. The remainder were unavailable because of leave/sickness or pressure of work. All eight training co-ordinators were interviewed. Interview schedules were designed by both authors. The interviews and observations and initial data analyses were carried out by DP. Subsequent analyses using the theoretical frameworks were carried out by both authors.

Data collection

During the course of the personal and telephone interviews, answers to questions posed were written on pro forma devised for the study. Hand-written notes were then transcribed into electronic format and tabulated by site. Permission to use anonymised quotes from the interviews was granted by each of the interviewees upon completion of the interview.

A process of emerging themes [11] was used to categorise responses into general statements which reflected the context and process of interagency training and collaboration in each project site. Thus, if there was agreement on a particular topic area, e.g., that the training programme needed more funding, then the general finding would be “training programme requires more funding”. If, on the other hand, there was disagreement by some partner agency representatives, this was recorded as “some felt that funding levels are adequate, while others felt that more funding is needed”. The same process was applied for each topic/question area and series of interviews. This cumulative process of thematic analysis allowed us both to compare and contrast opinion on interagency training for joint working within sites and also between sites.

Data analyses

A deductive approach was employed for analysing the data. The thematic content analysis of interview data was guided by the Effective Partnership Working Inventory [12]. This inventory is based on substantial research and theory within the field of organisational psychology. It informed the ‘mechanism’ (or process) component within the Realistic Evaluation framework. The Effective Partnership Working Inventory aims to assess seven dimensions: shared commitment to goals and objectives; Interdependence of outcomes; Role clarity; Cultural congruity; Focus on quality and innovation; True cooperation; and, Interprofessional trust and respect [12, p. 81].

Outcomes (vs. outputs)

In applying the Realistic Evaluation framework to the study of interagency training, we differentiated outputs (the training courses provided) from the outcomes, which were defined as ‘benefits for staff’ in terms of learning outcomes and better working relationships between staff from partner agencies resulting from participation in interagency training. There might also be beneficial outcomes to the organisations involved, e.g., more effective and efficient use of scarce resources and, potentially, for children and families.

Determining the government’s intended outcomes for interagency training was straightforward because these were handed down in statutory guidance through Working Together [2]. Interviews and training strategies revealed that a clear consensus among partners, i.e., creating a better-trained workforce to work together to safeguard children and to maintain their well-being, e.g.

Another manager believed that these outcomes were already being achieved, considering that there was,

However, keeping focused on these outcomes could be difficult in a busy policy context and a pressured practice environment,

Past research [13] distinguished three different types of outcome that a partnership may be trying to achieve: synergy, transformation and budget enlargement. Interagency training has an element of ‘synergy’ as it brings together partners with different assets and powers to create something which is greater than the sum of its parts. There was also some sense that the work of the training sub-group leads to ‘transformation’ in that it brings partner agencies together to change the objectives and culture of the organisations, with the direction of change depending on the power of each individual partner. However, this was more often an unstated than stated outcome. Although enlargement of the training budget did result from resources contributed to core funding by partner agencies, unlike in many cases of interagency partnerships there was no additional government money to promote the activity.

There were varying degrees of ‘resource’ and ‘policy’ synergy [14]. Agency managers commented repeatedly on the ‘added value’ from the resources contributed to core funding; they believed that there has been increased effectiveness and to some extent efficiency. In terms of policy synergy, it was suggested that there had been new perspectives and solutions developed in some of the sites with respect to the courses delivered and special awareness raising sessions, e.g. one of the sites held an away day for Local Safeguarding Children Board members to discuss emerging issues and training needs.

Past research [14], which built on the concept of ‘transformation’ [13], suggested that different outcomes can occur depending on the power balance between partners. The interviews revealed that there were indeed power imbalances in the training partnerships, particularly between health and social care and other partner agencies, but they do not appear to have either hampered or facilitated the work of the training sub-groups.

There was a clear sense that partner agencies had accepted the need for change and to learn from—and more importantly, about—each other, so ‘mutual transformation’, whereby all partners change and differences between them begin to reduce, had occurred in varying degrees across the sites.

Context

When interpreting the findings of the interviews, it is important to bear in mind that safeguarding training had been mandated or ‘forced’. It had started from different positions in each site and its success was likely to depend on a number of factors, including past working relationships. Relationships in some sites were thought to be very good,

whereas in others they were not so conducive to effective partnership working, e.g.

Mechanism/Process

In seven of the eight sites, a training sub group had been established. Here we used the Effective Partnership Working Inventory [12] to assess the seven domains in which these groups were operating effectively as partnerships.

The key role of the training co-ordinator

The importance of the training coordinator in the organisation and delivery of interagency training cannot be understated; this was the key person responsible for designing, delivering and coordinating the programme. In seven of the eight sites training coordinators they delivered training themselves. They were responsible for drafting and reviewing the training strategy and the training budget. They promoted the training and, with the help of their administrators, circulated the training programme to partner agencies. Some also managed administrative staff and provided training and support for the trainers. In two sites (Site F and Site G) the training coordinator was the chair of the training sub-group; and all acted as intermediaries between the training sub-group and the Local Safeguarding Children Board itself. Training coordinators commissioned training courses and generated revenue through delivering safeguarding courses to non-contributing partner agencies, such as voluntary and private sector organisations. Finally, they played a key role in monitoring and reviewing courses at all levels of delivery.

There are clearly positive aspects and risks with arrangements based so significantly on the efforts of one individual. On the plus side, their personal knowledge and commitment was a significant factor in ensuring that the programme continued. However, this made it vulnerable as was highlighted in Site D during the course of the study when the training coordinator went on extended compassionate leave. The training support group members (one of whom sat on the Local Safeguarding Children Board) pulled together to provide cover and additional administrative staff were employed but it was not easy to replace him. Although this make-shift arrangement did manage to deliver the necessary safeguarding training, this highlighted the vulnerability of the training programme as a whole.

Outputs

Although the organisational structures for interagency training were similar, there were quite large differences in the ‘outputs’ measured in terms of the numbers and types of courses delivered by each Local Safeguarding Children Board in the training year 2007/8 (Table 2).

The courses were categorised in terms of their target audiences as defined in government guidance [1, p. 100]. ‘Foundation’ courses provided a general introduction to safeguarding. Training at level 1 was for staff in ‘regular contact’ with children aimed at identifying and responding to child abuse. Training at level 2, focused on ‘working together’ for those who ‘worked regularly’ with children. Level 3 referred to specialist courses for children who were vulnerable because they were disabled, living with domestic violence or parental mental illness etc.

Differences in the kinds of courses are apparent (Table 2). Four sites offered free access to foundation level e-learning courses; minutes showed that these were chosen as an inexpensive way of training large numbers of front-line staff. In other sites, foundation training was agreed to be the task of single agency training. Twenty per cent of the foundation courses were for two days. Three Local Safeguarding Children Boards did not offer any of these, having decided that it was too difficult for partner agencies to release staff for all this time and concluding that the same learning outcomes could be achieved in one day. The only evidence to support this was from the ‘happy sheets’. Site G had prioritised specialist level 3 courses (58%) of their provision, compared to only 12.5% of such courses in site F. This was an example of different understandings of training needs for agency staff rather than the profile of families.

In order to assess volume of the output (training provision) in relation to children’s needs, the numbers of courses and the days of training provided were divided by the local authorities’ own estimates of the numbers of children in need (all vulnerable children, including those in need of child protection). The figures for 2009 (the nearest available) were published by Department for Children Schools and Families [15]. These analyses show that the number of courses per children in need varied from one per 54 children (Site C, a county council) to one per 156 in Site H (a London Borough), which was providing roughly two-thirds number of courses of the average for the sites. Taking into account the length of the courses offered, Site C was providing nearly twice as many course days per child in need than the average, as well as the largest number of courses and course days. This is attributable to a strong and active training subgroup and a particularly energetic training coordinator who was supported by two assistants.

The output of training courses is likely to be influenced by the size of the financial contributions made available to the training subgroup by the partner agencies. As shown in Table 2, Site C had the second highest budget provided by the partner agencies, more than twice that of Site B, which had a similar number of children in need; in this respect it was very similar to Site F, but that Local Safeguarding Children Board was providing only two-thirds of the course days offered by Site C. Consequently, the cost per course day (ratio C:D) was over £1000 more in Site F compared to Site C. Course days were also good ‘value for financial contribution’ in sites A, B and E. However, sites G and H were considerably less cost efficient. Site G had easily the highest proportion of children in need and the largest budget, but also spent close to 50% above the average per training course day. Cost efficiency estimates, however, do not take account that in many of these the partner agencies made significant contributions ‘in kind’ through members of their staff who acted as trainers at no charge to the Board. So although Site G may appear less cost efficient relative to the other sites, this is large part explained by the fact that the training budget is based on annual ‘subscriptions’ paid by partner agencies, which were ear-marked for training and as such already included various in-kind contributions found in some of the other sites. Site H, on the other hand, was the only Local Safeguarding Children Board which did not have a dedicated training subgroup. There was relatively little involvement of agency representatives, the training coordinator was not supported by an active group of colleagues and the Local Safeguarding Children Board had taken the decision to commission all its training from external trainers, which was more expensive than in-house training.

In terms of partner agency contributions per child in need (ratio B:D), estimates varied from a low of £29.30 per child in need in Site A to a high of £62 per child in need in Site C. However, there was no simple correlation between these contributions and the numbers of children in need. This is not surprising because, in the absence of a government funding formula, Local Safeguarding Children Boards are left to negotiate the contributions from partners and in a recent survey, over half of Local Safeguarding Children Board chairs considered these insufficient [16, p. 34–36]. Anticipated cuts in partners budgets (June 2010) highlight the vulnerability of the system: a third of the 57 chairs in the recent survey ranked training first or second in importance when asked about the effects of an inadequate budget [16, p. 35].

Organisational outcomes

Organisational outcomes varied in terms of synergy, transformation and budget enlargement in this research [13]. Interagency training displayed elements of synergy as it brought together partner agencies with different assets and institutional powers to implement an interagency training programme which no one agency could effectively provide on its own. Organisational transformation was occurring; albeit on a very small scale through joint working of members on the training sub-groups, some of which were effected by power imbalances. Budget enlargement did occur via the pooling of several sources of training funds, but that this was more distributive between partner agencies as no additional monies were made available by central government. Instead, partners pooled their own staff’s time and skills to develop and, in most cases, deliver the programmes.

Sites showed varying degrees of ‘resource’ and ‘policy’ synergy [14]. Agency managers felt that interagency training for joint working had increased the effectiveness of joint training as well as the efficiency in terms of the efficient use of available funding resources. Some sites have introduced new training courses and information sharing sessions, all of which have created some policy synergies between partner agencies. In short, these synergies have led to clearer standards for training staff who work with children and families and increased staff awareness about child protection issues, both of which have had positive consequences for children’s services in the Local Safeguarding Children Boards participating in the evaluation. Although there have been some synergies, the extent to which real transformation has occurred is much less evident [14]. Partner agencies have accepted the need for change and to learn from other partner agencies, and a lot of ground has been made in terms of the breaking down of prejudices, misperceptions, misunderstandings, and power imbalances between partner agencies.

Context

In the case of interagency training for joint working, we encountered a situation or ‘context’ wherein we have a ‘forced partnership’ placed on Local Safeguarding Children Boards by a central government mandate, in other words, there is no other choice but for partner agencies to work together. Moreover, mandatory interagency training for joint working was taking place without any additional funds from central government. This mandated collaboration has affected the outcome and outputs of interagency training for joint working [17]. Some partner agencies felt that the costs borne of interagency training are too high and that they have been forced into relationships that appear suboptimal to them, yet potentially beneficial to the Local Safeguarding Children Board as a whole. This finding supports literature on organisational collaboration which states that networks that have been mandated by central governmental have different dynamics than those that come together by mutual consent [17]. However, even within a strong and well-established culture of mandated collaboration, interagency training regimes varied considerably across Local Safeguarding Children Boards. These findings are consistent with a body of research evidence on some of the negative consequences of ‘mandated partnerships’ in the public sector [18–21].

Agency managers displayed optimist, pessimist and realist approaches to interagency training for joint working [22]. That partner agencies were forced to collaborate as a direct result of government policy represented the ‘realist’ approach, while others looked at this as an opportunity to change organisational policy and practice to safeguard children and enhance their well-being, i.e. taking the ‘optimist’ approach. There were very few managers across the sites who took a wholly ‘pessimist’ approach to interagency training for joint working [17].

Mechanisms or ‘process’

The training sub-group is the main vehicle for ensuring that the outcomes of interagency training were achieved. Application of the Effective Partnership Working Inventory [12] showed that shared commitment to goals and objectives and interdependence of outcomes among partner agencies were the strongest domains, followed by role clarity and cultural congruity. Focus on quality and innovation, true cooperation, and interprofessional trust and respect were the weakest domains in the inventory [12].

The success (or failure) of the training sub-groups in organising and delivering interagency training for joint working (the ‘mechanism’) was affected by past working relationships between partner agencies, and that between Area Child Protection Committees (the forerunners to Local Safeguarding Children Boards) and partner agencies. Where there was a history of positive working relationships prior to the formation of the Local Safeguarding Children Board, then it was more likely that interagency training working under the Local Safeguarding Children Board continued as before. This too holds true for joint working relationships that did not share a common vision—and shared commitment—to safeguarding children and were not characterised by trust and reciprocity, and where autonomy was lacking [23]. As past research has pointed out [12], it is important to have interdependence of outcomes, which—in the case of interagency training for joint working—we do. There were some challenges in the mechanisms in terms of agency managers not knowing exactly what their role was on the training sub-group and organisational culture clashes, but these were the exception not the rule and did not appear to impact on the effectiveness of the training sub-groups in organising and delivering training [12].

A training sub-group made up of agency managers ‘forced’ or ‘mandated’ to work together was successful because the responsibilities set down to them by central government act as a unifying force, which acted as an impetus for overcoming some of the challenges and barriers encountered in the organisation of interagency training for joint working in the past. Vested interests, lack of organisational and role clarity, inter-professional antagonisms, and funding conflicts—although present—did not take centre stage as each partner agency was responsible for contributing funding and obliged to collaborate under Government legislation. Moreover, having clear objectives handed down by Government in terms of training content and participation in interagency training has resulted in less training duplication and a reduction in conflicting interagency training for joint working ‘agendas’.

Limitations

When interpreting the findings of this research, it is important to keep in mind that the evaluation team—through its inclusion criteria—chose Local Safeguarding Children Boards that would be in a position to contribute meaningfully to the study. The recruitment of sites with what could be deemed well-functioning inter-agency training programmes was intended to enhance the chances of observing effective partnership working and the identification of good practice from which other Local Safeguarding Children Boards may learn. Findings from an evaluation which included sites without a well-established structure for the commissioning and provision of training (e.g. training sub-committee and training co-ordinator/s) or a substantial programme of interagency training courses might have resulted in different findings.