Post project evaluations often fall by the wayside, but they have an important role to play in creating successful and efficient healthcare facilities
Post Project Evaluation (PPE) should be a fundamental part of healthcare projects. After all, it is a mandatory step in the Capital Investment Manual (CIM) – the NHS bible that provides guidance on all stages of the project procurement process. The requirement is also in every full business case document that makes the case for investment in a facility.
One assumes that the CIM includes this final step because, within the vast organisation of the NHS, with everyone working to a common goal of achieving equality of excellence for all users, lessons learned should be passed between providers for the greater good of patients and staff. The potential benefits are numerous – avoiding the repetition of mistakes, repeating successes, improving processes and making the entire procurement and delivery process smarter so that resources are applied where they have most impact.
At a more strategic level, PPEs also now offer Trusts the opportunity to collect and share evidence that a project has demonstrably improved service quality across a range of key performance measures. As choice evolves, this information can be used to support corporate marketing initiatives aimed at patients considering where they want to be treated.
Handing the project over
We all know that at the end of most projects, the project teams that have nursed the scheme throughout its development hand over the building to the operational staff, many of whom have had limited involvement along the way.
Project team members then either revert to their previous ‘day jobs’ or move on to other projects. No one then has the opportunity or motivation to pause, look back and formally review the way the project developed and what it really provides when the business of delivering services beds down. The context of decision making is lost, people live with the building, making it work despite any obstacles, and praise or censure becomes anecdotal at best. Continuity of project team involvement, together with cost pressures which may cause the appraisal process to be viewed as a low priority, mean that in most cases, formal evaluation never happens.
All providers of design and construction services to the NHS have an interest in PPE, not just from each speciality’s own perspective, but also from the point of view of the end users. The best projects are full partnerships between everyone involved and the best evaluations look at the project in the round. Whilst we may conduct evaluations within our own businesses against our own commercial objectives, we all need to know whether the building works well in use – otherwise, the expertise we have built up is dissipated and we have no evidence base which we can use as we move on.
Communication
We have all experienced the frustration of reinventing the wheel with every project – endless debates over finishes and materials, critical dimensions, the feasibility of open plan offices, how to organise a theatre suite, the best organisation of out patient suites and so on – when we should be able to share evidence from other projects in an informed debate and reach quick decisions with confidence. For example, where is the evidence base to confirm that operating theatres should have natural daylight?
Our own experience, despite the interest and involvement of organisations such as the Future Healthcare Network and CABE, is that there is surprisingly little informal communication between NHS organisations. Project teams can be entirely unaware that a facility very similar to the one where they are embarking on the planning stage has recently opened in another part of the country and is bursting to pass on the lessons they have learned.
There are, of course, exceptions to the rule and examples of excellent evaluations. However, with the new waves of investment recently announced for community hospitals in Our health, our care, our community: investing in the future of community hospitals and services; the renewed emphasis on and support for Procure 21 from the Department of Health heralded by its director of estates and facilities, Rob Smith and the expansion of PPP and Hub into Northern Ireland and Scotland respectively, it is vital that we make PPE a reality. The industry needs to underpin these new developments with evidence-based decisions as a result of experience from everyone from designers through to the end users.
The tables on these pages reveal some of the lessons Cyril Sweett uncovered when evaluating two PFI acute hospitals and how we have subsequently captured and applied the successes and sought to mitigate any shortcomings on our subsequent projects.
Stephanie Brada is director of project management at Cyril Sweett (stephanie.brada@cyrilsweett.com).
Learning from experience: what needs to change?
Having carried out a number of PPEs for health sector clients, Cyril Sweett has identified several areas that we think should be re-emphasised or changed by the Department of Health to enhance the quality of future evaluations:
Realistic benefits realisation plans
We have found that full business cases on some projects have enormous benefits realisation plans that require extensive data collection and analysis at the PPE stage. For example, a benefits realisation plan on a PFI project we reviewed included 10 headline benefits which were then subdivided into 100 specific benefits with 231 different measures proposed. Another PFI benefits realisation plan had nine headline benefits, 45 specific measures and 238 different measures.
Asking busy Trust Finance and Clinical Audit departments to collect mountains of statistical data has proved very onerous for many who have more pressing day-to-day concerns and needs to meet. Therefore, when preparing business cases, Trusts should consider using only measures for benefits that are easily accessible, include baseline data and not use half a dozen data sets to measure a benefit where one will do.
Making an early start
Our experience has encouraged us to promote the formal recording of evaluation early in the process – with key staff recording as they go along the decisions which caused them the most anxiety, where they believe they made a compromise under pressure and what gave them sleepless nights. Comparing these concerns with the experiences of users at three and 15 months can be a vital aid to robust decision making on subsequent projects - and reduce the number of grey hairs acquired during the average project!
Collect qualitative before and after data
We also believe it is useful to have good quality, comparable data sets for before and after the transition into the new facilities. To do this, we use questionnaires to capture the views of patients and staff in the old facilities and ensure that the same topics are covered in questionnaires that are provided once the services have settled into the new facilities. On one project, we found that owing to staff turnover rates or short memories, 16% of staff surveyed could not provide an answer to the question: ‘does the new hospital provide a better environment than the previous hospital?’ A more statistically robust sample size and more detailed comparative analysis would have been possible had the Trust considered the starting the PPE prior to occupation of the new hospital.
Make the PPE relevant to stakeholders
People will make quality contributions to the process if they can see that there is a tangible benefit for them and that they are making a positive contribution to the wider NHS. One way of making it relevant is by responding to issues raised during the process. For example, addressing heavy doors that bang when they close may not be seen as a priority on a snagging list but when the feedback is received in the context of patients’ day-to-day experiences, the evaluation team is in a position to make better quality recommendations on remedial actions. On our PPEs, we have captured and categorised user’s comments on the building and fed them back to the Trust’s project team for action.
Spread the word
Finally, and crucially, the lessons learned across all projects should themselves be collated, evaluated and disseminated to all those involved in the development of healthcare projects. Perhaps the creation of an accessible, centralised evidence base has been unwittingly prevented by a combination of factors such as handing over responsibility for project delivery to individual and competing Trusts; the division of responsibility for overseeing projects to agencies such as the Private Finance Unit, Partnerships for Health and the Procure 21 Framework Managers; and further devolution of responsibilities for capital project delivery to yet more agencies in Scotland, Wales and Northern Ireland. These factors are allied to the demise of NHS Estates.
Post Project Evaluations have the potential to yield tangible cost and process benefits for everyone involved in capital projects in the NHS. All too often the pressures within the NHS cause it to fall by the wayside. We believe it is time for the investment, design, service delivery and construction industries to take a lead on research and evidence based project procurement and implementation. We should initiate and contribute to a comprehensive PPE programme for the benefit of us all.
Source
Building Sustainable Design
No comments yet