NHS England invested £200 million in digital elective recovery schemes as part of the Targeted Investment Fund (TIF) in 2020/21.
Arden & GEM’s Health and Care Transformation team was commissioned to deliver an evaluation to assess the impact of the TIF investment in digital schemes and provide insight and learning for future funding streams.
By bringing together qualitative and quantitative methods, the evaluation team was able to engage with scheme leads to gain a better understanding of impact and implications which was shared with NHS England through a detailed final report and audience-specific slide packs.
The challenge
NHS waiting lists and times worsened considerably after the COVID-19 pandemic. To help the elective care recovery, NHS England invested over £200 million in digital ‘elective recovery tech schemes’ in England. Funding was granted to 481 digital schemes with the aim of supporting a reduction in elective care waiting lists through delivering digitisation, connectivity or other transformation of pathways.
NHS England commissioned Arden & GEM’s Health and Care Transformation (HCT) team to carry out an evaluation of the impact of TIF funding. The objective was to get a national view of the impact and benefits realised from the TIF funding on digitally transforming patient care and accelerating elective care recovery.
NHS England also wanted to understand the barriers and enablers providers faced in implementing and realising the benefits of their schemes, to gain insight and learning to improve the impact of future funding streams.
Our approach
The HCT team brought together experience from evaluating other national digital programmes using expertise in both qualitative and quantitative methods to assess the impact of the TIF digital funding.
Light-touch evaluation
A quantitative ‘light-touch evaluation’ approach was used across all funded schemes. Each scheme chose one of 12 standardised metrics to report their performance against for two consecutive quarters in 2022.
Engagement with providers was initially challenging but by working in collaboration with NHS England regional digital teams and Trust Chief Information Officers, reporting leads were identified for 72% of all eligible schemes. Ongoing communication with and reporting across all 481 digital schemes was managed through Microsoft Power Automate to streamline processes and ensure efficiency.
Deep dive interviews
Semi-structured interviews with clinical and delivery leads took place with 13 schemes to capture learning and best practice as well as improving understanding of reporting. Interview sampling focused on high-value schemes and/or those with the potential to have the greatest impact for patients.
The outcomes
Interviews with schemes revealed that, in many cases, TIF funding had a transformative impact which allowed providers to:
- accelerate the pace and scale of projects
- negotiate better terms/contracts with vendors
- conduct testing of technologies/ tools that they may otherwise not have been able to invest in.
Insights from the interviews enabled us to develop implications for practice for future funding stream administrators and for teams supporting providers in implementing and reporting schemes.
From the light-touch evaluation, we were able to provide benefit highlights from some reporting schemes, however, the main outcome was learning points for those planning to measure impacts over multiple schemes in the future. For example, a key finding was that most schemes had not been fully implemented and started realising benefits within the reporting period. By bringing together key findings and insights, we were able to identify common themes, barriers and challenges across schemes and develop a series of recommendations for future funds and programmes.
The final outputs included a detailed written report for NHS England, along with tailored slide decks to reflect the interest and needs of different audiences and stakeholder groups.
"With 481 digital schemes supported via the Targeted Investment Fund in 20/21, NHS England needed support to evaluate the initial outputs and manage the data returns. Arden & GEM successfully engaged with multiple trusts who received the highest value awards and interviewed leads on the impact of funding, alongside collecting metrics from a wider range of trusts.
A determined and professional approach to updating key contacts and collating available evidence and metrics allowed key learning to be incorporated into future digital funding schemes.
Arden & GEM were flexible and professional in their delivery of reports and presentations, adapting to new requests as required and meeting regularly to update NHS England colleagues on progress and any issues. We were able to build a strong working relationship with multiple colleagues across the team and I’d be pleased to recommend their input to anyone requiring expertise and support on similar project or programme evaluations."
Richard Samuel, Director of Outpatient Recovery and Transformation at NHS England
Find out more about our support for digital transformation here.