Notice: Function Elementor\Core\Experiments\Manager::warn_removed_or_hidden_dependency was called incorrectly. Feature mega-menu depends on hidden experiment nested-elements. Please see Debugging in WordPress for more information. (This message was added in version 4.4.1.) in /home/u63711/domains/thecharityspark.co.uk/public_html/wp-includes/functions.php on line 6260
Working with STEM Learning to evaluate their summer camps  – The Charity Spark

Working with STEM Learning to evaluate their summer camps 

Intensive STEM (science, technology, engineering and maths) summer camps were delivered at 18 schools across England during the summer of 2021 through a partnership of STEM Learning and Goldman Sachs Gives. The camps were designed to support students in improving their curriculum knowledge and re-engaging them with formal learning environments after the disruption of COVID-19. The camps also provided teachers with the opportunity to learn new techniques and gain CPD (continuing professional development) from experienced specialist teachers.

The Charity Spark were delighted to collaborate with STEM Learning to produce a comprehensive independent evaluation report that demonstrated the measurable impact of such a key project.

STEM Learning has a commitment to STEM education. Whether that’s delivering teacher CPD (continuing professional development) in STEM subjects, bringing STEM role models into schools as part of the STEM Ambassador Programme or providing bespoke, long-term support for groups of schools in collaboration with companies through the ENTHUSE Partnerships, their aim is always the same – to provide a world-leading STEM education for all young people across the UK.

The Charity Spark responded to an invitation to tender published by STEM Learning who were looking for support in evaluating a new project which was a partnership with Goldman Sachs.

“STEM Learning has a dedicated in-house Monitoring and Evaluation Team, which conducts evaluation of all the programmes STEM Learning offers. However, the team is small and for this piece of work it was decided that appointing an impartial external evaluator would be the best use of our resources.

We therefore advertised an invitation to tender through the UK Evaluation Society website, and The Charity Spark applied for this work.

The quality of the examples of The Charity Spark’s previous work and their proposal submitted in response to our invitation to tender was impressive so they were appointed to conduct the evaluation.

We didn’t have sufficient evidence of the summer camps’ efficacy, and what we did have wasn’t in a compelling, digestible format. We needed someone to gather the evidence and present it in a suitable format.”

Dr Amy Newman, Evaluation Lead for Enrichment and Enhancement, STEM Learning.

Our Evaluation

The evaluation of the STEM summer camps was designed to be part of the programme delivery.
Nine of the 18 schools were selected to be case study schools. An evaluator from The Charity Spark visited each of the schools during the summer camp to observe how the camp was run, to speak with students and teachers, and to build a portfolio of evidence to demonstrate how effectively the programme was making progress towards the intended outcomes.

All the schools were provided with a survey for students to take at the beginning and end of the summer camp. A total of 209 (58%) students responded to the survey at the beginning of their summer camp, with 197 (55%) students submitting a response to the survey at the end of the programme. At the time of writing the interim report, 18 summer camps had taken place with approximately 380 students attending.

The lead teachers at each school were sent a survey to complete before and immediately after their camp. Out of 18 camps, 12 teachers (66%) completed the pre-camp survey and 8 (44%) returned the post-camp survey.

A case study was produced for each of the nine schools visited. This included qualitative and quantitative evidence, which provided a snapshot of the student and teacher experience of the camps.
Further contact with each of the nine case study schools provided an opportunity to interview key teachers to determine what the medium-term impact of the summer camps had been. The interviews discussed student attitudes towards and attainment, benefits to the teachers, and the practicalities of the camp’s structure and timing. Seven interviews were held in October and November 2021. A survey was sent to the remaining participating schools, providing them with an opportunity to feedback on the areas of interest.

The report intended to include data that analysed the attainment of participating students, comparing mock exam data from the summer term before the camp (summer 2021) with the most recent autumn term mock exam (autumn 2021). This data was not available to us when we wrote the interim report but will be considered when the final report is written. The final report into this programme will include quantitative exam results to allow further analysis into this area of interest.

The interim report (published December 2021) has informed the remaining evaluation period, which culminates in a final report scheduled for summer 2022.

Our report found that the summer camps had supported students and teachers in making good progress towards the outcomes which the camps set out to achieve. More detail on these outcomes and how the evaluation methodology considered a range of contributing factors can be found in the report.

Evidence-based decision making

STEM Learning were really pleased with the outcome of this collaboration and the interim report was well received across the organisation. The report was submitted to external stakeholders including the funder, Goldman Sachs, and the Department for Education.

“We found the support to be very accommodating, clear and concise communication and extremely reliable. We couldn’t have asked for more, or say there was any room for improvement. We were very happy with how the team dealt with any issues that cropped up along the way.”

“Our fundraising team have been able to secure further investment from funders as a result of using The Charity Spark’s evaluation report and case studies in their discussions. We would absolutely recommend The Charity Spark and we are now collaborating on further projects.”

Dr Amy Newman, Evaluation Lead for Enrichment and Enhancement, STEM Learning.

The Charity Spark has now been commissioned to write the final end of year report into the efficacy and impact of the summer camps for STEM Learning which will be delivered in September 2022.

Need evaluation support?

The Charity Spark launched in 2019, and our focus is on collaboration to create change within the charity sector. We support charities in reaching more beneficiaries today, proving they do what they say they do and planning for a sustainable future. 

The consultancy offers tailored support across evaluation, impact and outcome measurement, and fundraising. Our team has a broad range of experiences across the not-for-profit sector including in programme development, fundraising, evaluation, and charity operations.   

Our founder and expert lead on all evaluation projects, Beckie Denny, is a full member of the Chartered Institute of Fundraising and the UK Evaluation Society. She has trained with the NCVO and Charities Evaluation Service and is a trustee of two charities.  The Charity Spark are also organisational members of the Charity Evaluation Working Group who exist to support good evaluation within the charity sector.

Get in touch today to see how we can bring insight and clarity to your project.