Every good evaluation can run into difficulties but how can you overcome them without impacting the evaluation? Here’s a few of the most common problems you might find and how to solve them.
- Confused objectives or goals
Problem: It can be really difficult to measure project effectiveness without clear goals set and you may even find that different stakeholders expect different things from the evaluation. This can lead to a segmented evaluation that doesn’t lead to clear results.
Solution: Decide on clear, measurable and realistic goals (these should be SMART: Specific, Measurable, Achievable, Realistic and Time-Bound), ensure that all stakeholders agree on these goals before beginning the evaluation or make sure that anyone taking part in a research activity is informed about the goals of the evaluation at the beginning of their interaction. If anyone is unclear take the time to talk it through before you start the evaluation process.
- Collecting poor quality data
Problem: Gathering incomplete, inaccurate or inconsistent data gives incorrect findings. This can be due to the methods used to collect the data, a low response rate or a small sample size.
Solution: Reconsider your data collection methods, think about additional training for staff, try different data collection tools or start with a smaller sample size and build up. For example, if you’re using a pen and paper survey but finding that answers are often incomplete or inconsistent consider moving to an online survey system which allows greater control over the questions people have to answer before they can submit.
- Bias
Problem: evaluator or participant bias can result in inaccurate findings. This can happen by the evaluator selecting certain data or misinterpreting data, participants giving specific responses or selection bias on the part of the organisation being evaluated, for example by selecting specific people to connect with the evaluator.
Solution: Consider using multiple evaluators where possible or an outside evaluator who is impartial, use both quantitative and qualitative data and anonymise all answers/responses if you can. Identify multiple ways for participants to be recruited in order to reduce selection bias.
- Lack of Stakeholder Engagement
Problem: Stakeholders (these may be your project staff, funders, beneficiaries or volunteers) may be unmotivated if they don’t understand the evaluation project
Solution: Involve as many stakeholders as you can in planning stage where appropriate and ensure good communication to all stakeholders. Celebrate the purpose of the evaluation and communicate the process well.
- Limited Resources
Problem: your project may be limited by time which could lead to research being conducted quickly and things being missed. Limited budget and expertise could also lead to incomplete or inaccurate data.
Solution: Prioritise important evaluation questions and areas, consider partnering with external consultants where possible and keep your project realistic within budget and timeline.
Two final considerations to keep in mind are to ensure that your proposed evaluation is an appropriate size for the programme being evaluated and that, when using an external evaluator, you may need to give more background and context of your wider business than just the details pertaining to the project being evaluated – don’t assume knowledge!
For support with your evaluation projects, do get in touch today hello@thecharityspark.co.uk