Notice: Function Elementor\Core\Experiments\Manager::warn_removed_or_hidden_dependency was called incorrectly. Feature mega-menu depends on hidden experiment nested-elements. Please see Debugging in WordPress for more information. (This message was added in version 4.4.1.) in /home/u63711/domains/thecharityspark.co.uk/public_html/wp-includes/functions.php on line 6260
Using AI effectively in evaluation and impact measurement – The Charity Spark

Using AI effectively in evaluation and impact measurement

What is AI?

Artificial Intelligence (AI) is the “simulation of human intelligence in machines that are programmed to think like humans and mimic their action” (www.investopedia.com). It also refers to machines or computers that are able to exhibit traits that are associated with the human brain, such as the ability to learn, problem-solve, plan, adapt and improvise.

AI can be used for many different things, some examples are navigation and maps, facial recognition, text-editors and autocorrect (to a much more detailed and accurate level than we have previously used), search engine and recommendation algorithms, chatbots and customer services, digital assistants and social media. Some of these things you are probably already using in your daily life, some you will be using a basic version of many of these and as AI becomes more accurate, cheaper to produce and more powerful we will be using it more often. 

Like everything, there are pros and cons to using Artificial Intelligence:

Pros:

  • Error-free processing (this can be particularly useful in evaluation)
  • Can help with repetitive jobs
  • Available 24/7
  • Can aid in correct, and fast, decision-making 
  • Can provide digital assistance
  • Can come up with new ideas and inventions

Cons:

  • High cost of creation
  • Could replace individual jobs or reduce number of hours needed to complete tasks
  • Lacks in creativity
  • Lacks in improvement of skills
  • Can’t completely replace the need for humans

We’ve taken a look at some different ways AI can be used in evaluation and impact measurement. Where you see ‘Try it’ you will find examples of how to use AI apps in your day to day work.

Data Analysis

Managing the data we collect to measure impact and quantify outcomes can be labour-intensive. We need to extract, integrate, catalogue, label and organise all the information we gather. This process can be slow and frustrating. What if we could use AI to analyse large amounts of data, including financial data, programme data, and impact data? The main ways we can use AI to do this is through:

  • Classification: acquiring, extracting and structuring data from documents, photos and other media
  • Cataloguing and locating data
  • Reducing errors in the data
  • Keeping data secure
  • Helping to build master lists and merging lists together
  • Analysing data to identify patterns and themes

This can help charity evaluators to reduce the time spent analysing the data themselves and free them up for other work. AI is also beginning to be able to deal with nuances and can suggest helpful insights rather than rely on the evaluator to run the correct tests. 

Did you know that you can use a basic AI function that it built into Excel? Click ‘Analyse Data’ to find suggestions form the Excel AI function – it isn’t hugely powerful…yet. But there are other Excel AI add-ins available like the AI Excel Bot that can help you to analyse data differently. You can bolt these on to Excel and run natural language prompts to help you with the formula you need to use but aren’t quite sure of how to write. With AI you can use natural, descriptive language for what you want the output to be, and it will create a matching formula for you. 

Natural Language Processing (NLP)

NLP is a machine learning AI that allows computers to break down and understand text similarly to how a human would. It can use sentiment analysis to read and classify the text for opinion polarity (positive, negative and neutral), into the feelings and emotions of the writer. Topic analysis is used to organise text by theme and to categorise it. NLP uses text classifiers to detect the intent of a text, is it a complaint? Does the customer want to buy a product or service? AI can read emails and redirect them to the correct member of staff. 

NLP can also be used to analyse text from charity websites, annual reports and other documents. This can help charity evaluators get a better understanding of a charity’s mission, values, and impact. This can also be used to create case studies from audio transcripts. 

Chat GPT has the ability to process and summarise chunks of text or to write copy based on previous input. Copy and paste the transcript from an interview into ChatGPT and ask the bot to ‘create a case study from the following’, it will certainly give you a starter that you can then edit and refine. 

ChatPDF also allows you to interrogate the content of a PDF and ask the AI bot natural questions about the content, your very own briefing assistant!

Image recognition

Image recognition is when a computer can identify and categorise images and videos. You’ve probably come across this being used in reverse image searches, where you can take a photo and put it into Google to find either that photo or similar ones on the web. You may have also have a smart phone that categorises photos by theme or faces. In the charity world, this pattern recognition can be used to analyse photos and videos from an event, if you are using photo journaling, AI can identify patterns in the images to draw out themes to cut down the hours of analysis needed. 

Audio transcription

AI audio transcribers can support qualitative evaluation by providing a transcript of an interview, conversation or focus group which can then be used to extract quotes, revisit key themes or analyse through thematic assessment. 

At The Charity Spark we’ve been using Otter.ai in our evaluation practice for a while now. This is a transcription tool that creates transcripts from audio or video and provides insights into the conversation e.g. themes, most used words etc as it transcribes. Otter.Ai can receive audio and video files via upload, or you can record directly into it. Whilst it isn’t a perfectly accurate transcription, it is good enough to start with and then you can listen and edit within the app, cycling to the part of the conversation you want to hear and review. It’s hugely cut down our transcription time and provides an easy on-the-go tool for qualitative interviews. You can also add otter.ai into meetings to take summary notes, supporting your own project management. 

Watch this space for part two of this blog which looks at how to use AI to support predictive analytics, considers the challenges and limitations of using AI technology in charity evaluation and summarises the benefits.

What are your favourite AI apps to use in your day to day work? Let us know!

Some links in this blog are referral links which means The Charity Spark benefits if you sign up to the service after clicking the link. We only recommend products or apps that we have used and think are great. The referral benefit is typically a month’s free premium membership or similar membership benefit to our paid-for premium accounts with those suppliers.