The Evelyn Oldfield Units

Community Impact Toolkit 2011-2013

Welcome to the new, easy and accessible tool to support refugee and migrant community organisations to capture their outcomes and impact.

www.evelynoldfield.co.uk

Index
Section 1 - Acknowledgements Section 2Introduction 2.1.An overview 2.2 Key definitions 2.3 Sample end of year funders requirements 2.4 Potential effects 2.5 Getting started Section 3 - Tools 3.1 Monitoring form example 3.2 Equalities and Monitoring form EXAMPLE 1 3.3 Distance travelled formEXAMPLE 2 3.4 Distance travelled form EXAMPLE 3 3.5 Outcomes stars 3.6 Key questionnaire 3.7 Sample Post Even Questionnaire 3.8 Volunteer impact survey 3.9 Advice session evaluation 3.10 Focus groups 3.11 Case studies Section 4 Extras 4.1.Involving users 4.2 Showcasing SROI 4.3 Monitoring and evaluation plan 4.4 Monitoring and evaluating matrix planning 4.5 Example stakeholder interview/ survey 4.6 Participatory methods p32 p33 p34 p35 p36 and 37 p38 p13 p14 p16 p17 p19 p21 and 22 p23 and 24 p26 and 27 p28 p29 p30 p6 p7 p9 p10 p11 p4

Please turn over

Section 5 -Post Data Collection 5.1 AnalysisSurvey Monkey 5.2 What next? 5.3 End of your report 5.4 Challenges 5.5 Further advice and support 5.7 Contact details p40 p41 p42 p43 p44 p45

1. Acknowledgements
This toolkit has been created with feedback from 8 training sessions, over 20 consultancies and with several pilot group consultancies where we directly tried and tested the tools on refugee and migrant groups. We have also had support from Evelyn Oldfield Unit interns and volunteersSamantha Piergiovanni, Alessandra Scipioni, Klajdi Gjondedaj and Iman Nafi. Thanks all!

2. Introduction

2.1 An Overview
We as a sector are under pressure to provide evidence of the effect we are having. At the Unit, we are very aware that our groups are doing excellent work but there is a need for groups to better articulate and evidence this. Good monitoring and evaluating is key. You need a regular system where all your staff and key volunteers are involved. The information collected needs to be precise (in terms of monitoring) and preferably from a mixture of methods (in terms of evaluation). Yes, a good questionnaire is important but you need additional tools and systems to capture the in-depth details. Plus you need to do this over the longer-term, allow space for negative feedback, and collect good data at the start of any intervention. Theres a lot to be done. We hope this toolkit helps you to achieve this. There are two parts in this toolkit. The first is called a diagnostic toolkit. This means that we have designed an excel based tool to assess how effective you are at measuring your work. Please contact Sarah to ask for this. This document is the second and is designed to give you tools to better capture the effect of your work. If you would like support to assess where you are and to implement tools from this toolkit, please do get in touch. You can contact the in-house specialist on monitoring and evaluating to discuss more about.

Sarah Menzies Development Worker Tel. (+44) 0207 6974102 E-mail. [email protected]

2.2 Key Definitions

It is quite difficult to differentiate between outcomes and impact. Some funders see the terms are as the same thing. There is no single definition nor single tool that you can use to capture this elusive concept. The NCVO definition (http://www.ncvo-vol.org.uk/policyresearch-analysis/research/measuring-impact) seems to be one of the most accessible and practical; Impact is any change resulting from an activity, project, or organisation. It includes intended as well as unintended effects, negative as well as positive, and long-term as well as short-term. In this definition, outcomes and outputs are components of the much wider, all-encompassing definition of impact. Impact can be registered at the level of individuals, communities or wider society. However, this definition does not allow clarity between effect on the individual and effect beyond that and therefore, for the purposes of this toolkit, we would like to suggest that impact can be defined as the broader effects of an organisations work. An easy way to visualise this is as follows:

Key Terminology
121 advice on a range of issues, other language leaflets and publications, training courses, etc.

Less exclusion, less poverty, less unemployment, etc. Centres success leads to a number of other centres starting up, etc.

Need and demand

Resources, input

Process, activities (objectives)

Outputs and outcomes

Impact

Refugee advice centre with staff, volunteers and facilities for interviews and training

Outputs: number of users, phone calls, publications distributed, number of volunteers trained to give advice, etc. Outcomes: beneficiaries more skilled, volunteers use skills to get paid jobs, refugees supported to solve pressing immediate issues, etc.

Point to Consider - Negative Feedback Or Constructive Feedback :) You should also allow space for any negative feedback. Most groups shy away from this but you also need to know what is not going well in your work and if there are any negative outcomes or impacts. You can gather this information and think about is as constructive feedback. Remember and ask questions like - what could have been better? Is there anything you would like to change about our approach? Has our work effected you in any negative ways?

2.3 Sample End of Year Funders Requirements

It is typical for funders to ask you the following questions. This toolkit should help you to be in a stronger position to answer this.

1. Describe the work you have undertaken with our funding 2. Who and how many have benefited? 3. What difference has the grant made and how do you know? 4. What were your most significant achievements?

2.4 Effect on the Individual, Organisation and Wider Impact

Outcomes individuals and soft)
Health Behaviour Attitude & selfperception Political participation

in (hard

Outcomes for organisations

Impact

Organisational strength Service providers Reputation Sharing good practice

Environmental Economic Social (e.g. on crime rates, on cohesion) Cultural

Knowledge and skills

Increased capacity

The VCS

Circumstances Relationships

New funding Improved engagement with clients New connections better linked in

Policy Government

Networks

Health

Improved understanding of UK systems

Integration

Other areas where you may have an effect: Community Economy Environment Political Social Technology Personal development ..

10

2.5 Getting Started

In our experience, most of the groups we work with are very good at collecting basic monitoring and evaluating details and asking if their beneficiaries have enjoyed their services but this doesnt really capture the effect groups have had on someones life. You have to consider this over the longer-term. If you think about it, with a client who has been coming to you regularly for over a year - you will have had a real effect on their life and impact on their family life. If you think about the fact you have been helping hundreds of people of the same ethnicity over the year (and even over years) think about the profound impact you are having on the community. You need the right questions and tools to help you to gather this. Here are some tips and our favourite methods, tried and tested.

From: Charities Evaluation Services (CES) http:// www.cesvol.org.uk/toolsand-resources/ planning-formonitoringevaluation/cesplanning-triangle

1. Review the original aims and objectives of the project. If you have funding, you will have told the funder that you will do certain activities. Review this and work backwards. Bear in mind that your overall aim is what you hope to achieve and who with. For the Unit this is support to refugee and migrant groups across London. Specific aims - to support groups to better evidence the effect of their (good) work Objectives - by producing a very useful toolkit Once you have clarified these, you need to start thinking about the tools you will need to do this. Before you go any further... 2. Collect baseline information This means you need to assess where someone is at when you first start supporting them. Return to this after a specific time period - say 6 months and then again after a year and then when they leave. You can ask them how far they have progressed since them. There are lots of tools in the next section that should help you to capture that.

11

TOOLS

12

3.1 Sample monitoring form for an advice group

(thanks to Islington Somali Community for allowing us to reproduce this)

13

3.2 Equalities and Monitoring Form

This is the updated monitoring form that we use with the EOU.
MONITORING FORM We will be grateful if you could take a little time to complete and return this form. Please go through it and tick all the categories that most accurately describe you. The information you provide on this form will be held in the strictest confidence. We will collect, analyse and use this information to help us to see how our policies and activities are affecting various sections of our communities. In employment and service provision, it helps us to identify any existing inequalities and where new inequalities may be developing and take action to tackle them.

(A) Age Group 7 65+

1 4

Under 18 2 35-44 5 8 Prefer not to say

18-24 45-54

3 6

25-34 55-64

The Equality Act 2010 defines a person as disabled if they have a physical or mental impairment which has a substantial and long term (i.e. has lasted or expected to last at least 12 months) adverse effect on ones ability to carry out normal day to day activities. This definition includes such conditions as cancer, HIV, mental illness and learning disabilities.

(B)Disability

Do you consider yourself to have a disability? 1 Yes 2 No 3 Prefer not to say 1 2 3 5 6 7 8 Hearing (Deafness of partial loss of hearing) Learning Difficulty Mental Ill Health 4 Physical disability Sight (Blindness or partial loss of sight) Developmental disorder Other (please specify)_____________________ Prefer not to say

(C)Gender / Identity

1 3 4

Female

Male

Please tick if you also identify as transgender Prefer not to say

(D)Marital Status

1 4 7 8 9

Single 2 Married Civil partnership 5 Separated Widowed Other (please specify)_______________ Prefer not to say

3 6

Partner/co-habiting Divorced

Please turn over

14

(E)Sexual Orientation 1 3 5 6

Heterosexual 2 Bisexual Gay 4 Lesbian Other (please specify)_______________ Prefer not to say

(F) Employment status 1 3 4 5 7 9 10

Employed full-time 2 Employed part-time Unemployed short-term (<6 months) Unemployed long-term (>6 months) Homemaker/ carer 6 Student Unable to work 8 Retired Other (please specify) _____________________ Prefer not to say

(G) Immigration Status Refugees and Asylum Seekers Are you? 1 An asylum seeker 2 A Refugee 3 A Migrant 4. What country or region are you a refugee/asylum seeker/ migrant from? _______________ 5 Prefer not to say (H) Ethnicity 1. I would describe my ethnic origin as: I would describe my ethnic origin as: _______________ (E.g. White British, British African, Somali, Kurdish etc) 2. Prefer not to say

(I) Religion * Please tick as appropriate Christian Muslim Jewish Buddhist Hindu Sikh Rastafarian No Religion Other (please specify): Prefer not to say

The information provided will be stored in a database in accordance with the Data Protection Act 1998 and case study material will only be used in agreement with the client. Registered Charity No 1044681

15

3.3. Young Advocacy Project Distance Travelled Form (real group example)
First meeting I make decision on my own and I know what I want I do things on my own- for example I will register with a GP on my own I like the house where I live now My financial situation enables me to achieve my potential I have a dreams and plans for the future I say what I think and what bothers me I like living here I am proud of my culture and origin I have many friends from outside .................... community I feel welcomed and accepted in the U.K. People (doctors, Job Centre workers etc.) from outside know about ......................... community (please write in the breakers) your country) After six months

NB you have to take notes to accompany this to capture the details.

An alternative version of this is over the page (where the questions are broader)

16

3.4 Young Advocacy Project Example 2

Initial Assessment Name Organisation/Relationship

Welfare Housing Health Other Problems

Concerns Carer (if applicable) Name Relationship Contact Number

Comments:

Name: _________________________________________________________________________ First assessment date: ___________________________________________________________ Second assessment date: __________________________________________________________

You can then use this tool again after 6 months or so to see how far the beneficiary has progressed since this initial interview, making note of any changes as a result of your services.

17

Young Advocacy Project

Action Plan
(Agreed by the Young Advocate and the Service User)

CLIENTS NAME: _________________________________________________________________ DATE: __________________________________________________________________

Actions to be taken

Who is going to take this action

Young Advocate
Name: _________________________________________________ Signature: _______________________________________________ Date: __________________

Service User
Name: ___________________________________________________ Signature:________________________________________________ Date: __________________

18

3.5 Outcome Star

This is a tool as designed by Triangle Consultancy. There are a range of stars that can be downloaded for free. Each star comes with a pack. In this, you will find support in how to use the star, what each point means, an action plan and a follow up plan. Once you have set up an account, you can access many of the stars for free. You can also Please follow the link to see further examples - http:// www.outcomesstar.org.uk/ It is a very simple and effective tool for tracking where your beneficiary is (at the start of your project) and how they progress over the months. This star is a well-being star.

19

Here is a second example of the star for family life.

Thanks to Sara Burns from Triangle Consultancy for permission in reproducing these images

20

3.6 Key Questionnaire

There are long questionnaires and short; there are open-ended questions and closed; you can send via email (perhaps using Survey Monkey www.surveymonkey.com), by post or in person; you can choose a few of your participants or as many as you can; but in all of these cases, remember that reading through the responses and analysing the results takes time for you and for the person filling it in. Please also watch this space as the Unit runs many trainings throughout the year on this topic. You need to always bear in mind what your funders require from you ...What did you say you would achieve? Start with this and work backwards. E.g. In year 1, 100 young people will report feeling more positive about their future. Your questions must directly reflect this: Introduction The key questions needed in any evaluation form are: How have we supported you? To what effect? How satisfied have you been with our support? Is there anything we could improve in our project delivery? Is there anything else we can do to support your needs? How you ask these questions can vary but these are required questions. We will concentrate on the question: how have we effected you as it is the most important. Refer back to what you said you would achieve in your project and ensure the question allows this to be explored. Example 1: What effect has the support of X refugee organisation had on your life? * I feel less isolated * I have improved my skills/gained new skills * My wellbeing has improved * My understanding of UK school system improved * I have made new friends * My understanding of UK health system improved * I feel supported * My familys wellbeing has improved * My understanding of housing issues improved * Other (please specify) ________________________________________________

21

Example 2: How has our organisation effected you? Please tick only one answer per row.
Increased greatly My confidence My skills set My ability to find a job My income My family life Increased The same as before Decreased Decreased greatly No answer

There are many other questions you could ask on your effect. It depends on your project. Examples include: Cultural: Do you feel you understand UK culture now more? Do you feel more connected to your culture of origin as a result? Integration: Do you feel more settled into the UK as a result? Family life: Do you feel more positive about your home life now? Has your relationship with your family improved? Community: Do you feel more part of your community as a result of our input? Have you been participating in more local services? Environment: Do you feel that you are taking more green actions as a result of our input? Socialising: Have you made new friends as a result of our project? Do you feel more confident in accessing new social events? Do you feel less lonely now? Communication: Have you built your communication skills? Can you listen to people better now? Mental health: Do you have a more positive outlook now? Do you feel that your life has gotten better? Political participation: Are you more likely to vote now? Do you now understand how the UK political system works?

NB always allow space for unanticipated outcomes or impact. If you only list certain questions, you will only get answers to them. This is easy to overcome. You need to ask: Is there are any other ways that we have had an effect on your life or impact on your community?
22

3.7 Sample Post-Event Questionnaire

EVALUATION FORM PLEASE COMPLETE AND RETURN THIS FORM TODAY

The Evelyn Oldfield Unit aims to provide high quality training for refugee and migrant community organisations. In order to do this, we need your help in identifying topics and methods of delivery. This evaluation form is one of the ways by which you have your say in the kind of training the Unit will provide for you in the future. NAME OF SESSION: Date: Note: 1 = lowest / weakest score 5 = highest / strongest score

1. Overall assessment: To what extent did the training session meet its objectives? (Please circle)

2. To what extent did the training session meet your expectations? 1 2 3 4 5

3. What were the highlights of this training session for you? a) b)

4. Can you identify any ways in which you think the training session could be improved? a) b) 5. To what extent can you apply what you learned at this training session to the work you do in your organisation? (1 = very little; 5 = a lot) 1 2 3 4 5

How do you think you could apply what you learned today in your work?

23

6. Please comment on todays trainer (1 = very weak; 5 = excellent) Clarity of presentation Training / facilitation skills Interaction with participants Handouts 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5

7. Is there anything else you would like to tell us about todays training, about future training, or about the training provided by the Evelyn Oldfield Unit in general?

24

Expanding your Evaluation

If you want to think about the impact of your work, you have to move beyond only including the points of view of your direct service user. You might want to speak to their families or to professionals who might be involved in their lives. An example would be with running a supplementary school. You could ask their parents for feedback but also school teachers. They will potentially see many changes that would add weight to your evaluation .. Also, what about your volunteers? They will be learning many new skills and gaining confidence as a result of their time with you? What about agencies that know your work? They will have a point of view on how strong you are and what else you could do to better meet need the need of your beneficiaries ...the following two tools are tired and tested methods to collect this information ... All of this combined will help you to move towards assessing your impact ..

25

3.8 Volunteer Impact Interview/ Survey

Question 1: What are the best aspects of volunteering with X Refugee Community Organisation?

Question 2: Have your expectations been met?

Question 3: Can you give us some examples of what you have learnt from volunteering with X RCO?

Question 4: Can you give some examples of how volunteering with RCO has contributed to your personal development?

Question 5: Do you feel that you have learnt new skills useful for employment or future education as a result of your volunteering experience here? Please explain your answer.

Question 6: What have been the most interesting or rewarding experiences you had whilst volunteering here?

Please see overleaf for more questions.

26

Question 7: Were you satisfied with your supervision arrangements?

Question 8: What suggestions, changes or recommendations would you make to improve our volunteer programme?

Question 9: Is there anything else you would like to say about the volunteering youve been doing, or the effects it has had on you?

NB this is an open-ended series of questions. If you have a large number of volunteers, you may wish to give more tick box or closed box answers as openended questions require more time for analysis.

27

3.9 Advice Session Evaluation

(thanks to Islington Somali Community for allowing us to reproduce this)

ISC Links
Personal Details
Mr/Mrs/Miss/Ms/ Others Male Female Date of Birth:_____________ First Name: ____________________ Family Name/Surname _______________________Home Telephone ________________ Mobile Number______________________________ Address______________________________________________ Post Code____________ Email address_____________________________________________________________ First Language ______________________
Please tick the box below that best describes your current situation:

Education Employment Training

Fulltime Fulltime Fulltime

Part time Part time Part time _____________________________________

Other-Please specify
Issue of Concern Education Employment Training Housing Health Benefit Others

What was the outcome? ______________________________________________________________________________________ _____________________________________________________________________________________________ _____________________________________________________________________________________________ _______________________________________________________________

(Please attach further pages if necessary)

Agencies and organisations contacted Tel: ______________________ Case workers Name ________________________ Date_________________________

The information provided will be stored in a database in accordance with the Data Protection Act 1998 and case study material will only be used in agreement with the client
Registered Charity No 1027814

28

3.11 Focus Groups

Focus groups are a great way to collect the opinions of multiple people at once. The focus group (FG) is a form of group interview in which there are several participants, in addition to a facilitator/ moderator. There is an emphasis on a particular topic and the accent is on the interaction of the group and the joint construction of meaning. Participants are usually pre-screened and selected according to certain shared traits.

The facilitators role is important, as he or she moves the discussion on when appropriate and makes sure that the group stays on track. Reference: Bryman (2008) Social Research Methods, Oxford, Oxford University Press For more reading on focus groups, please follow this link: http://www.ces-vol.org.uk/ index.cfm?format=748 TIPS I like to use prompts, so writing the key question areas on coloured card, having 7 or 8 areas and the allowing the conversation to flow freely from there. E.g. Employability Aspiration Self confidence Job search skills Job-specific skills Social skills Family life Other effects

Examples of focus groups: The Olympics and local youth- part 1: http://www.youtube.com/ watch?v=17LsW0G3na0&feature=related Part 2: http://www.youtube.com/watch? v=DGJk6D3Tghw&feature=related

29

3.9 Case Studies

Trust for London has some interesting examples of case studies of individuals and organisations on its website. See http://www.trustforlondon.org.uk/casestudies/01-how-we-make-a-differnce/ for further details. A case study approach is an excellent way to highlight interesting examples of how you have supported individuals/ groups . This allows for a longer-term representation and more in-depth details. The case study must explain the Who, What, When, Where, Why, and How. In addition it must demonstrate the outcome and impact on service user and/or service development - the so what. Below is an adapted sample from the Unit demonstrating a good practice case study. NB details have been removed to ensure confidentiality. Example: We supported Somali group x in area y to develop a new project for youth in 2010. The area is the fourth poorest ward in the whole of London and young people have very low educational attainment and are exhibiting many anti-social behaviours, with high crime rates. We supported the group to run consultation with young people, working with the staff and volunteers to develop a project proposal and then agreeing on which funder would be most appropriate. Together we submitted 4 applications. Of these, 2 were successful and the group have been able to start up a two-year youth club. Already there have been over 100 youth accessing this project. The project workers have already reported that the young people seem more positive and initial evaluation with the parents indicates improved behaviour. In addition, the young people have reported that they have made new friends and have been enjoying the sports and cultural activities. We will support the group to track the progress of the youth throughout the programme.
I just feel so much better. There is no space for me at home and my family are always shouting. I love coming here

30

Extras

31

4.1 Involving Your Users in the Evaluation

It is very good practice to involve your users in carrying out the evaluation and trying to understand what the results actually mean for you and your organisation. We have done this many times in the past and found it has allowed our evaluation to be more tailored towards our users needs. Here are some ideas on how you could involve your users ... we have starred these according to how useful they are.

In the design *** By piloting any research tool on them* In understanding the results** In running the evaluation** On an evaluation steering group*** By presenting results to your members at your AGM*

32

4.2 Showcasing SROI

Social Return On Investment (SROI) measures change in ways that are
relevant to the people or organisations that experience or contribute to it. It tells the story of how change is being created by measuring social, environmental and economic outcomes and uses monetary values to represent them. This enables a ratio of benefits to costs to be calculated. For example, a ratio of 3:1 indicates that an investment of 1 delivers 3 of social value. SROI is about value, rather than money. Money is simply a common unit and as such is a useful and widely accepted way of conveying value. For more information, please see .http://www.philanthropycapital.org/download/default.aspx?id=1110 NB - The Evelyn Oldfield Unit will produce a paper on the potential benefits and challenges of SROI specifically for refugee and migrant community organisations very soon. Watch this space.

SROI is .. A framework for measuring social value, which expresses this value in financial terms. At its heart, SROI is a type of economic analysis rooted in cost-benefit analysis Based on stakeholder perspectives Guided by principles Summed up in the SROI ratio: for every pound spent, charity A creates however many Y pounds of social value (New Philanthropy Capital presentation)

33

4.3 Monitoring and Evaluating Procedures

It is very useful to set out what you plan to evaluate, when and by whom. We have done this with many groups attached to the Unit. Evaluation of this shows it is very useful for staff members to have this set out in such a clear and understandable way.. Here is an example of one. ________________________________________________________________
Resource Centre Monitoring and Evaluation Procedures Evaluation and Planning We record how many users we have and why people attend our centre and feedback from the users. This information is regularly collated and reviewed by both the staff team and the management committee with the aim of ensuring that: The Centre is meeting both its funding targets All the work of the Centre continues to further our aims Our services enable groups to organise more effectively in the community We are providing equal access to all services The organisation of the Centre remains efficient and cost-effective We identify the changing needs of groups and gaps in service provision Our analysis of the above information is put together with external information such new government policy, local Council strategies and research reports. All this material forms the basis for the development of services and funding plans in succeeding years. The draft report is presented at our AGM and disseminated through our website and through mail outs to stakeholders such as local councillors, MPs, local infrastructure support organisations and the centres member -groups in order to give the wider community the opportunity to comment. Monitoring the take-up of services We record the name of each group which visits the centre and its use of each service. This data is collated weekly and turned into a monthly summary which is evaluated at staff meetings. A full analysis of the data is given in half-year and all-year reports, with a comparison done of previous years. These are discussed by both staff and the management committee to ensure that the centre is meeting its targets. More of this can be found by clicking on this link:http://www.resourcecentre.org.uk/

i n f o r ma t i o n / p o l i c i e s _ p r o c e d u r e s / s a mp l e _ p o l i c i e s / s a mp l e _ p o l i c i e s _ p d f / rc_monitoring_evaluation.pdf 34

4.4 Monitoring and Evaluating Plan

Another way to set out what you need to do and by when is to use the Charity Evaluation Services matrix as below. You start by setting out your projects aims and objectives and then based on this, consider your over-arching question. From this you further break this down into a set of indicators, i.e. A means of assessing how you are achieving your aims but in a more manageable way. Where you get this information from and how you will collect it comes next. A timetable is also a useful column.

Project Aims and Objectives Young Afghans will learn mother tongue and learn about Afghan culture and heritage

Key Question

Indicators

Data Sources

Who is accessing this service? To what effect?

*summary of course content *numbers attending school *number of children improving their language skills *number of children reporting an increase in understanding of Afghan culture

Photographs Films of events Examples of projects undertaken Pre-and post course assessment Feedback form children and parents Staff observation

Data Collection Methods Monitoring of who is accessing the service Questionnaires Interviews Recording events Case studies

Timetable

Continuous feedback 6-monthly survey 6-monthly interviews

Quarterly reports from staff on observation notes

35

4.5 Other Stakeholders Interviews/Survey

It is very worthwhile to include the opinion of your wider stakeholders in any evaluation that you run. This allows you to assess your wider impact..This could include: Potential users Staff and volunteers Board members Community members Other charities/ voluntary and community groups Professionals (police, teachers, doctors etc) Funders Local councillors And any others who have an informed point of view about your work Here are some very simple survey questions that we have used to interview other stakeholders in the past that has proven very useful in terms of assessing the impact of an organisation on the community. In this instance, we interviewed 8 professionals who knew the organisation well.

XX Group Evaluation, June 11

1. Can you describe your relationship with XX group (i.e. how you have worked with the group)? __________________________________________________________________________________ __________________________________________________________________________________

2. Can you describe some of the strengths of XX group? __________________________________________________________________________________ __________________________________________________________________________________

2b. How have these strengths have benefitted your organisation? __________________________________________________________________________________ __________________________________________________________________________________
Please see overleaf for additional questions.

36

3. What effect do you think that XX group has had on the community in XY borough? ______________________________________________________________________________________ ______________________________________________________________________________________

4. Do you feel there are any other unmet needs that XX group could tackle? If so, please expand on your answer. ______________________________________________________________________________________ ______________________________________________________________________________________

5. What could XX group do better? ______________________________________________________________________________________ ______________________________________________________________________________________

6. Do you have any other comments about the effect of XX RCO? ______________________________________________________________________________________ ______________________________________________________________________________________

Example Mapping Exercise by the Unit to Identify Study Participants

37

4.6 Participatory and Alternative Methods

There are many very positive ways to involve your users in the evaluation process. If your users influence this, it is called participatory. The picture above is a participatory circle whereby I asked attendees at my training to decide upon the categories of the evaluation and then to decide on how they would approach it, agreeing on categories themselves, without the tutors input. They then decide what is important to them. The image above was the result.

Creative methods Evaluation does not need to be all about words. Often photos can tell a very strong story. As can audio, video etc. There are countless creative methods you can use for your evaluation. See Trust for Londons case studies for some ideas - http:// www.trustforlondon.org.uk/case-studies/02-case-studies-of-londoners-inpoverty/

There are many, many other creative approaches to evaluation. Please see this link for other ideas: http://www.ces-vol.org.uk/index.cfm?format=748

38

Post-data collection

39

5.1 Analysis
Survey Monkey is an online resource that is very easy way to analyse and manage the data that you collect. We would highly recommend that you explore this free software www.surveymonkey.com. You can also use this for free online surveys.

Designing the Questions

Here are some examples of how we have used it for previous consultations:

2. Collecting the Results

40

5.2 Whats Next

Once you have gathered all of this information ,it is very important that you dont just use it to write a report for your funder. You really need to think about what the results mean and ask yourself critical questions such as is this project working? What changes could we make to ensure we are meeting our users needs ? Make changes according to this and most importantly, learn...

I have always grown from my problems and challenges, from the things that don't work out, that's when I've really learned... Carol Burnet [http://favoriteinspirationalquotes.com/thats-when-ive-really-learned ]

What Next?

Reflect

Learn

Implement

41

5.3 End of Year Report Example

Please find a sample end of year report from here http://www.gbtc.org.uk/documents/ CC16SampleTrusteesAnnualReport.pdf And also from the EOUs website here http://www.evelynoldfield.co.uk/files/publications/ EOU%20EOY%20Report%202009_2010%20Final.pdf

42

5.4 Challenges

Bias: often those conducting the research will also be reporting back to funders. You have to be careful of only looking for positive aspects of your work. This can be overcome by having lots of people involved in analysing the data

Defining success criteria can be a challenge how do you define when your project is working well?

It is difficult to capture unanticipated outcomes and impacts.

It is hard to attribute effect to one organisation when there are multiple influences on your clients.

Be careful about setting too high targets in first place.

43

5.5 Sources of Further Advice and Support

The Association of research in the Voluntary and Community Sectorwww.arvac.org.uk Charity Evaluation Services (CES) Free Downloadshttp:// www.ces-vol.org.uk/index.cfm?pg=112 Including Becoming more effective a monitoring and evaluation guide for refugee community organisationshttp://www.cesvol.org.uk/Publications-Research/publications-free-downloads/ becoming-more-effective Jargonbusterhttp://www.ces-vol.org.uk/index.cfm? format=171 NCVOhttp://www.ncvo-vol.org.uk/strategy-impact/learn/ impact/measuring-impact/how-to-monitor-and-evaluate The STAR approachhttp://www.outcomesstar.org.uk/ KnowHow Non-Profit - http://www.knowhownonprofit.org/ organisation/quality/mande/monitoring-and-evaluation-the-

44

Contact Details
Sarah Menzies The Evelyn Oldfield Unit 356 Holloway Road London N7 6PA 0207 697 4100 [email protected] www.evelynoldfield.co.uk

Evelyn Oldfield

45