Stakeholder engagement

The usefulness and reliability of our impact measurement is dependent on the quality of the stakeholder engagement it draws on. The purpose of stakeholder engagement is to help identify the most important outcomes to the community, the organisation and its partners; and to set out an understanding of those outcomes that has been informed by stakeholders.

Theory of change mapping

This approach maps out in theory how the organisation is achieving social impact. The maps were based on conversations with staff, project partners, funders and people who attended our activities. The aim of the conversations was to explore individual experiences of what changed (if anything) as a result of engaging with Manor House Development Trust and the importance of this change to them.

Our Theory of Change Map

 

Defining indicators to measure change

This is challenging for outcomes which are hard to define, for example someone reporting they feel more empowered or happier. Our approach to tackle this issue is:

  • Carrying out focus groups to ask people what change means to them. For example, if someone said they feel more confident as a result of a project, they were asked what has tangibly changed for them as a result. This approach allowed us to define outcomes in a way that has meaning for our beneficiaries.
  • Researching how other organisations measure change like empowerment and wellbeing. This allowed us to use UK-comparable indicators used by HACT, New Economics Foundation and others.
  • Articulating how the organisation defines change. Through the mapping exercise, we were able to create a Terms of Reference which describes time-based and measurable changes we are aiming for to achieve the Vision.

Methods of data collection

Survey

We developed a survey based on the Theory of Change map, where questions were framed to test the theories on how we think we are achieving social impact. The survey questions were designed to isolate changes that beneficiaries experienced that they believed to be specifically due to the Trust’s activities. The survey collected information about people’s opinions and attitudes in a quantifiable way by using scaled questions. The survey was sent to approximately 1,000 people who had previously attended at least one Manor House Development Trust activity. We received 99 responses. The margin of error (sample error) is calculated to be:

  • With a sample size of 10% this would create a 95% confidence interval, which is reasonable and provides confidence that the sample of respondents will roughly reflect the general population of the Trust’s attendees

Focus groups

Interviews with 6 focus groups with an average of 8 people per group were carried out. Each group had attended a different activity, in order to record the range of experiences felt depending on the type of activity. The responses of the focus groups were used to map out the outcomes which were experienced as a result of each activity. Interviews with funders and partners: Representatives from a number of the Trust’s partners and funders were interviewed to record the range of experiences felt from working with the organisation.

Project examples

For the purpose of this investigation, a number of Manor House Development Trust’s projects have been evaluated on their ability to achieve our Business Outcomes. Projects have been evaluated based on conversations with attendees; monitoring information completed by attendees; and results from the survey.

Case studies

8 case studies have been analysed to investigate individual experiences of some of the main projects.

Impact Resources & Guidance

Download the presentations from Manor House Development Trust’s Impact Masterclass.

Social Impact Breakout session  SROI Breakout session