Blog Post

The purpose of impact data collection: thinking through what you need and why

Becky Nixon • 16 November 2020

I've had similar conversations with different managers recently about what data they collect and why. Different purposes call for different types of information, so it is helpful to think this through - most of us can probably think about data that we have collected but that we haven't done anything with.

A future post will look at how you can compose the content of surveys, distance-travelled monitoring and other frameworks, this one starts with some basics to lead you through a process of considering:
  • Why you need data and what are the key messages that you want your data to convey
  • What type of data you need
  • What you will do with it
As you do each exercise you may want to go back to a previous one to revise it with further thinking.

Contact us using the details below if you would like a Word version of the pro formas.

Why do you need data? Who are your audiences and key messages?

Start with considering key messages for each of your audiences, and the "call to action" - what do you want people to do? You can use a table like the one below, replacing the list in the left hand column with your own, segmented to your particular organisation.

Your data will need to support the key messages that you've identified above, for example if you want to try to encourage people to get support around managing on a low income, include case studies of activities that you've run, quotes from people who have been helped, or statistics about how many people said that your service has helped them to manage better.

What can you measure?

There are a range of things that you can measure, all of which have different purposes:

  • Inputs – anything put into the service, e.g. staff or volunteer hours, money, partner time or resources, equipment.
  • User data – information on who your clients are such as demographics and where they live.
  • Outputs – number of sessions / activities / relationships.
  • Outcomes – what has changed for people using the service, the community, partners, or the environment etc. (these are usually related specifically to the organisation’s aims).
  • Impact – the wider/longer term difference to people, the community, partners, the environment etc.as a result of your work.
  • User feedback – what people think of the service.

What type of data do you need for different projects and services

You'll need different types of data for different purposes - common purposes people collect data are listed, let me know if you can think of any more:
  • Strategic planning, monitoring and reviewing
  • Learning and reflection and service development
  • Individual work planning and performance monitoring (mainly staff but may include volunteers in some roles but be careful around expectations around employment law)
  • Contract or grant monitoring
  • Promoting the organisation to a range of stakeholders
  • Income generation including for funding applications and tenders.
A form like the one below can be used, with the name of projects, services or activities across the top, as different types of data may be useful for different parts of your organisation.

The graphic below show types of data collection and in the management structures box where these might be discussed.

How will you communicate impact?

The next thing to think about is how you will communicate your impact with the data collected; which of the following methods will you use, are the different vehicles aimed at any audiences in particular, and what type of data is needed (identify this in the notes column)?

It is often good to think about a mix of quantitative data (numbers, e.g. 68% of participants reported feeling more positive about the future) and qualitative (descriptive, e.g. case studies, quotes, video stories, observations) but different methods or "vehicles" will lend themselves to different types of data.

What next?

You should then be clearer about:
  • What are the key audiences for your data and what messages do you want to give
  • What the different purposes are for collecting data and what type of data is needed
  • The methods that you will use to communicate impact through your data.

Following this you can start to think about the content of the data you collect, e.g. what questions or statements should be in a survey or distance travelled tool or what topics do you want to focus on in your case studies?

How can Ideas to Impact help?

We can:
  • Work with your organisation to develop an impact measurement framework
  • Undertake an impact assessment or evaluation
  • Provide information, advice, coaching and mentoring around impact measurement, including through a " power hour " of one-off or ongoing advice - email Marlen Tallett, admin@ideastoimpact.co.uk to schedule a session.

Red and white toadstool
by Becky Nixon 22 May 2024
People sometimes struggle to understand what a trauma-informed approach is. This is in part because although there are frameworks and tools to help to implement trauma-informed practices, the core of it is really a shift in thinking that once you've "got" (although it's also an ongoing learning process) you can start to apply to any situation. This article is partly an example of how to think about management in a different way that is more sympathetic to people, takes a systems approach rather than putting the problem with one person, and takes into account how trauma, adversity or discrmination may affect a situation and how it's viewed. I’ve been meaning to write something about the framing of “toxic” workers since I read this article , which the date tells me was nearly a year ago now. The article is in People Management by the Chartered Institute for Personnel and Development (CIPD). This is just one example, there are many more, but this is framing that, in my view, is likely to cause more problems than it solves. Everyone can probably think of a colleague who might have been described as "toxic", but I have a strong reaction to the labeling of human beings like this, particularly by a worker in relation to other people who have been selected and welcomed into their organisation. The use of such a strongly negative term can feel dehumanising and dismissive of the complex factors that contribute to an individual's behavior in the workplace. It is also counter to the trauma-informed philosophy of looking at what happened to people that explains their behaviour rather than what's wrong with them. What’s the problem with the "toxic" label? Labelling a worker as "toxic" fails to recognise that people are individuals with strengths and weaknesses and a lifetime of experience that has shaped who they are, including processes in the organisation or previous work roles, and the person's life experiences such as childhood trauma, adversity or exclusion. It minimises the role of workplace processes in influencing worker attitudes and performance and the impact of the type of work that people often do in the voluntary and public sectors with dangers of burnout, vicarious trauma, moral injury and compassion fatigue. This framing can lead to a blame-oriented mindset that further alienates the worker and hinders progress, and may have an impact on the culture in the wider team. Assessing workplace processes and worker needs Every person in the organisation is part of a system, and relational issues are rarely "caused" by just one person. Worker unhappiness or difficult behaviour may stem in part from inadequate workplace processes or poorly supported managers. The same publication reports that although the majority of people say they have a good manager, 53% of those wanting to change roles were doing so because of their manager. Instead of labelling a worker as "toxic," it’s important to ensure that people have the necessary resources and support to perform successfully. There are evidenced and established frameworks about what contributes to worker stress, burnout and vicarious trauma or what correlates to greater worker performance, satisfaction and retention. These can be used to assess workers’ needs, actively seek their feedback, and engage in structured and open communication. Rather than viewing the worker's behaviour as resistance, it can be seen as valuable data that highlights potential areas for improvement within the organisation. When workers express unhappiness, it presents an opportunity to actively engage with their concerns, which may be shared by other team members who aren’t so confident in speaking up. These are workers that can prevent avoidable mistakes and failure. By structuring meetings in a way that encourages constructive criticism, for example allowing workers to formally play the role of devil's advocate, organisations can gain valuable insights into potential issues and foster a culture of continual learning and improvement. Furthermore, this is a person that your organisation has recruited and inducted, and possibly passed probation, so it’s worth considering whether something has gone wrong at this stage as well as recognising that you have a contract with them that includes supporting them and dealing with any problems. The contribution of psychologically and trauma-informed approaches An individual's behaviour is often conditioned by their experiences of adversity and trauma or other family dynamics This may be especially the case for people in “helping” roles who are more likely to have experienced adversity . It can be helpful to start from a position that people are trying their best and of understanding that people’s behaviour makes sense to them based on the context that they are in although you – and they – may not understand where this comes from. Some examples of how family dynamics influence how people are at work are explored in this article as illustration. While there are limits as to what issues can be addressed within the workplace, adopting psychologically and trauma-informed management can create a more supportive environment. The trauma-informed principles I use based on research are: 1. restoring power, choice, and control; 2. creating safety; 3. building strengths and self-worth; 4. recognising identity and context; 5. facilitating connection; and 6. supporting coping mechanisms. Each of these recognises and supports the wider psychological and emotional needs that all workers come with in a way that is appropriate for the workplace but also recognises boundaries - this also comes with a warning that managers need to recognise their own limitations and avoid initiating discussions they aren’t competent to handle, focusing on maintaining appropriate boundaries and directing workers to seek professional support when needed. Engaging with worker concerns and addressing inclusion A further factor to consider is whether issues of inclusion and belonging are contributing to worker unhappiness. Fifteen percent of workers have experienced bullying , but there can also be lower-level issues of exclusion or "othering" that lead to individuals feeling left out or overlooked that can create what appears to be hostile behaviour. The picture of how diversity in teams contributes to performance is complex and not a straightforward correlation - probably because it needs skillful management and psychological safety to take advantage of the benefits that it brings, it doesn’t happen automatically. When everything else has failed After exhausting other avenues, in some cases being in the organisation is not the best thing for the person, for the organisation or for its beneficiaries and other stakeholders, and a performance improvement plan may ultimately be necessary. The actions of one person can have a big effect on a whole team - one of the factors influencing an individual's performance and engagement is their perception of their colleagues' commitment to quality work – but this can be a complex mix, workers who exhibit behaviour that is deemed "toxic" can be highly skilled and dedicated to their roles with their frustration stemming from feeling unable to perform to their full potential due to workplace structures or limitations. Organisations need to ensure that they have clearly set out values that preferably workers have contributed to developing, and that it is clear that their performance will be assessed against these and not just task-related outcomes. How Ideas to Impact can help Ideas to Impact's training, consultancy and mentoring considers some of these wider issues in relation to how to pinpoint and address problems. Training includes: A half-day introductory workshop to trauma-informed practices in general, appropriate for people working in any role in the organisation. From stress to strength: managing a psychologically and trauma-informed team - focusing on psychological insights and how trauma-informed approaches can apply to the workforce. Building strong foundations: psychologically and trauma-informed systems and culture - focusing on organisational systems and processes for managers, leaders and anyone else in organisations with a role in determining these. Courses includes a set of frameworks and tools that can be used on an ongoing basis as well as considering wider psychological and group dynamics factors. Training can be provided within individual organisations or to groups of organisations (we have found that this is a good way to make links between organisations that can have wider benefits for the work as well as sharing good practice) and delivered as whole day sessions or broken down into workshops. We also know that whilst people find one off training useful, it can be difficult to put it into practice in the workplace, so individual consultancy and mentoring is also available.  Have a look at our services page or get in touch directly at becky@ideastoimpact.co.uk
by Becky Nixon 13 February 2024
New research identifying the need for an organisational approach to resilience and wellbeing
by Becky Nixon 8 February 2024
New research into loneliness and older men and its implications for services
CALS logo
by Becky Nixon 24 November 2023
Putting psychologically and trauma-informed management training into practice in the workplace
by Becky Nixon 16 October 2023
Supporting workers, ameliorating stress, and providing a more trauma-informed approach - a programme run by Leicester women's organisation New Dawn New Day , and Ideas to Impact .
by Becky NIxon 28 September 2023
Feedback from participants about how they do or could implement actions towards trauma-informed working
by Becky Nixon 18 August 2023
Why and how to use resources and evidence from the Office of National Statistics and other verified sources to improve your impact measurement and needs analysis, including a download of suggested indicators
by Becky Nixon 15 April 2021
With a lot of people in flux at the moment with working patterns, with workplaces starting to open up and people going back to working elsewhere, I was reflecting on my early morning walk on what Gretchen Rubin calls the strategy of the clean slate to make habit change. For me, the change is not my own, but my daughter's. She has recently started an apprenticeship at a medical centre in a town around 12 miles away. In the morning she gets on the bus at 7.45am, so I have got into the habit of leaving the house with her and having an early morning walk, usually along the canal above, before settling to work. For some reason, the last bus back leaves at 3.25, before the schools and work has actually finished for most people (and then they wonder why it's not used much!) so I pick her up every night. Her work is closer to my gym than where we live, and now the pool is open again it motivates me to finish work an hour earlier and to have a swim before I get her as I'm driving halfway there already. Hopefully my daughter will soon be driving herself now lessons and tests have started again, so I was wondering whether I would keep these activities up when our routine changes again. This is where the power of habit takes over. Habitual behaviour happens in a different part of our brain from conscious thought ( this blog I wrote earlier talks a bit about this) and can easily over-ride other intentions. Habit change is about reprogramming our neural pathways until we do that activity "without thinking". If we repeat an action enough times, then our brains start to associate one thing with another in an "if...then" pattern, e.g. if my daughter is leaving the house to get her bus then it's time for me to go for a walk. Think of this rewiring as a route across a park - as more and more people walk along new route the grass gets worn away and the pathway gets more established, like these . Hopefully by the time my daughter passes her test and I wave her off on her own, my morning walk routine will be so established that I will carry on doing it, same for the early evening swim. It's difficult to say how long it will take any individual to form a habit, a study by Philippa Lally and colleagues at UCL identifies an average of 66 days, but so many factors come into play that it can vary, for example some of these could be how committed are you to the habit, how convenient/inconvenient it is, what social reinforcement you have for doing/not doing it. With some provisos Lally's article identified a range of 18-254 days. Action point When you are facing a change in your life such as changing where or how you work as you may be doing at the moment, think about whether this could help you to make or break any habits. If the strategy of the clean slate doesn't work for you, Gretchen Rubin in her book Better than Before has many other practical strategies that you can try. Using psychology for individual and organisational development I am working with colleagues to develop a series of courses, workshops and resources around using psychology in the VCSE to improve individuals, organisations and services, for example around areas of motivation and engagement; personality in the work place; working effectively including looking at habits, understanding yourself and avoiding procrastination; and the effects of trauma on the brain. If you are interested in any of these, please sign up below for my newsletter where they will be announced later in the year or contact admin@ideastoimpact.co.uk
by Becky Nixon 23 March 2021
Background The Investing in Volunteers standard was last reviewed in 2010, so it was due for a refresh in line with changes in volunteering and the wider world. Ideas to Impact won the tender, and working with colleagues Janet Lewis-Jones (IiV lead assessor) and Ann Gilbert (formerly chief executive of Northampton Volunteering Centre) and assistant Marlen Tallett, we undertook desk research and a consultation to refresh the standard, working closely with Jo-Ann Maycock, Trish Kiss, and Adam Fox at NCVO and liaising with leads from the other UK organisations, WCVA , Volunteer Scotland , and Volunteer Now . The project was overseen and the final version agreed by the UK Volunteering Forum with representatives from each of the four countries. Aims of the refresh included: Streamlining the standard and updating practice and language in line with current volunteering good practice and wider developments in volunteer-involving organisations. Ensuring that it is relevant to a wide range of volunteer-involving organisations, including specific guidance for different types and sizes of organisation and different types of volunteering opportunities. Focusing more on the volunteers' experiences, including incorporating the findings from NCVO's research with over 10,000 volunteers Time Well Spent . We engaged with around 200 people through an online survey, online workshops, individual telephone calls, attendance at the National Volunteering Forum, individual email conversations, and three workshops held by country leads in Scotland and Northern Ireland. Once we had a draft standard, 24 organisations filled in a self-assessment against it, we undertook 17 volunteer and trustee mock interviews, and a focus group of different types of organisations reviewed the materials. Comments that we received on the new standard included, “I really like the way the standard is now more readable, and easier to use as a tool to explain to others not so knowledgeable about volunteering what underpins a good quality volunteer involvement process." “I think it hits the big picture elements for diversity and inclusion and reminds people that we need to be pro-active in reaching out to encourage people to get involved." “I think it's great to see people as the first point in the explanation of the quality area. Volunteering is about people first and foremost." Main changes in the 2021 standard There are now six quality areas instead of nine: Vision for volunteering Planning for volunteers Volunteer inclusion Recruiting and welcoming volunteers Supporting volunteers Developing volunteers Some of the main changes include: A greater emphasis on volunteering being part of the wider organisational strategy and a requirement that the impact of volunteering is understood and communicated - impact assessment has changed a lot in the 10 years since the standard was last reviewed. This is crucial so that organisations value their volunteers, but also so that volunteers can see that their work is important and makes a difference - feeling meaning and purpose is key to volunteer engagement and satisfaction. This includes ensuring that any volunteer managers are appropriately integrated into the organisational structure so that there is an ongoing mechanism for communication across the organisation. Detail relating to specific processes removed to guidance, with indicators focusing more on outcomes and the bigger picture . Included in this category of changes are: thinking through the purpose of holding information about volunteers (not just data protection), having a positive and managed exit process (not just collecting feedback from volunteers who are leaving) and supporting volunteers' future aspirations (not just providing references). Following feedback in Time Well Spent that many volunteers found volunteering overly bureaucratic we've also included making sure that systems and processes are proportionate and volunteers know why they are in place. "Volunteers feel..." statements that focus on how volunteers experience the organisation, for example volunteers feel supported rather than the organisation provides support. Other "volunteers feel" statements include that there is good communication, their contribution is meaningful and rewarding and they feel valued and a part of the organisation. Some consultation respondents were concerned about whether these would be difficult to assess, but when we interviewed volunteers we found this was the sort of information they naturally provided, often without us actually asking specifically! From the perspective of organisations, finding out whether volunteers do feel these things is part of day-to-day activities anyway through formal and informal processes, and is important in ensuring that volunteers have a quality experience and remain volunteering. We also broadened out the scope in the indicators relating to equity, diversity and inclusion , relating what happens with volunteer inclusion to the aims of the organisation as a whole, and ensuring that the whole organisation is welcoming to a wide range of volunteers. Ensuring that organisations are proactive in increasing volunteer diversity and tackling under-representation was also strengthened, along with further guidance for organisations that work with a specific section of the population, for example particular health conditions, or select volunteers from members or service users. In quality areas five and six we used our experience of assessing IiV to make strengthen areas where we'd heard frequent comments from volunteers about their experiences but that were not explicit enough in the previous version. This included making sure that volunteers were able to discuss how they are doing - not all volunteers want feedback on their "performance" but they do generally want to know what has happened as a result of their volunteering. The indicator on volunteers being able to learn and develop was changed to include organisations recognising the skills and experience that volunteers bring with them where appropriate, as it's not infrequent that volunteers say they have skills that could be put to use that organisations are unaware of. Also in this section we emphasised that volunteers often say they would like more contact with other volunteers - we found research that indicated that peer support from other volunteers was more highly correlated with volunteer engagement and satisfaction than support from paid volunteer managers. The new version of the standard will be available on the Investing in Volunteers website following the launch on 24 March 2021 About Ideas to Impact Ideas to Impact provides a range of services to organisations and individuals including relating to volunteer management and developing and assessing quality standards. This includes consultancy, research, workshops and one-to-one support in coaching or through a one-off Power Hour where we can talk you through any issues you have with volunteering and what we know from research and evidence and our experience of talking to hundreds of volunteers. Our news and blog gives more information about different aspects of our work. This includes courses that fit with Investing in Volunteers, including Encouraging Inclusion , Volunteers' Voices , and How to have happy and engaged volunteers , which also incorporates psychological research around engagement, motivation and wellbeing. Please email becky@ideastoimpact.co.uk if you would like to be added to the list to be informed when open versions of these courses will be run, or sign up to the newsletter below.
by Becky Nixon 3 January 2021
If you're anything like me, your heart might be sinking at all the things that you were overwhelmed by at the end of last year that went onto the "I'll think about it in the new year" list, because that time has now come and things possibly don't feel any easier. This post describes one mechanism to identify and prioritise the things that you need to do - this activity it is aimed at individual workers rather than whole organisations. Take some Post-It notes Or similar sized pieces of paper (I once worked in an organisation that didn't allow us to have Post-It notes - or decent coffee - in case funders visited and thought that we were frittering away their money on luxury items). On the Post-Its write down all the things that are on your mind that need doing - one per Post-It so that you can move them around to rearrange them. These may range from small to large items, e.g. send out agenda for team meeting or prepare forms for annual development reviews to finalise and submit tender for main services or develop new strategic plan. Once you have completed this, for each item put a score out of ten where ten is high and zero is low: (a) in the top left corner for what impact this activity will make to progressing you towards your organisational mission if you achieve it (for people familiar with the urgent - important matrix, this is the importance element); (b) in the top right hand corner for how urgent the action is; and (c) in the bottom left hand corner for how much mental or emotional energy this is taking up - this isn't a traditional way to plan and prioritise, but since your wellbeing and mental and emotional state is crucial to how well you perform, it is critical to consider this. Do not use a 7 for any of the scores. I heard this on a podcast but can no longer remember who suggested this, but the idea was that a seven is a cop out and it's easy to end up with lots of sevens with no way of differentiating between them - make them a 6 or an 8. The next bit is more of an art than a science (sorry to anyone who was hoping to add up the scores and get a neat formula to make the decisions for them!) Make sure you have a system for recording tasks now and into the future. I use ToDoist - there are other similar apps or you can just use a diary and a notebook. This gives me the ability to sort by projects (which I review weekly) and by discrete tasks to do on specific days. It gives me a list of the activities that I need to do in the shorter term, and the ability to note things that need doing in the future - I may not have dealt with the actual task, but at least I know when I need to start thinking about it so it's not cluttering up my brain. (I also have a separate "sometime maybe" list of things that I want do but aren't time critical to stop cluttering up my daily to do list.) First try to get rid of anything that doesn't need to be done or can be done some other way. For anything that has a low impact score - ask why you are doing this at all. If it does need to be done, how can you minimise it, automate it, or delegate it? One issue I often see in not-for-profit organisations is lack of admin support for managers or team leaders - there's a pressure to reduce core/administrative costs, but ultimately this is generally not effective or efficient. If it's not appropriate to employ someone, think about a freelance virtual assistant who might help you a few hours a week or month. In terms of delegation, you don't want to dump things on other people who are over-loaded themselves, but it's worth considering delegating authority and decision-making to empower workers to deal with more activities which then don't need to come back to you - this may help to make everyone's lives easier. Identify anything that can be done quickly (e.g. in about 10 or 15 minutes), start by clustering these Post-Its in preparation to them in - most of these are probably not going to be things that need a lot of brain power, so batch it with other similar tasks and put it into a time in your day that's not your best thinking / quality work time. (However, recognise if you're someone who needs to get some of these things out of the way before you settle on bigger pieces of work because otherwise they are sitting there bugging you! There is no one-size-fits all approach to effectiveness.) For anything that has a high impact score - this is a priority and needs to be in your (or someone else's) diary. Separate out the larger and more significant pieces of work. These are often the things that make most difference, yet don't get enough time allocated to them. How are you going to make time for them? Who else needs to be involved? Where can you get help from? When do you need to start thinking about them to give you the time you need to work on them? See organisational psychologist Adam Grant's TED Talk about scheduling work for optimal creativity. For anything that has a high mental and emotional energy score - the things that are bothering you - ask yourself why to think about what you need to do about it. Is the problem that you don't know what to do but haven't put aside the time to think about it in more detail? Do you not have the knowledge or skills or time to do it? Does it involve difficult conversations or relationships? Is there a particular emotion you are feeling about it? Procrastination is often about avoiding negative emotions, sometimes it's helpful to just recognise this - read this New York Times article around the work of Dr Fuschia Sirois. Otherwise you might just want to "Eat the frog" and get on and do them, so they are not continuing to have a negative impact on you. (Mark Twain possibly said something like "Eat a live frog first thing in the morning and nothing worse will happen to you the rest of the day".) Go through each of the Post-It notes and put an activity and timescale into whatever system you have. Hopefully this has helped you feel more in control and either demonstrated that you are able to manage the work you have and to get it out of your overloaded brain, or it's demonstrated that there are more fundamental issues requiring a revision of your wider organisational strategy or structure to ensure that your organisation's work is more realistic. Need some help? Ideas to Impact can help you work through planning and prioritisation through Power Hours - a customised mix of coaching, mentoring and/or consultancy depending on your needs, or through workshops and facilitation for your organisation or through various workshops and training sessions . Contact Marlen Tallet, admin@ideastoimpact.co.uk if you would like to book a specific session, or Becky Nixon becky@ideastoimpact.co.uk for a more general discussion about how we can help you.
Show More
Share by: