Unit 1: Get started with impact measurement

[Alicia_Schmidt]

Impact measurement is a journey

Remember, impact measurement is a journey. You don’t need to do it all tomorrow. In fact, companies that start small and prioritize are often most successful. You likely are already reporting outputs for most, if not all of your programs. To start to measure impact and outcomes, list the potential programs and their priority and feasibility. Choose 1  program to start with and create a rough plan for when the others will be onboarded. 

Ready to embark on your own impact measurement journey?  


Step 1 - Define what is “in scope.”

Which programs are you interested in measuring? Don’t feel pressure to measure everything right away. Prioritize! Pick 1 or 2 programs or aspects and get comfortable with them first.  

 

Step 2  - Get explicit about your impact goals for those programs.

What do they aim to achieve and for whom? 

 

Step 3 - Create your framework.

Build your logic model or theory of change. This will  help you prioritize your purpose and values, and keep your measurement and  reporting organized. 

Step 4 - Choose metrics.

Pick metrics to measure your framework – Don’t feel pressure to measure  everything on day 1. Pick a small number of metrics that are meaningful and  attainable to build your processes.

The goal when choosing good metrics is to pick data points that will enable you to understand what needs to be done, make decisions and talk about your work. The framework below suggests that good metrics are actionable, accessible, easy to calculate and possess a common interpretation.

Lead with Logistics & Potency

When determining metrics, the goal is that they will be useful and practical to collect on a regular basis. It doesn’t matter how brilliant your metric is – if it is too burdensome to get, or difficult to explain, it won’t get you far.

  

 

Pick Metrics That Matter

You also want metrics that matter – to connect to your goals and give meaningful and actionable  information about what you are trying to achieve. A Results-Based Accountability Framework offers  another approach for selecting metrics (or Performance Measures). Like the framework from Gemignani  & Gemignani (on page 13), Performance Measures aim to make changes to programs to see if changes  yield the desired results.

All measurements fall into one of three categories:

• How much are we doing? 
• How well are we doing it? 
• And is anyone better off?
  

Results-Based Accountability Framework

	Quantity	Quality
Effort	How much is being done? # grants $ dollars spent # hours volunteered	What is the quality? How well is it done? % services/activity success rate  % attendance/participation
Effect	Is anyone better off? # or % with improvement in: skills, attitudes, behavior and circumstances

Step 5 - Set your collection cadence.

Cadence and method for each metric – Set your cadence and method for collecting each metric. You may not collect everything all at once, or everything  each year. Identify which metrics you will prioritize.  

Step 6 - Set your audience and communication expectations.

Who needs what data points and how frequently?

Step 7 - Collect, report and have fun!

Impact Measurement is an on-going practice 

Measuring impact is a practice that builds over time. Whether you are just starting out or have been doing it for a while, there is always room for growth. An organization can be advanced in some areas and still early or emerging in others. The chart below illustrates the different aspects along the maturity continuum. 

Aspects	EARLY	EMERGING	DEVELOPING	ADVANCED
Breadth	Does not collect data or metrics  for any programs	Collects data metrics for fewer  than 25% of programs	Collects data or metrics for 25% -  50% of programs	Collects data and metrics for at least 50% of programs
Depth: Outputs and Outcomes	Does not report data or metrics  for any programs assessed	Reports only outputs and activity  data for programs assessed	Reports some outcomes, but not  consistently or across-the board	Reports outcomes for all programs assessed
External Reporting	Does not report data in external  reporting	Reports single year at a glance	Reports year-to year change OR  progress to targets	Reports year-to-year   change AND progress to targets
Learning	Impact data isn’t reviewed or   discussed (if it is, it’s very general  for external purposes)	A few metrics are discussed in an ad hoc way, not tied to any decisions	Developing infrastructure to  support routine data review and   data-informed decisions	Infrastructure is in place to   consistently review data in context of decision-points and  business units