Measuring Effectiveness provides a tool for Ashoka to better understand the changes that social entrepreneurs—with the help of the Ashoka Fellowship—are making in their societies. Social entrepreneurs create complex changes in their societies, often working from multiple angles and on several levels to solve a problem. The approaches of individual entrepreneurs, working in an array of fields and toward varied aims, make it even more difficult to design a standardized tool for measuring impact.

Design

Each Ashoka Fellow in a particular cohort or survey class receives a self-response questionnaire. A cross section of the Fellowship also participate in in-depth interviews.

Proxy Indicators

The multiple-choice questionnaire relies on a series of “proxy indicators” that serve as measures of Ashoka’s successes toward strengthening civil society by supporting social entrepreneurs, their ideas and institutions.

Does the idea persist and has it spread?

• Are you still working toward your original vision?
• Have others replicated your original idea?
• Have you had impact on public policy?
  

Has an institution been created or expanded?

• What position does the institution currently hold in the field?
  

Has the Fellow’s relationship with Ashoka “enhanced” his or her work?

• Do you identify yourself as a social entrepreneur?
• Have specific aspects of your relationship with Ashoka enhanced your work?

Timing

The Measuring Effectiveness study captures a snapshot in time. Ashoka expects Fellows’ trajectories to change as they develop new strategies and improve their ability to spread their ideas.

Qualitative information

In-depth interviews supplement the surveys and provide a basis for understanding Fellows’ work. These case studies carried out by Ashoka staff introduce some of the richness lost by quantitative and multiple-choice responses alone. The reader learns, for example, which groups of citizens have been impacted, the systemic nature of the change, and the proposed strategies for long-term spread.

Limitations

The methods used for Ashoka’s Measuring Effectiveness project introduce some limitations.

• Self Reporting: All of the information presented here is reported directly by Fellows themselves. Fellows are encouraged to respond honestly and are explicitly told that the Measuring Effectiveness study does not evaluate their success but rather Ashoka’s impact on the field.
• Internal Process: Ashoka staff around the world execute all steps of the Measuring Effectiveness project from design to analysis. Ashoka has maintained this internal process both because it reduces resource intensity relative to external evaluation and because it allows the study to account for Ashoka’s particular perspective on social change.
  
• Irregularities: This report includes data collected over six years in more than 20 countries. The questionnaire was refined slightly through the years but never fundamentally changed. Translation naturally introduces some distortions. While most Fellows were contacted and responded via e-mail, a portion responded to survey questions over the phone or in person.