Evaluating Community Change
This two day workshop is for workers at all levels in the not-for-profit and related sectors. It focuses on the newly emerging concepts and methods to evaluate efforts to tackle tough community issues, such as neighbourhood renewal, poverty and homelessness, community safety, educational achievement and health.
The number and diversity of initiatives designed to make a big impact on vexing social challenges is increasing. Whether it is community safety, homelessness or poverty, educational achievement or kids in care, many community, government and private sector leaders seek to “turn the curve” or “move the needle” at the community, population or regional level.
In the process, they have learned that traditional evaluation practices can short-circuit – rather than enable – community change efforts because they were designed to address small scale and simple – rather than large scale and complex – problems. Happily, evaluators have made progress in crafting concepts and methods better suited to the work of community change.
This workshop will review the dynamic nature of addressing tough social issues and explore five major evaluation challenges that social innovators (e.g. lag time between activities and results, unanticipated outcomes, attribution of results to activities) face and how evaluators can help them address them.
For many years evaluation practice has tended to focus on three broad purposes. Formative evaluation is used to help improve a program or policy. Summative evaluation is employed to judge the merit or worth of a program or policy to determine whether it should be sustained, discontinued or scaled up. Accountability evaluation is used to assess the extent to which an organization or group is ‘implementing a detailed model with fidelity’ to an already approved – often rigid – blueprint.
There are, however, plenty of ‘developmental’ situations where neither formative, summative or accountability evaluation are appropriate and may even be counterproductive. These include: creating an entirely new program or policy out of thin air, adapting a proven program or policy in a fast moving environment, importing a program or policy that proved effective in once context into a new one, scaling up a successful model, dealing with complex issues where solutions are uncertain and/or stakeholders are not on the same page.
This will introduce participants to the key concepts, principles and practices of developmental evaluation – an increasingly popular approach to helping decision-makers navigate developmental situations with rapid feedback, critical thinking and rigor sense-making – and share some emerging resources that they may find helpful in their own work.
Evaluating Emerging Strategy
Traditional evaluation works best when the “intervention” being assessed has clear outcomes, well-articulated pathways to change, and well-developed mechanisms for change. More often than not, social innovators’ interventions are emergent and ever-evolving. When evaluators require them to meet the standards of traditional evaluation, they can short-circuit the work of community change by forcing social innovators to pre-maturely and artificially clarify – and inappropriately preserve – their strategy or model. This session will explore different ways to conceptualize, track and assess emergent and dynamic strategies.
Attribution & Contribution
The traditional methods for attributing outcomes to an intervention – randomized controlled trials – are rarely ethical, feasible or useful for community change efforts. The two ways of coping with the gap – (a) claiming that results are the attributable to activities or (b) not trying to answer the question at all – are unhelpful for serious change makers. This session will explore the relatively new methodology and emerging techniques of contribution analysis which seeks to understand the relative contribution of a community change effort on community changes.
The pathway to moving the needle on population level outcomes (e.g. graduation rates) is not through more or bigger programs and organizations. It is through changing systems (e.g. policies, resource flows, culture, practices) that lead to those outcomes in the first place. This session will explore and test some of the latest frameworks and methods for evaluating changes to systems, including outcome mapping, outcome harvesting, bellwether evaluation and most-significant change.
Intended & Unintended Outcomes
All social innovators and evaluators quite rightly spend a great deal of time trying to agree on, measure and judge their progress towards some tangible outcome (e.g. reducing rates of teen pregnancy). n social innovation and community change efforts, there are plenty of unanticipated outcomes as well, some of which turn out to be more important than the intended ones! This session will explore several principles and methods designed to capture both intended and unintended outcomes.
Accountability Versus Strategic Learning
The original intent of evaluation was to encourage communities, organizations and policy makers to employ evaluative thinking and processes to test big ideas about how to tackle complex issues. For a variety of reasons, the focus for evaluation over the last thirty years has shifted dramatically to using evaluation to encourage social innovators to account for their activities, use of funds, and generation of outcomes to external bodies. This emphasis, while understandable, dramatically narrows the scope of experimentation and discourages the learning and critical thinking so central to social innovation.
This session will explore the question: What does accountability look like when it is not possible – or even desirable – to rely on traditional focus on a high fidelity to a (emergent and evolving) plan and delivery of (unpredictable) results on a fixed (but really shifting) schedule?
Be inspired by the following presenter:
Mark is an Associate of Tamarack – An Institute for Community Engagement and Vibrant Communities Canada. Mark is also the President of the company From Here to There.
Mark’s current focus is on developing practical ways to assist groups understand, plan and evaluate policies, programs and initiatives that address complex issues. This includes challenges such as neighborhood renewal, poverty and homelessness, community safety, educational achievement and health. He is particularly focused on expanding the ideas and practice of developmental evaluation, a new approach to evaluation which emphasizes learning and design thinking in emerging and sometimes fast-moving environments.
Mark brings experience from a variety of sectors. In the 1990s, he served as the Foreign Assistance Coordinator for Grants in Poland’s Ministry of Privatization, was the Mission Coordinator for the United Nations Development Program’s first regional economic development initiative in Eastern Europe, and worked with International Privatization Group-Price Waterhouse. In Canada, he was the Coordinator of the Waterloo Region’s Opportunities 2000 project – an initiative that won provincial, national and international awards for its multisectoral approach to poverty reduction – and served briefly as the Executive Director of the Canadian Community Economic Development Network (CCEDNet). From 2002-2010, he was the Executive Director of Vibrant Communities Canada and a Director at Tamarack.
Mark lives in Edmonton, Alberta with his wife Leann and their children Isaiah and Zoë.
Two full day workshop: $395.00 + GST.
Included in the registration:
- Attendance at the 2 day workshop
- Networking session at the end of day one, drinks and nibbles provided
- Name tag, delegate list, delegate bag
- Welcoming coffee/tea, morning and afternoon tea, and lunch
Note: Accommodation is not included, delegates must make their own arrangements.
There are 3 steps in the online booking process. You will receive an automated confirmation email. If you have any issues or questions about your booking, please contact Cris Sanders on 06 878 3456 or email: firstname.lastname@example.org