This is a repost of my blog post at the ASCN (Accelerating Systemic Change Network) blog. Check out and subscribe to their blog for more posts like this!
For many of us, it’s proposal writing season. If you are submitting an NSF-IUSE proposal, there are increasing expectations that the proposal will include a theory of change for how the project aims to achieve its outcomes, and a well-developed evaluation plan for assessing progress toward those outcomes. As an evaluator, I am often asked to help people flesh out these objectives and metrics, and I have found several of the resources on the ASCN site very useful. I was lucky enough to be helping out with the ASCN project when these resources came in, and acted as a temporary librarian to create the list of resources on the site, and so am quite familiar with the breadth of resources. This blog post is to point you toward some of my favorites that have been useful when writing a proposal aimed at institutional change.
ASCN Working Group 1 (“Models and Theories”) is tasked with better supporting peoples’ use of theories, models and scholarship in their planned change efforts. As they work on their guidance to us, there are many resources that I have already drawn on from the Guiding Theories Resources section of ASCN.
- Why theories of change matter (WCER Working Paper No. 2015-2): Connolly, M. R., & Seymour, E. (2015). (Working Paper, 2015). Connolly and Seymour. Argues that making programmatic theories of change explicit, by helping organizers specify their reason for their theory of change or action, can be useful.
- Promoting Evidence-based Change in Undergraduate Science Education. (Paper, National Academies, 2011). Ann Austin. Drawing on the literature on organizational change, the paper uses a systems approach to identify and understand barriers to change in STEM teaching practice and to highlight factors and strategies that relate to individual faculty members’ decisions to integrate evidence-based approaches into their teaching practice, including an overview of the literature and the usefulness of a systems approach.
- Increasing the use of evidence-based teaching in STEM higher education: A comparison of eight change strategies (Journal article, J. Eng. Educ., 2014). Borrego, M., & Henderson, C.) and Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature(Henderson, Beach and Finkelstein, 2011). Presents a framework of four change strategies typically used in the literature, and argues for diversifying approaches across multiple strategies.
- Exploring the dynamics of organizational learning: identifying the decision chains science and math faculty use to plan and teach undergraduate courses (Article, 2011). Hora and Hunter. Describes organizational learning theory as a way to conceptualize how decision cains are implicated in change processes, including organizational memory.
- A guide to building educational partnerships: Navigating diverse cultural contexts to turn challenge into promise. (Book, 2011) Hora and Miller. This book provides insights and guidance to enable prospective and existing education partners to develop answers to the questions that are critical to success: Why engage in this partnership? How can you communicate the potential benefits of partnership? Helpful for establishing working groups.
And not part of the ASCN resources, but highly valuable are:
- Change Model Required – Day 3 of Developing a Competitive NSF RED Proposal webinar. See recordings and the slide deck on change theories and how they can help your revolutionary proposal succeed. This was cited as a highly valuable resource by ASCN participants.
- Center for Theory of Change (website). For those new to the idea of a Theory of Change, or even experienced practitioners, this site defines a Theory of Change, and walks you through the process of identifying goals, assumptions, indicators, and planning interventions. See in particular the Project Superwoman guided example, which was valuable to me recently in developing a Theory of Change for a complex program.
- Curious about how a logic model differs from a theory of change? See this presentation and this article.
- See also this nice set of visual representations of theories of change.
See also the Costs and Benefits Resources section for links to help you make the case for change in terms of cost-return of investment in student education.
After you identify your Theory of Change, the next step in a “backwards design” of an educational intervention is to identify the metrics to assess that change, and develop an evaluation plan.
- Check out the Assessment of Change Resources for a list of tools for measuring student learning, classroom practice, departmental readiness, and organizational change.
- American Evaluation Association (Organization). Network and professional society for evaluators, including ability to find evaluators, and many resources for performing evaluation and communicating results to stakeholders. Includes a topical group for STEM education evaluation.
- Increase the Impact (Website). Website and guidebook providing guidance to educational innovators for strategies for sharing and disseminating their work in a way that’s likely to engage and affect faculty, such as avoiding telling faculty what to do, and engaging adopters earlier in the process. This is highly recommended to review for developing a plan to involve stakeholders early, so you have a powerful dissemination plan that is woven throughout the project.
Not in the ASCN resources but very useful:
- Better Evaluation (website). This is the best collection of resources I know for planning an evaluation, with information on developing logic models, defining outcomes, and a range of evaluation approaches.
- Assembling a winning team – Day 2 of Developing a Competitive NSF RED Proposal webinar. See recordings and the slide deck for a webinar on choosing the right team, including the evaluator.
In the Strategies and Programs Resources are many excellent documents to help guide you in designing a program to include what is known in educational reform and systemic change. Consult these when you are trying to develop the “big picture” of your intervention, to make sure you are not missing key elements. Citing these and similar sources will also show that you are familiar with the national landscape.
- See the Leaders of Change Resources area for recommendations on how to effectively prepare and support change agents, recommendations for societies and networks, and faculty professional development.
- Increasing Student Success in STEM: A Guide to systemic institutional change. (Guidebook, 2016) From AAC&U and PKAL, authored by Elrod and Kezar is aimed at campus leaders and administrators who are leading comprehensive reforms, including addressing implicit theories of change, avoiding mistakes, project management, team development, and sustaining change, with a practical focus.
- Achieving Systemic Change; a sourcebook for Advancing and Funding STEM Education (Workshop sourcebook, AACU , 2014). The sourcebook discusses how best to effect systemic change in undergraduate STEM, including the rationale for change, areas of investment, and key reports.
- Understanding Interventions that Broaden Participation in Science Careers (Website). Network of individuals: Our broader mission is to positively impact the outcomes of efforts aimed at broadening participation in research and research careers through the expansion and dissemination of the relevant body of knowledge, and facilitate its implementation in practice, policy development, training, and professional development.
- Transforming Institutions (Book, 2015). This book brings together chapters from scholars in the 2011 and 2014 Transforming Institutions conferences. It provides an overview of the context and challenges in STEM higher education, descriptions of programs and research, and summary of lessons learned, plus next steps.
- How Colleges Change (Book, 2013). Adrianna Kezar. Outlines changes processes and reform, in theory and practice, in higher education. Includes discussion of cultural issues.
See also the Cross Cutting Resources area for links related to equity and inclusion, two year colleges, and instructional techniques.
This is just a sampling of what I have found useful in the ASCN resources. Part of the value of the network is in gaining access to the knowledge across different disciplines and domains of scholarship. And please, if you have additional resources to add, please do so through the Contribute a Resource page!
Happy proposal writing!
Image credit: Jess Mann [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons