The Step-by-Step Guide to
EVALUATION
H o w t o B e c o m e S a v v y E v a l u a t i o n C o n s u m e r s
The W.K. Kellogg Foundation, founded in 1930 by breakfast cereal pioneer Will Keith Kellogg,
is among the largest philanthropic foundations in the United States.
Based in Battle Creek, Michv, WKKF engages with communities in priority places across the country
and internationally to create conditions that propel vulnerable children
to realize their full potential in school, work and life.
© 2017 W.K. Kellogg Foundation.
This publication may be shared without charge for educational, charitable, nonprofit or other non-commer-
cial purposes. Such use must include acknowledgement that it is “with the permission of the W.K. Kellogg
Foundation.” All other use is prohibited without the prior written permission of the W.K. Kellogg Foundation.
For permission requests, please contact [email protected] or 269-969-2079.
The Step-by-Step Guide to Evaluation | page i
Foreword
At the W.K. Kellogg Foundation, we believe that evaluation is an effective management tool to both inform
strategy development and track the progress and impact of strategy implementation. We have long been
committed to supporting our grantees’ ability to derive and share lessons learned from their work. To that
end, the foundation published the first Evaluation Handbook (The Kellogg Foundation, 1998) almost two
decades ago to guide evaluation for our grantees. Since that time, as the discipline of evaluation grew and
expanded, the demand for evaluation has risen. More and more nonprofit leaders and practitioners strive
to design evidence-based programs, and more and more funders require their grantees to provide evidence
to demonstrate the success of their funded work. The democratization of evaluation makes it necessary that
evaluation is both rigorous and practical. How to achieve the balance motivated us to update the handbook.
Over the years, the foundation has learned a lot from our grantees about the challenges of evaluation. This
handbook is our continuous effort to demystify evaluation and facilitate its use, for the foundation’s grantees
and for all organizations committed to learning and strengthening their work. It is designed for people who
have little to no exposure to formal evaluation training and provides a starting point for them as they
consider evaluating their work. It is intended to help them become more informed consumers of evaluation.
Evaluations can be simple or extensive depending on the scope and complexity of the work being evaluated.
The scope of the evaluation could potentially include a single program, a multi-site initiative, or a
multifaceted strategy aimed at systems and community change. Regardless of the complexity of the effort,
the basics for evaluating it remain the same and this handbook was written to impart information about
these basics. This handbook complements two other products produced by the Kellogg Foundation: the Logic
Model Development Guide (The Kellogg Foundation, 2004) and the Systems-Oriented Evaluation Guidebook
(The Kellogg Foundation, 2005).
We would like to thank Community Science for writing and HGF, Inc., for designing this handbook, WKKF staff
for their leadership in developing this resource, as well as the following individuals who served as advisors:
Jara Dean Coffey, Traci Endo, Johanna Morariu and Jianping Shen.
Huilan Krenn, Ph.D.
Director of Learning & Impact
W.K. Kellogg Foundation
September 2017
Page ii | The Step-by-Step Guide to Evaluation
Contents
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Chapter 1: Principles to Guide Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 W.K. Kellogg Foundation’s Principles about Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Community Engagement, Racial Equity and Evaluation . . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Importance of Culture in Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 American Evaluation Association’s Guiding Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 2: Definition of Evaluation and the Evolution of Evaluation Practice . . . . . . . . . . . . . . . . . 13
2.1 What is Evaluation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.1 Definition of Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.2 Definition of Evaluative Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Evolution of Evaluation Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Chapter 3: Evaluation Types, Methodologies and Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1 Evaluation Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2 Evaluation Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3 Evaluation Methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Chapter 4: Overview of the Evaluation Process that Reflects Evaluative Thinking . . . . . . . . . . . . . . 37
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1 Overview of the Evaluation Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Creating a Learning Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.3 Stages of the Evaluation Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.4 Navigating Evaluation Choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.4.1 Capacity, Program, or Population Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.4.2 Focus or Unit of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.4.3 Internal Versus External Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.4.4 Power Dynamics Among Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
The Step-by-Step Guide to Evaluation | page iii
Chapter 5: Preparing for the Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.1 Why You Might Consider an Evaluation of Your Strategy, Initiative or Program . . . . . . . . 66
5.1.1 Considerations if Your Evaluation
is Part of a National or Statewide Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.1.2 Considerations if the Findings from Your Evaluation
Have Significant Implications for a Theory, Practice or Policy . . . . . . . . . . . . . . . 68
5.1.3 Getting Ready for Evaluation Whether or Not It is a Funding Requirement . . . . . 69
5.2 Who Benefits From the Evaluation and What Are the Potential Risks? . . . . . . . . . . . . . . . 70
5.2.1 Consideration of Different Types of Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2.2 Use of Evaluation by Different Types of Stakeholders . . . . . . . . . . . . . . . . . . . . . . 73
5.3 Consider Your Organization’s Capacity to Participate in the Evaluation . . . . . . . . . . . . . . 75
5.3.1 A Staff Person Dedicated to Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.3.2 Budget for Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.3.3 A Culture of Data and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.3.4 Share and Protect Data Across Program Elements or Program Locations . . . . . . 79
Chapter 6: Determine Stakeholders and Engage Them in the Evaluation . . . . . . . . . . . . . . . . . . . . 83
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.1 Evaluators as Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.1.1 Keep Your Evaluator Informed on a Frequent, Regular Basis . . . . . . . . . . . . . . . . 84
6.1.2 Two Types of Evaluators: Internal and External . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.1.3 The Evaluator’s Role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.1.4 The Evaluator’s Cultural Competency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.2 Other Key Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.2.1 How to Identify Key Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.2.2 How to Engage Key Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Chapter 7: Developing a Logic Model, Evaluation Questions,
Measurement Framework and Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
7.1 Overview of Logic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7.2 Use the Logic Model throughout Your Effort’s Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . . 109
7.2.1 Use of Logic Models to Design Your Strategy, Initiative or Program . . . . . . . . . . 109
7.2.2 Use of Logic Models to Inform Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Page iv | The Step-by-Step Guide to Evaluation
7.2.3 Use of Logic Models to Develop Evaluation Questions . . . . . . . . . . . . . . . . . . . . 110
7.2.4 Use of Logic Models to Develop an Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . 110
7.2.5 Use of Logic Models to Create an Outline for Your Evaluation Report . . . . . . . . . 111
7.2.6 Use of Logic Models To Improve Your Strategy, Initiative or Program . . . . . . . . . . 111
7.3 What You Can Expect From Logic Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.3.1 It is a Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.3.2 The Process Can Surface Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113
7.3.3 The Process Can Bring About Innovation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.4 How to Go About the Logic Modeling Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.4.1 Whom to Involve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.4.2 How to Involve Them . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
7.5 Components of the Logic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.5.1 Alignment Between Strategies or Activities and Outcomes . . . . . . . . . . . . . . . . 118
7.6 From Logic Model to Evaluation Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.6.1 Two Different Ways to Formulate Evaluation Questions
About Your Strategy, Initiative or Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.6.2 Typical Scenarios When Formulating Evaluation Questions . . . . . . . . . . . . . . . . 123
7.6.3 Other Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7.7 From Logic Model to Evaluation Questions to Measurement Framework . . . . . . . . . . . . 124
7.7.1 Key Components of the Measurement Framework. . . . . . . . . . . . . . . . . . . . . . . . 125
7.7.2 How to Use a Measurement Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.7.3 Considerations for Developing the Measurement Framework . . . . . . . . . . . . . . 127
7.7.4 Taking a SMART Approach to Measurement Framework Development . . . . . . . 127
7.8 Your Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.9 Budgeting for Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.9.1 Types of Expenses Typically Associated With Evaluation . . . . . . . . . . . . . . . . . . 134
7.9.2 Time Estimates for Specific Evaluation Activities . . . . . . . . . . . . . . . . . . . . . . . . 136
Chapter 8: Data Collection and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.1 Determine Data Collection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
8.1.1 What You Need to Ask Before Collecting Any Data . . . . . . . . . . . . . . . . . . . . . . . 143
8.1.2 Use of Quantitative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
8.1.3 Use of Qualitative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
The Step-by-Step Guide to Evaluation | page v
8.1.4 Use of Mixed Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
8.2 Critical Considerations in Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.2.1 Who the Data Collector Is . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.2.2 How to Engage Populations Who Have Been Traditionally
Excluded or Treated as Invisible . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.2.3 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2.4 Confidentiality and Anonymity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.2.5 Obtaining Consent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.2.6 Institutional Review Board (IRB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.3 Analyzing and Interpreting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
8.3.1 Quantitative Data Analysis and Interpretation of Results . . . . . . . . . . . . . . . . . . 167
8.3.2 Qualitative Data Analysis and Interpretation of Results . . . . . . . . . . . . . . . . . . . 175
Chapter 9: Summarize, Communicate and Reflect on Evaluation Findings . . . . . . . . . . . . . . . . . . 183
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
9.1 Communicate and Report Your Evaluation Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
9.1.1 What You Need to Ask Before Putting the Findings Together . . . . . . . . . . . . . . . 184
9.1.2 Different Communicating and Reporting Formats and Options . . . . . . . . . . . . . 187
9.2 Considerations in Developing and Implementing a Communications Plan . . . . . . . . . . 188
9.3 Keeping It Simple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
9.4 Ways to Display Your Evaluation Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
9.4.1 Displays of Patterns and Trends Over Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
9.4.2 Displays of Distribution and Spread of Responses . . . . . . . . . . . . . . . . . . . . . . . 194
9.4.3 Displays of Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
9.4.4 Displays of Frequency of Words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.4.5 Displays of Themes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
9.5 Reflecting on Your Evaluation Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
9.5.1 Use of Evaluation Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
9.5.2 Considerations in Preparation for Reflecting on,
Discussing and Using Evaluation Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Guide to Evaluation Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
Page vi | The Step-by-Step Guide to Evaluation
What is evaluation? How can it help your organization?
And how can you work more effectively with your evaluator?
This handbook is designed to demystify evaluation and help you
get the most out of evaluation for your organization.
The Step-by-Step Guide to Evaluation | page vii
Overview
What is evaluation? How can it help your organization? And how
can you work more effectively with your evaluator? This
handbook is designed to demystify evaluation and help you get
the most out of evaluation for your organization.
The evaluation profession is multi-faceted and can be
characterized partly by its theoretical debates, ethical
considerations and proprietary interests. This handbook will
not attempt to provide in-depth evaluation information that has
little relevance to the day-to-day evaluation needs of nonprofit
directors and staff, nor will it transform anyone into an
evaluator. On the contrary, the goal of this handbook is to
educate busy nonprofit directors and staff, such as yourself,
about the essential elements of evaluation, so you can work
more effectively with trained evaluators; hold evaluators
accountable to the highest standards of quality, integrity and
competency; and maximize the usefulness of evaluation to your
organization.
The scope of your evaluation may include a single program, a
multi-site initiative or a multifaceted strategy aimed at systems
and community change. Regardless of the complexity of the
effort, the basics for evaluating it remain the same and this
handbook was written to impart information about these basics.
Before discussing each chapter in detail, here are a few things to
know about how this handbook is organized:
When the term “you” is used in this guide, it refers to you –
LOOK FOR THESE
This symbol
indicates a new
term is introduced.
The handbook
provides many definitions, all of
which are combined into one list
(“Glossary”) at the end of the
guide. This can be printed as a
reference.
This symbol is
used to summarize
tips.
This symbol refers
to an important
point.
Page viii | The Step-by-Step Guide to Evaluation
the reader or user of this handbook.
“Highlights” will be included in most chapters to provide a
concise summary of key points.
Brief examples are included within the chapters to make it
easier to understand the information.
Short exercises are included at the end of chapters to help
you retain its key points.
Many publications were consulted in creating this handbook.
You will find the references to these publications at the end
of the handbook.
An annotated resource guide for evaluation was created to
supplement this handbook; you will find it at the end of the
handbook. The resources are organized by evaluation topic
as well as issue such as education, poverty, health and
workforce development. Additionally, relevant and practical
resources cited in each chapter are included here.
This handbook contains nine chapters.
Chapter 1 synthesizes the W.K. Kellogg Foundation’s philosophy
concerning evaluation and the American Evaluation
Association’s guiding principles.
Chapters 2 and 3 explain essential information about the
evaluation field. As mentioned before, the evaluation field is
vast, so only some aspects of evaluation are summarized here.
Chapter 2 lays out the historical development and significant
contributions leading to the current status of the evaluation
field. Chapter 3 describes the various evaluation types, method-
ologies and approaches.
The Step-by-Step Guide to Evaluation | page ix
Chapter 4 is a brief overview of the overall evaluation process and its stages, which align with the
subsequent chapters.
Chapters 5 through 9 provide more detail about each stage in the evaluation process and explain
what you should know to work most effectively with an evaluator or how to use evaluation in your
work. Chapter 5 discusses some key considerations when planning your evaluation. Chapter 6
explains how to determine your evaluation stakeholders and engage them in the evaluation
process. Chapter 7 focuses on developing and using a logic model and on generating evaluation
questions, a measurement framework and an evaluation plan. Chapter 8 provides steps for
determining what type of data to collect and guidance on analyzing and interpreting the data.
Finally, in Chapter 9, you will learn considerations for summarizing and communicating your
evaluation findings.
To the extent possible, each chapter is written as a stand-alone section. Depending on what you
want to learn or what you already know, you can select the chapters you want to read. Thus, you
will find some information (e.g., definitions, concepts, resources) repeated in several chapters.
Page x | The Step-by-Step Guide to Evaluation
1
The Step-by-Step Guide to Evaluation | page 1
Principles to Guide Evaluation
introduction
This chapter is intended to
help you understand the W.K.
Kellogg Foundation’s (WKKF)
principles and the American
Evaluation Association’s
principles concerning
evaluation. As a current or
future grantee of the
foundation, you could find the
foundation’s principles
informative. Accordingly, the
guiding principles—upheld
by the American Evaluation
Association—will help you, as
consumers of evaluation,
to know what to expect of
professional of evaluators as
you work alongside them.
The foundation emphasizes
the necessity of incorporating
culture, community engagement
and racial equity into efforts to
effect community and systems
change; as such, you can see
these topics throughout the
handbook.
1.1
w.k. kellogg foundation’s principles
about evaluation
The Kellogg Foundation believes that all people have the
inherent capacity to effect change in their lives, in their
organizations and in their communities. What often limits the
ability to effect change is the availability of tools and resources
to actualize community desire for change. The foundation is
therefore committed to supporting the change process by
providing tools and resources that assist individuals,
organizations and communities in the generation and practical
application of knowledge that will result in improvements in the
quality of life for children and inform the narratives about certain
groups of people. Evaluation is one such tool.
Evaluation supports the ability to monitor and measure the
quality, pace and direction of change that individuals,
communities and organizations undertake. It does this by
systematically generating knowledge that can support learning,
How this chapter is organized…
1.1 W.K. Kellogg Foundation’s Principles about Evaluation
1.1.1 Community Engagement, Racial Equity and Evaluation
1.1.2 Importance of Culture in Evaluation
1.2 American Evaluation Association’s Guiding Principles
Exercises
Page 2 | The Step-by-Step Guide to Evaluation
quality improvement and good judgment in decision-making. Evaluation also can align purpose,
action and impact to ensure that longer-term change at the societal level unfolds progressively.
The foundation believes the following principles should guide evaluation:
Planning for evaluation should begin the moment new strategies, initiatives and programs are
conceptualized, evaluation findings should be used both to strengthen strategy development
and programming along the way, and to measure the extent of change.
Evaluation should support an organization’s efforts to become stronger and more effective and
should enhance its ability to obtain and effectively use new resources.
Evaluation should be adapted to the contexts of the community being evaluated and to the
important outcomes identified by the community (e.g., policy, impact on equity). Mixed
methods and different perspectives (e.g., a racial equity lens) can help capture the reality and
outcomes experienced by community members.
Evaluation should be designed to address real issues and to provide staff and stakeholders
with reliable information to address problems and to build on strengths and opportunities.
Mixed methods intentionally use two or more kinds of data gathering and analysis
tools — typically a combination of qualitative (e.g., focus groups and
interviews) and quantitative (e.g., multiple choice surveys and assessments) —
in the same evaluation.
A stakeholder is any person or group who has an interest in the strategy, initiative or
program being evaluated or in the results of the evaluation, including the evaluator.
Evaluation should invite multiple perspectives and involve a representation of people who care
about and benefit from the program.
Evaluation should be flexible and adaptable; strategies, initiatives and programs don’t exist
in a vacuum and events such as staff turnover, elections, legislation and economic recession
can affect their implementation and outcomes. Therefore, evaluators and implementers must
be flexible and work together to adapt to such events and respond to the needs of community
members.
The Step-by-Step Guide to Evaluation | page 3
Evaluation should build the skills, knowledge and perspectives of individuals to self-reflect,
dialogue and act based on data and knowledge. This strengthens the capacity of all
participants to establish a learning environment and work together to solve problems.
1.1.1
Community Engagement, Racial Equity and Evaluation
Community engagement and racial equity are central to the Kellogg Foundation’s work. In
evaluation, they are interconnected, especially in community change initiatives because people
of color usually make up Kellogg’s stakeholders and are usually the ones left out of discussions
about the evaluation design and implementation. Consider, for instance, an initiative designed to
improve the identification, referral and treatment of children exposed to violence in a Native
American community. Due to historical trauma experienced by Native American communities,
the evaluator must engage tribal leaders in discussions about the best ways to measure the
identification, referral and treatment, along with realistic expectations for change over the
initiative’s grant period. The evaluator should not assume the success measures for this
community are the same as for other communities because of the history, governance structure,
location, socioeconomic conditions and traditional norms of the tribe. This means paying explicit
attention to factors such as:
How tribal leaders and community members talk about the violence they experience.
What tribal and nontribal mental health resources exist.
What nontribal mental health resources are available and how culturally competent are the
resources.
Who needs to give the evaluator permission to collect data.
Who owns the …
Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.
You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.
Read moreEach paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.
Read moreThanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.
Read moreYour email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.
Read moreBy sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.
Read more