Project Overview

The field of evaluating nonprofit communications campaigns is not keeping pace with funder demands or grantee needs. Foundations desire to determine whether their investments are achieving targeted results. Meanwhile, grantees have little information about how to conduct meaningful evaluations of their communications activities.

Therefore, the Communications Consortium Media Center (CCMC) has established, with initial support from the David and Lucile Packard Foundation, the Carnegie Corporation and the W.K. Kellogg Foundation, a multi-year Evaluation Project that aims to provide foundations and nonprofits with methods of gauging the effects of strategic communications campaigns both large and small.

Background

Two trends are bringing about significant change in the philanthropic and nonprofit world.

1. Foundations are becoming increasingly focused on outcomes and results and are asking their grantees to measure impact.

Showing results has become a keystone of philanthropy. Foundations increasingly refer to themselves as outcome-focused and data-driven. They want to demonstrate that their grant investments are achieving meaningful, positive returns. They are hiring evaluation staff, contracting out for evaluations, and holding themselves and their grantees accountable in new ways. In 1992, several foundations formed the Grantmakers Evaluation Network (GEN) to promote the development and growth of evaluation in philanthropy and to build a culture of critical thinking and informed decisionmaking.

2. Strategic communication activities are becoming an established component of both foundation and grantee work.

Both foundation and grantee communication efforts are becoming more prevalent, strategic, and sophisticated. Communication goals may center on increasing public awareness, individual behavior change, or building public will to influence policy change. Strategies can range from traditional press conferences and press releases to innovative electronic and virtual approaches. The Communications Network, an affinity group of foundation public affairs staff, promotes communications as an essential and integral component of grantmaking. Overall, foundations are devoting more resources to reaching the public and targeted audiences through the media and other sources/vehicles-and they are supporting more communications activities as major components in grantee initiatives, both collaborative and individual.

While the strength of these two trends continues to grow, they have yet to converge in a viable way. In general, nonprofit program evaluators are not communications experts, and nonprofit communicators do not possess in-depth knowledge of evaluation. Evaluating strategic communications in the nonprofit sector is a fledgling field, and best practices and proven methods are not yet established. Since strategic communications require considerable time, expertise, and funding, foundations need to know: What is the best way to demonstrate the return on investment in nonprofit communications campaigns? Grantees want to know: How do we measure outcomes when we receive a grant that includes a communications component?

Since 1988, CCMC has assisted other nonprofit organizations in using effective communications strategies, tactics, and technologies to achieve greater public awareness and social and policy change. In 1999, we highlighted the importance of evaluation in the final chapter of our book The Jossey-Bass Guide to Strategic Communications for Nonprofits. Although this book has become a standard text for the nonprofit community, our work to date in evaluating nonprofit communication represents only a starting point. Now, CCMC wishes to bring together leading thinkers-both evaluators and communicators-from the nonprofit, for-profit, foundation and academic communities to define evaluation of nonprofit communications more fully. We will research, develop, and test principles for conducting meaningful evaluation of nonprofit communications campaigns and build a constituency for the results among both foundations and nonprofits.

Project Activities

In April 2002, CCMC commissioned the Harvard Family Research Project to investigate what is known about evaluating nonprofit communications campaigns. The result is a comprehensive paper entitled Public Communication Campaign Evaluation: An Environmental Scan of Challenges, Criticisms, Practice, and Opportunities.

In July 2002, CCMC assigned Voices for Change: A Taxonomy of Public Communications Campaigns and Their Evaluation Challenges to the Berkeley Media Studies Group, which mapped the universe of significant communications campaigns in the nonprofit arena. It identified key foundations and other nonprofits that are sponsoring and/or implementing strategic communications campaigns and summarized the projects.

In June 2003, two additional working papers were completed for the Evaluation Project. Dr. Charles Salmon of Michigan State University and his colleagues completed the first paper, Mobilizing Public Will for Social Change. The paper is a study of the "theory of change" behind many strategic media campaigns initiated by nonprofit organizations and foundations. And, the Harvard Family Research Project completed a second working paper, Lessons in Evaluation Communications Campaigns. This paper outlines the various methods that are currently employed to evaluate strategic media campaigns coordinated by nonprofits and foundations. This paper also details lessons learned from this previous evaluation research.

In April 2004, a concluding working paper was completed by the CCMC. The paper entitled “Guidelines for Evaluating Non-Profit Communications Efforts” summarizes the main findings of the first four Working Papers and offers guidelines for the evaluation of nonprofit communications efforts. The paper also articulates a set of evaluation principals, details the challenges of communications evaluation and stresses the need to incorporate a theory of change in any evaluation plan.

Key Findings

Planning activities have led us to four significant observations that will inform this multi-year project.

This project will involve field building. Integrating evaluation, communication, and social change in the nonprofit arena means building a new field. It will require bringing together the best minds from several disciplines in inclusive ways and providing ample opportunities for evaluators and communicators to focus on the challenge of evaluating nonprofit communication campaigns. Through these opportunities, both evaluators and communicators can advance and integrate their fields.

This project will involve examining how current theories apply to public will campaigns and developing principles for evaluating them. Traditional social marketing campaigns have been studied quite a bit, and there are several theories of change that assess how they work. However, cogent theories to support public will campaigns strategies are not as well defined. This project will face that challenge, drawing upon a wide range of existing knowledge, and establishing a theoretical basis for judging public will campaign strategies. Further, it will develop principles for evaluating the results of nonprofit, social marketing and public will campaigns.

Communications in Four ArenasStrategies from outside the nonprofit arena should be explored. For project purposes, the communications universe can be divided among four arenas. While the project will focus on those most relevant to nonprofits-social marketing and public will-other arenas must be examined for applicable communication and evaluation knowledge, theories, practices, methods, and tools.

To achieve broad acceptance, building a constituency within the nonprofit and foundation community will be an important aspect of the project. We have established a core work group of experts drawn from academia, the evaluation community, foundations, for-profit companies that help nonprofits communicate, and our own staff. We will also establish a second layer of 10-15 senior advisors to provide feedback on project activities and products. Face-to-face meetings and discussions within and between these groups will build the trust and credibility necessary for successful collaboration. In later stages of the project, we will need to reach out to many more people in the nonprofit community. The objective will be to provide information about and gain feedback, understanding and acceptance for new principles of evaluating nonprofit communications. Outreach strategies will center on written materials (such as papers) and events (such as meetings).

Project Objectives

The work plan for the multi-year project is based on the following objectives:

  • Create opportunities for evaluators, communicators, and other experts to come together to examine a new field that integrates communication, evaluation, and social change.

  • Engage a core work group of innovative experts from the nonprofit, academic, evaluation, and for-profit communication communities in seeking solutions to the challenge of evaluating social marketing and public will communication campaigns in the nonprofit sector.

  • Establish a group of advisors drawn from the nonprofit, academic, foundation and for-profit communities to provide feedback on the core work group's activities and products.

  • Gather knowledge about theories of behavior change—as well as theories, practices, methods, and tools for evaluating communications campaigns in the social marketing, commercial, and political arenas—and document findings in interim papers and the final working paper on the CCMC hosted extranet.

  • Develop a set of viable preliminary principles for evaluating social marketing and public will communications campaigns in the nonprofit sector that are articulated in the conclusive Working Paper.

  • Identify a nonprofit constituency and implement effective dissemination methods to achieve understanding of the principles and promote their use. CCMC will ensure that key Foundation representatives and grantees are part of this outgoing consultation process.

  • Engage selected nonprofit organizations and foundations in testing the preliminary principles to evaluate actual communications social marketing or public will campaigns.

  • Revise the preliminary principles, based on test outcomes.

  • Incorporate the revised principles into a set of tools foundations and their grantees can use to evaluate strategic communication social marketing or public will campaigns. Ensure that both communities understand what our recommended evaluation process can and cannot accomplish.

  • Disseminate project outcomes and tool kits throughout the nonprofit community.

 

 

CONTACT US:
Communications Consortium Media Center
401 Ninth Street, NW Suite 450
Washington, DC 20004-2142
(202) 326-8700
www.CCMC.org