Tracking Systems to Measure Impact & Outcomes

Greetings All,

I'm interested in learning of "who's doing what" relative to tracking
impact and outcomes. Specifically, the impact and outcomes practitioners'
programs are having on their program participants. Additionally, of
performance standards and software they may be using.

Thanks in advance,

Walter Merkle


This post transferred from the cdb-l mailing list

Walt:

Talk to the Aspen Institute in D.C. - they are leading a few projects which
get at this very issue. You want Peggy Clark, but its hard to get a hold
of her. Also try Joyce Klein. Thanks

At 09:09 AM 3/2/99 -0500, Walter J. Merkle wrote:
>Greetings All,
>
>I'm interested in learning of "who's doing what" relative to tracking
>impact and outcomes. Specifically, the impact and outcomes practitioners'
>programs are having on their program participants. Additionally, of
>performance standards and software they may be using.
>
>Thanks in advance,
>
>Walter Merkle
>
>


This post transferred from the cdb-l mailing list

This is THE issue today in the CDFI and community development lending
industry. Those who can manage both their primary business, and
communicate their mission impact through data will grow & thrive. Those
who can't will not.

With each funder and donor asking for more data, and in unique formats,
social impact reporting needs to be a core competence. Reporting is also
not just an end in itself, it also needs to have a relationship with
management and program evaluation.

--
Donald R. Hinkle
Director, Affordable Housing Group
The Reinvestment Fund
718 Arch Street, Suite 300N
Philadelphia, PA 19106
Voice: 215-925-1130 x212
Fax: 215-923-4764


This post transferred from the cdb-l mailing list

Just FYI, a seminar instructing banks how to apply for the CDFI and BEA
awards will take place on May 27, 1999, at the South Branch Inn in
Moorefield, WV from 11:00AM to 4:00PM. The WV Commissioner of Banks, FDIC,
Lightstone CDC, NeighborWorks, and various other community organizations
are sponsoring. Please do not hesitate to contact me if you are interested.
Elizabeth R. Kelderhouse
Fair Lending Specialist
FDIC
Atlanta Region
Ekelderhouse@fdic.gov <mailto:Ekelderhouse@fdic.gov>


-----Original Message-----
From: Donald R. Hinkle [SMTP:hinkled@dvcrf.com]
Sent: Tuesday, March 02, 1999 10:33 AM
To: waltmerk@valinet.com
Cc: COMMUNITYDEVELOPMENTBANKING-L@cornell.edu
Subject: Re: Tracking Systems to Measure Impact & Outcomes

This is THE issue today in the CDFI and community development
lending
industry. Those who can manage both their primary business, and
communicate their mission impact through data will grow & thrive.
Those
who can't will not.

With each funder and donor asking for more data, and in unique
formats,
social impact reporting needs to be a core competence. Reporting is
also
not just an end in itself, it also needs to have a relationship with
management and program evaluation.

--
Donald R. Hinkle
Director, Affordable Housing Group
The Reinvestment Fund
718 Arch Street, Suite 300N
Philadelphia, PA 19106
Voice: 215-925-1130 x212
Fax: 215-923-4764


This post transferred from the cdb-l mailing list

We are in the final editing stages of a fairly major piece on this very
issue...it will be released in May of this year.

Briefly, based on our experience over the past ten years, evaluation (as
widely conceived and practiced) is useless. It tends to be punative,
retrospective, lacking in integrity and does not inform current practice.

Therefore, we have abandoned evaluation and re-framed our efforts along the
lines of building information systems that are designed jointly with
practitioners, provides them with more "real time" information and, in the
process, creates a set of data points we as the philanthropic investor can
use to assess social return on investment.

We have spent the last 18 months working with our investees/grantees to
build a management information system called WebTrack that monitors both
social and business performance indicators. At the point of contact, each
program participant responds to a fairly comprehensive survey that "frames"
where they are at that point in time. A similar survey instrument is used
at point of exit. And then ongoing follow-up takes place over a 36 month
period. These instruments are then "marked" against various social
indicators identified by the program managers and tracked over time. We
have beta tested this system and went "live" this year.

As far as our economic development component in which these folks
participate, we also have a set of business indicators (we operate social
purpose ventures that employ homeless and very low-income folks). These
indicators are tied to finanical reports for each of the 23 businesses in
our portfolio and tracked on a monthly basis.

Finally, we have also developed a set of financial templates that will,
later this year, be "dropped" on top of both the social and business
indicator MIS data in order to assess the capital required to achieve this
socio-economic value and calculate a social return on investment based upon
the economic value of the social impacts generated by virtue of those
investments, calculating a social share value for each investee/grantee and
our portfolio as a whole. This "SSV" will then be tracked over time and
adjusted based upon both shifts in the capital structure of the nonprofit
and actual social earnings generated (ie. social impacts achieved).

In some ways, more of the same, in others, bleeding edge....

love to hear about other's efforts as well!

yours,

j.





At 09:09 AM 3/2/99 -0500, you wrote:
>Greetings All,
>
>I'm interested in learning of "who's doing what" relative to tracking
>impact and outcomes. Specifically, the impact and outcomes practitioners'
>programs are having on their program participants. Additionally, of
>performance standards and software they may be using.
>
>Thanks in advance,
>
>Walter Merkle
>
>

************************************************** **************************
Pursue Peace and Profitability!

Please note new address and numbers!

Jed Emerson
Executive Director
Roberts Enterprise Development Fund
Presidio Building #1009
P.O. Box 29266
San Francisco, CA 94129-0266
415-561-6680 Phone
415-561-6685 Fax
http://www.redf.org/ Web Site (with helpful info and links!)
************************************************** **************************


This post transferred from the cdb-l mailing list

Jed Emerson,
Have you done a time/value study? How much time does it take for all
these practitioners to complete your evaluation forms. Representing a
CDFI with over 800 philanthropic investors, 100 annual donors, and
programs funded by most major foundations, my living nightmare is that
each creates such a reporting system...........and one day a week I get
to implement programs!

Publicly traded companies do not fill out unique forms for every major
stockholder.....rather, they standardize the data they will distribute.
This new focus on data and program evaluation must be led by the
practitioners, not funders/investors. If we are passive to philanthropic
demands for unique data sets, we will slow to a crawl.

--
Donald R. Hinkle
Director, Affordable Housing Group
Delaware Valley Community Reinvestment Fund
718 Arch Street, Suite 300N
Philadelphia, PA 19106
Voice: 215-925-1130 x212
Fax: 215-923-4764


This post transferred from the cdb-l mailing list

Well, let's think about this...

Is the time spent building an MIS to track the creation of social value
greater than the returns generated from such an investment of time?

Our practitioners are working with us to build integrated reporting systems
that will allow them to track a variety of outcome measures and ultimately
to generate custom reports on monthly/quarterly/other basis to satisfy the
"reporting" requirements of HUD, CDFIs, foundations and other folks. Is it
worth their front end investment of time to convert manual systems with no
integrity into automated systems with some greater degree of integrity?

Our practitioners are spending their life energy devoted to a cause that
all agree can be difficult to assess the attainment of (ie. the creation of
social value). Is it worth it to spend time with those practitioners,
helping them enunciate the specifics of what these "changes" in people's
lives look like, understanding how you KNOW your program works and how you
can create information systems you can use to assess what parts of your
program don't work so well and need to be improved? Presently, as I know
you must realize, most social programs justify their existence more on the
basis of politics, perception and persuasion than upon any valid, ongoing
measurement of real worth. Is it worth social and community workers taking
the time to develop their own metrics to reflect what value is generated
through the investment of their careers and lives in pursuing "community
and individual change?"

As a practitioner, I always resented the fact that funders (and lenders,
for that matter!) all want you to achieve these wonderful social goals
("This $10,000 loan will allow us to more effectively conquer poverty in
our neighborhood"), and are willing to put nickles and dimes into annual
funding/lending requests to support "programs", yet were unwilling to make
the added investment necessary in the administrative and operating capacity
of nonprofit organizations in order to effectively manage such programs and
know whether they were, in fact, as valuable as our rhetoric claims they
are. Well, as a funder i'm in a position to invest fairly heavily in those
organizations we support ($350K last year, an equal amount this year) and
to bring other funders to the table in order to help practitioners build
MISs capable of proving what we all claim we know to be true: that there
is, in fact, value in changing people's lives and the communities in which
they live and that funds invested in such efforts are worthwhile
investments of limited social dollars. Is the time spent moving such an
agenda worth the value of engaging in such an analysis?

Finally, have we figured out how to do this in a way that will withstand
all scrutiny and question? No... in fact the more I push this, the more I
realize how little you and I understand about this issue and how completely
unprepared the Sector is to even talk about developing such practices. We
are building this in partnership with our folks one step at a time. Some
have an infrastructure to build off of, many do not. Some are a year away
from automation, some are more than a year away. All deserve the chance to
build something that works for them according to their scale, needs and
capacity. It will take time.

Is our nascent effort to track social impact and tie those impacts back to
the capital investments necessary to achieve them going to change the world
and unleash millions of dollars in new support of a renewed social agenda
in this country? Highly doubtful.

However, is our effort going to be able to tell our folks (ie.
practitioners) that their lives are being spent in the pursuit of something
worthwhile, with strategies that really do what they say? Is our effort
going to be able to tell program participants that they are getting the
best service delivery possible from XYZ community group? Is our effort
going to allow me to talk to my investor (cuz there is NO such thing as
charity...all social dollars represent some type of investment in one
strategy over another) in terms he will understand and present our
experience in ways that will encourage him to put more money into our work?
Yes on all counts.

No, I haven't done a time study of how long it takes our practitioners to
complete "my" evaluation forms, cuz they aren't my forms and i'm not
requiring our folks to complete anything. We stopped the charade of
"evaluation" several years ago. They have asked me to help them build a
better way to document the effectiveness of how they are spending their
life energy and my donor's funds. Our Fund and the folks we are able to
bring to the table are working together with our investees to do so. And
while it does take time, it is certainly time well spent.

So, back to you, my friend:

How do YOU measure the social value of the loans (below market rate and
otherwise) that you put into communities? How do you know the "jobs"
created are achieving the social value you claim? How do you calculate the
social return on your investments?

Celebrate the Struggle!

yours,

j.


At 09:11 AM 3/3/99 -0500, you wrote:
>Jed Emerson,
>Have you done a time/value study? How much time does it take for all
>these practitioners to complete your evaluation forms. Representing a
>CDFI with over 800 philanthropic investors, 100 annual donors, and
>programs funded by most major foundations, my living nightmare is that
>each creates such a reporting system...........and one day a week I get
>to implement programs!
>
>Publicly traded companies do not fill out unique forms for every major
>stockholder.....rather, they standardize the data they will distribute.
>This new focus on data and program evaluation must be led by the
>practitioners, not funders/investors. If we are passive to philanthropic
>demands for unique data sets, we will slow to a crawl.
>
>--
>Donald R. Hinkle
>Director, Affordable Housing Group
>Delaware Valley Community Reinvestment Fund
>718 Arch Street, Suite 300N
>Philadelphia, PA 19106
>Voice: 215-925-1130 x212
>Fax: 215-923-4764
>
>

************************************************** **************************
Pursue Peace and Profitability!

Please note new address and numbers!

Jed Emerson
Executive Director
Roberts Enterprise Development Fund
Presidio Building #1009
P.O. Box 29266
San Francisco, CA 94129-0266
415-561-6680 Phone
415-561-6685 Fax
http://www.redf.org/ Web Site (with helpful info and links!)
************************************************** **************************


This post transferred from the cdb-l mailing list

Jed,

Sorry, I shut down when I hear judgemental comments like the field has
been reporting with "no integrity" (mentioned in both your posts). Also
there can sometimes be "real worth" in politics, perception and
persuasion. I believe there is great quality in both our program efforts
and in our reporting. I agree that we all need more integrity in our
reporting, more is always nice.

I'm glad to hear your efforts are to empower practitioners to report on
their efforts in their own fashion.

The Reinvestment Fund has been greatly assisted by a few funders that
were willing to put their money where their data needs are. We are
presently looking for a Public Policy and Program Evaluation staff
person that will coordinate our social impact data tracking and mine
that data for public policy and marketing nuggets. The position will be
part data, part academic (so of course I will not get along with the
person! LOL)

I'd be happy to pass on the job posting to anyone interested.

--
Donald R. Hinkle
Director, Affordable Housing Group
The Reinvestment Fund
718 Arch Street, Suite 300N
Philadelphia, PA 19106
Voice: 215-925-1130 x212
Fax: 215-923-4764


This post transferred from the cdb-l mailing list

The National Community Capital Association in Philadelphia is a membership
association of high-performing community development financial institutions
(CDFIs) from around the country. National Community Capital is currently
reviewing the types of data CDFIs' collect and how institutions measure the
social impact they are having on the communities in which they work. After
identifying select best practices in social impact tracking and measurement,
National Community Capital plans to develop a "how-to" guide for CDFIs that
will help institutions to develop reasonable systems for tracking key impact
indicators. A Technical Assistance memorandum around this issue will be
developed as part of this effort.

Questions about the project can be directed to Kathy Stearns, Director of
Financing and Development Services, at National Community Capital Association
(kathys@communitycapital.org).

Thanks.


This post transferred from the cdb-l mailing list