Feedback is an innovative research, evaluation and information quality services provider, inspired by the challenges of sustainable development. By applying creativity, conscience, obsessive curiosity and uncompromising scientific rigor, we empower clients with the best possible empirical basis for taking action and making an impact.

0

Social Net Present Value: The Bottom Line

20120325-232006.jpg

Presentations at the recent Mining Indaba on using Social Net Present Value (SNPV) methods to report on corporate social investments and responsibility once again highlighted the risks of resorting to an accounting framework to measure impact. The highlighting was done by those remarking from the floor, not the presenters who appeared intent on selling a version of the methodology to CSR practitioners desperately seeking for heuristics to translate development language into corporate speak.

Some intensive work has been done on SNPV methodology and it is not without merit. But SNPV is by no means the panacea to impact measurement complexities that some tool peddlers insist. There are at least 4 areas that problematise SNPV as method:

1. SNPV Light. While the theoretical work on SNPV adopts a proper accounting formula to arrive at a number, CSR in practice does not mirror that level of rigour. I have yet to see a publicly available corporate report that values the enterprise’s social benefit AND social cost, then subtracts cost from benefit to arrive at a result. In addition some methodologies rely on the dubious technique of extrapolating impact from at the expense of actually collecting time series data, an astonishing error blatantly entertained by ‘service providers’ offering SNPV solutions at the conference.
2. The complexities of assigning value OR the reductionist error. This aspect alone is grist for multiple contending journal articles, but for the purposes of the post lets distill the toxic mix to the most self-evident point: how do we arrive at valid quantitative values for the outcomes of social investments? How do you value something like improvements in education or health services satisfactorily? How do you assign a single quantitative or monetary value to community cohesion resulting from improved social housing and reductions in migrant labour? And do so credibly?
3. The equivalence problem. Presumably one of the attractions of SNPV is its purportedly efficient aggregation of social investment performance to a single value. This implies however that somehow a quantitative method must be formulated that can validly assign equivalent units of value to say ‘improved employability prospects as a result of education programmes’ and ‘improved productivity as a result of health programmes’, and then add those values together for a net result. Anyone who grasped basic number theory in under-graduate statistics should be inspired to new heights of skepticism right here.
4. Perversely incentivized social responsibility. While we don’t have an instance in mind, behavioral economics will tell you its plausible: if there is a significant discrepancy between SNPV from health over education programming, then loading our CSI portfolio with health projects makes for a better bottom line performance. And this may become a criteria for investment decisions that trumps community needs. Development prioritized by corporate performance objectives . . . Does the phrase ‘short-termism’ come to mind?

SNPV has its uses, but social phenomena are complex and the constellation of social phenomena and processes constituting ‘development’ is inordinately complex. It has no single bottom line impact, at least nothing that will stand up to the scrutiny of the basic test of measurement credibility in social science – validity. In development there are many lines. And not just at the bottom. Everywhere.

20120325-232006.jpg

0

National Evaluation Policy Framework

SA Coat of Arms

The Department of Performance Monitoring and Evaluation in the Presidency has just released the National Evaluation Policy Framework. As the introduction to the document explains:

“The Policy Framework for the Government-Wide Monitoring and Evaluation System (GWMES) which was approved by Cabinet in 2005 describes three “data terrains” which underpin the monitoring and evaluation system, namely, programme performance information; social, economic and demographic statistics; and evaluation. While the Presidency is the custodian of the GWMES as a whole, National Treasury has published the Framework for Programme Performance Information and Statistics South Africa has published the South African Statistics Quality Framework to provide policy frameworks for the first two terrains. This National Evaluation Policy Framework completes the set of policies which make up the GWMES.”

With the publication of the Framework the GWMES now has a complete set of guidelines and the foundation for consistent assessment of government’s service delivery performance has been laid.

The framework is downloadable from the Presidency’s website or follow the link National Evaluation Policy Framework.

The guidelines for the additional two terrains of performance assessment can be downloaded by following the links in the text above, or by visiting the StatsSA and National Treasury websites.

0

Evaluation vs.Social Research

jumping-brain

John Lavelle’s diagrammatic representation of the difference between Evaluation and Social Research. What’s missing, I would suggest, is USE. Evaluation is driven by an intention to use research results immediately. Research is concerned with generating a body of knowledge but is not necessarily preoccupied to the degree evaluation is with Use issues. That is not to say that Use doesn’t matter to social researchers – but Use is not usually a primary objective for Research to the extent that it is for Evaluation. At times it may even be an incidental consideration.

Does that not make Evaluation an applied social research practice? And where would you put USE on the hourglass?