Recent development theory and practice has placed the concepts of participation and empowerment at its core. Despite vast experience across the world in translating these theories into practice, the sharing and systematising of this experience still has far to go, especially in the area of evaluation. Evaluations carried out in traditional ways tend to come up with over-inflated claims while missing out on a wealth of evidence of real changes and achievements, many of which may not have been anticipated by the project managers and planners.
This briefing is based on a detailed study of Reflect
projects in Uganda and Bangladesh published by DFID in 2003. The study explores
in depth how participation and empowerment translate from theory and rhetoric
into practice, and how the concepts are understood by the different people and
groups involved in such projects. Of particular interest for development practitioners,
planners and theorists are the insights into the complexity of evaluating projects
with soft or intangible outcomes such as empowerment. This briefing pulls out
some thoughts and conclusions from the study in order to further promote and
provoke debate in this important area.
Expectations of participation: what's in it for me?
All
the people and institutions involved in any development process have their own,
usually different but often compatible, motivations for their involvement. The
study mentions a range of motivations found in the sample projects in Bangladesh
and Uganda. Although on the surface these were all adult literacy programmes,
in practice the motivations of people involved were more complex, for example:
The study found that a 'purist vision' of participation held
by project planners and evaluators could in fact blind them to changes which
in participants' eyes were real gains. It also pointed out that 'baggage' attached
to particular sectors, in this case literacy programmes, can have an impact
on participants' expectations of a project. Space given for all parties (NGOs
and participants) to be honest and clear about their motivations could result
in evaluations which measure changes that are important to participants, and
not just those that fit in with the NGO or donor's definition of progress.
Defining empowerment: what are we evaluating?
Empowerment
is a very vague term, which can be (and usually is) interpreted differently
by different people. This may be part of the attraction of 'empowerment' to
development planners and professionals, as it can accommodate the divergent
needs and expectations of many different players in a project or process, but
the same ambiguity can also threaten useful evaluation and productive communication.
"A very detailed knowledge of the context is necessary before attempting a judgement of what constitutes empowering action and what doesn't."
The study found that women participants' expectations of, and strategies for achieving, empowerment depended heavily on their environment and circumstances. While participatory theory places high value on open debate as a pre-requisite to the empowerment of poor people, many of these Reflect participants actually aspired to recreate the kind of formal discourse practiced by community leaders, NGO staff and politicians, of which they had previously only been passive recipients.
"The empowerment discourse allows room for manoeuvre within an unresolved ambiguity about who is driving the change process and towards what ends."
Despite divergent interpretations of what it means in practice, evaluations tend to measure empowerment against the values of the dominant NGOs, donors and the middle-class people who staff them. This understanding of empowerment, the authors argue, derives from western ideals of individualism and self-confidence, which do not necessarily fit with other ways of life. In its most dangerous form, empowerment can become an evangelistic modernising force, encouraging poor undereducated people to accept modern practices and values for themselves, rather than imposing them directly, allowing practitioners to reconcile bottom-up approaches with their need to guide the change process in the 'right' direction.
"When we start out to evaluate empowerment we have too clear an idea of what we are looking for and then promptly find it."
Evaluating a project in line with the NGO's understanding
of empowerment runs a real risk of missing out on achievements and changes important
to, and perceived by, the participants. Without making a real effort to understand
what empowerment means to the different people involved in a project or process,
we will not be able to understand how and why participation in programmes might
be of benefit to them.
The broader environment: what are we evaluating?
The study found that evaluations tend to focus too hard on the internal actions and consequences of a project, without sufficiently taking into account outside factors influencing an individual's empowerment, such as social trends, changes in personal circumstances or involvement in other projects. Evaluations tend to oversimplify both the change process itself and the attribution of credit and responsibility. The study found that participants are likely to get involved in a group or project as one part of a complex strategy to improve their lives, and the very act of getting involved sets them apart from their peers, making control group evaluation difficult. Changes in people's lives can only be attributed to their whole strategy, not to any individual project.
"… learners carefully match what they pick up in the institutionalised education setting with their own ambitions, the boundaries of which are set by the circumstances they live in. The results normally reveal an accommodation of modern values in traditional practices… and always defy any notion of simplistic cause-effect relationships."
The authors of the study argue that the empowerment narrative
diverts attention from external factors keeping people poor, and focuses instead
on the lack of capacity of the individual poor man or woman. The reasons changes
come about can only be understood in terms of the structures and hierarchies
in which the participants live and operate. While an evaluation may claim that
a woman has improved access to health services because of new knowledge, the
woman herself attributes her increased access to her improved status, which
makes her feel, or be perceived as, more eligible to such services. Many increases
in power or command felt by participants will only be picked up by evaluations
which look beyond the formal and dominant spheres of influence. We need to explore
the behind-the-scenes mechanisms through which people can make themselves heard.
Regarding power: who's asking and why?
"The view of participation as a neutral, apolitical and non-ideological technology, which simply allows the poor to voice what they have always wanted to say… takes NGOs even further from a realistic and honest understanding of their own power and how and to whose benefit it is exercised."
Recognising and understanding the nature of power relationships between staff and participants in development projects is essential to good communication. While the ideal of bottom-up, participatory development requires that participants drive the process, influence planning and relate equally to NGO workers, in reality the relationships are likely to be far less collaborative and equal. NGOs are powerful providers of opportunity, goods such as funding and credit, and status by association. In these circumstances, participants are likely to pick up on what the NGO expects from them and tell the evaluation team what they think they want to hear.
"Statements made to NGO workers and researchers are part of an ongoing process of negotiation, not simple statements of fact."
The authors found that despite numerous
tales of attitude changes and positive action points reported in evaluations,
many of these could not actually be seen in practice. In some cases, commitment
to the new attitudes and behaviours was expressed in order to please the donor
or NGO, even though they were actually totally impractical. Evaluations need
to probe beyond the usual questions and testimonies, fully recognising the social
tensions between the parties to the evaluation.
"No matter how participatory their methods, or how profound their rejection of 'top-down' ethnocentric approaches, development NGOs are still seemingly compelled to provide evidence that progress is being made, and that their programme can be credited for it in some way. This is where any effort to understand the specificity of local contexts inevitably gave way to simplistic myth making."
Exacerbated by a competitive funding arena and the nature of large institutions, there is a real pressure to present projects and approaches as an effective way of orchestrating positive change. This inevitably leads to a process of selective reporting and listening at all levels, where interviewees, researchers, authors and editors seek to transform messy realities into success stories. In all of this, NGOs end up taking the credit for changes that are in reality the result of efforts by many actors, including the state and the poor communities themselves.
In the end, all of this serves only to increase the gap between
rhetoric and reality, as case studies and learning are carefully selected to
present a positive story, which frontline workers feel unable to match or keep
up with. Honest and useful evaluations have to be undertaken in the spirit of
true learning rather than positive spin, in the security of an environment where
honest or negative evaluations are not penalised.
The insights and conclusions contained in the study were very challenging for Reflect practitioners at all levels. As is reflected in this briefing, the study raises more questions than it answers. It exposes the very real risk that concepts of participation and empowerment are used to bypass, rather than really challenge, existing power relations. In the months since the study was published, Reflect practitioners have been reflecting on their own work, trying to make sense of the criticisms and insights, and come up with some answers of our own. Below are a few of our thoughts and recommendations:
