What to consider when designing gender-responsive program evaluations

Over the past decade, development practitioners have made robust program evaluation a crucial part of most programming approaches, reflecting an increased emphasis on designing cost effective projects that demonstrate significant change. This shift has coincided with a greater awareness of the need to focus more explicitly on gender as a key factor in a wide range of development-related issues.

These are generally welcome developments – as practitioners we should all aim to use evidence of what works and, crucially, what doesn’t work to improve our programs’ influence, effectiveness and reach, particularly for disadvantaged groups. Doing so also results in better outcomes for beneficiaries. However, the evaluation community has been slow to appreciate the fundamental role that gender may play in shaping the results of their program evaluations. In particular, they do not pay enough attention to understanding how gender inequities may shape program participation and response to specific interventions, or how gender relations may influence even the basic processes of data collection. Too often we still see evaluators assuming that disaggregating their data by gender constitutes “gendered analyses.”

As the world’s premier applied research institute focused on women and girls, the International Center for Research on Women (ICRW) develops and promotes a more gender-responsive approach to program monitoring and evaluation (M&E). This starts with an understanding that data collection and analyses are not gender-neutral processes, but rather are at least in part reflective of broader societal and cultural norms and expectations around male and female behavior. As a result, truly effective M&E must begin by making women’s and men’s concerns and experiences integral to the design and implementation of both programs and their evaluations. Doing so requires taking care from the outset to ensure that M&E systems fully account for how men and women may respond to program interventions in different ways. It also means being thoughtful about how gender relations more broadly may influence the effectiveness of the program, data collection and analyses. Simply collecting data from men and women is necessary but certainly not sufficient.

One example of how gender-responsive M&E can be implemented is the Towards Improved Economic/Sexual Reproductive Health Outcomes for Adolescent Girls (TESFA) project currently being implemented by CARE-Ethiopia and local nongovernmental organization partners. The project works to address the economic and health needs of more than 5,000 ever-married adolescent girls in the Amhara region through a group-based peer-education approach. In collaboration with partner organizations, ICRW led the development of the evaluation design used in the project and a customized monitoring and information system with an explicit gender-responsive approach.

All organizations took great care to think about how program and evaluation decisions would impact adolescent girls specifically. For example, while the group-based approach is effective with adult men and women in this context, the restricted mobility and time constraints young married girls face made this approach more challenging. We addressed this in a number of ways, including attentively constructing the groups to include girls living close to each other, engaging the community to encourage family support for girls’ participation, and emphasizing flexibility in terms of scheduling meetings and other activities.

The ICRW team designed a monitoring system to both collect information on the “process” of the project and to generate usable and timely information for the program team. For instance, in addition to gender-disaggregated quantifiable information, monitoring forms also collected qualitative assessments of activities, such as staff opinions of the atmosphere at meetings, assessments of reasons for girls’ absences and input from girls about their experiences at gatherings.

To collect evaluation data, we applied a multi-method approach focused on how gender inequalities shape economic and health behaviors. In particular, data were gathered on decision-making at the couple and household levels and other factors such as couple communication and experience with intimate partner violence. Qualitative data were also collected from girls and key individuals in their lives, allowing for the voices of participants to play a direct role in improving our understanding of the project’s impact.

In the case of the TESFA evaluation, the explicit focus on developing a gender-responsive approach from the early stages of program and evaluation design has resulted in a much richer understanding of the determinants of economic and health. These data have already proven to be extremely useful to program staff, highlighting misconceptions of the lives of adolescent girls and providing clear evidence for adapting the program model “on the fly.” Furthermore, the early focus on gender ensured that we focused on the “right” set of indicators and questions, moving beyond outcome and process to better understand the social processes shaping behavior. We believe that this approach can and should be broadly implemented as a part of evaluation science, as in the end this will give development practitioners a much better idea of what works, what doesn’t, and perhaps most importantly, why.

A version of this article was published on The Guardian Adolescent Girls Hub on June 6, 2013.