Coding in the Congo: Congo Trip #7: Search for Common Ground

No posts for more than a week. Did nothing interesting happen? On the contrary! Last Saturday evening I came back from a whole 2.5 days in Congo: 1.5 days travel to get there, 2.5 days in-country, and 1.5 days again to get back home. Why?

The reason to be in Bukavu was to work with Search For Common Ground (SFCG). As the readers of this blog know the Congo suffers from high levels of sexual violence and other abuses against civilian populations. The instigators of this violence are rebel armed groups, like the FDLR or local militia, but a major perpetrator is also the national army, the Forces Armées de la République Démocratique du Congo (FARDC). Search is the organization in Congo that works directly with the Congolese military to decrease these abuses. They do this by a set of social norms-changing interventions: cinema, distribution of comic books, sensabilisation sessions that include role-playing games, participatory theater, joint military-civilian activities, etc.

To date, none of the interventions that are intended to directly address abuses against civilian populations has been seriously evaluated (read: RCT). Important questions remain: Is this violence due to lack of community ties? Lack of monitoring? Because soldiers are underpaid? Because of social norms explanations? Lack of access to justice? And the body of evidence accumulated about what works is very weak. However, there exists evidence that social norms-changing interventions can be effective in other contexts, addressing related problems of prejudice (see for example, work by Betsy Paluck). Combined with the importance of the topic SFCG and us (Macartan, Raul and I) thought this to be a potential program for an solid evaluation.

Last Monday (yesterday) there was a deadline for a 3ie grant application -- 3ie is an organization that funds impact evaluations (they for example funded our TUUNGANE study with the International Rescue Committee). To learn about the causal impact of the SFCG program we want to conduct an RCT evaluation (randomly assigning treatment to units) and thus a lot had to be learned on both sides before we could send a successful application to 3ie:

What is the treatment? What is the theory of change? Do we and the organization actually belief the program makes a measurable impact, and how is this impact achieved? What are the mechanisms? And can we measure these mechanisms? Do we think some parts of the program to be particularly strong? Does a variation in treatment make sense?
What is the unit of analysis? Search works with the military units, but the outcome is at the population level. Is there geographical overlap between the two?
Is there enough statistical power? And at what military unit level should Search work? To have the largest impact possible, but also keeping into account that more treatment units increases statistical power. The more power the more likely false negatives (there is actually an effect but we didn't find it) are avoided.
What about spillovers? Many parts of Congo are "military active", and thus military units are moved around a lot. Does this lead to spill-overs? The treatment is information, and that might spread quickly between treatment and control units.
How to measure abuses by the military? Asking a soldier "Did you rape last week?" and then comparing treatment with control communities, is not very likely to work. So how to measure civilian abuses. Might we implement an SMS-based monitoring system (a la Voix des Kivus) in hundreds of treatment and control villages to learn about levels of abuse?
Are we ethically ok to conduct an RCT? We should not keep control units away from treatment if over the course of the program we find out that some control military units are abusing to an extend larger than some treatment units. Maybe methods different than RCT are better suited (regression discontinuity, process tracing, etc.)?

It were 2.5 very busy days. First for us to learn about the program. Then for Search to learn about evaluations. And then working together (and also with the Congolese army) to answer the above questions. In the end we decided not to go for the 3ie application yet, but probably we will in the future.