Posted by: ebtosun | September 16, 2011

El Varillal System Impacts and Updates to Evaluation Process

By Elise Tosun, Waslala Project Analyst

In my last post, I covered the basics of Impact Evaluation 101 (the need for a “treatment” and “control” group) and said I was looking forward to seeing the results of the follow-up survey from El Varillal #2. Now, the results have come in! Not only that, but we are using those findings to inform our next round of monitoring and evaluation for our system in El Guabo, which has an unprecedented 30 beneficiary households. Clearly there is a lot to discuss on the evaluation front!

First, let’s talk about the results we saw in El Varillal. In this community, we have about six beneficiary households (those who have a tapstand at their house), 20 households with children in school, and 48 households total. We are seeing positive results on the health front – most beneficiary households have children under the age of 3, and those households report fewer instances of children having diarrhea in the past month (53% reported instances of diarrhea in the last month prior to system construction, versus 12% now). We also know that households report washing their hands twice as often as they were before, which is potentially due to the health and hygiene workshops led by Virginia in the community.

Our findings on time savings and school attendance were more difficult to tease out of the survey results. In terms of time savings, most beneficiaries were previously getting water from sources 2-5 minutes away from their home, though those minutes added up over the course of multiple visits to the site each day. Some beneficiaries did travel over 10 minutes, and even as far as 30 minutes, to gather water for their households. We are currently measuring school attendance by asking families how many times their children have missed school in the last three months due to stomach illnesses, and we did not see any significant change in this metric – most families continue to report that their children do not miss any school because of diarrhea or other stomach illnesses.

Given these results, we are planning to update our system evaluation process to measure our health, time, and education impacts more accurately:

  • Use focus groups in each community prior to the survey to test out questions we plan to ask each household. Focus groups will be a critical step in ensuring that families understand the questions that are being asked of them and are able to answer them accurately.
  • Include health behavior changes in our assessment of health impacts. It should be no surprise that drinking cleaner water positively affects individual health outcomes. Perhaps even more important is whether Water for Waslala, through the health and hygiene workshops we conduct in each community, is effectively changing people’s behavior so that they are taking steps to prevent sicknesses. These behavioral changes include more frequent hand-washing, more frequent hydration, bathing, and washing linens and dishes.
  • Experiment with different ways of measuring time savings. Families in Waslala often don’t carry cell phones or even watches, so their concept of “30 minutes” might just not be the same as ours. Therefore, we can experiment with different ways of measuring how families spend their time before vs. after system construction: by asking them to fill out a daily pictorial diary of the activities they do for one day, asking them to allocate yesterday’s time among different activities, or even just asking whether they feel they spend less, more, or the same amount of time on certain activities.
  • Look at more objective measures of school attendance. Most community teachers keep attendance sheets which enable them to keep track of how many times children miss school. We are looking into matching these to beneficiary households to get a measure of school attendance that avoids the difficulty to parents of having to recall how often their children miss school. We can then double-check these attendance figures with household reports on the reasons that children missed school.
  • Capture other changes that could impact health, time, and education outcomes. It’s important that we continue to consider other factors that may impact our outcomes of interest but that might have nothing to do with the system construction. For example, in El Guabo (the next community we will do a follow-up survey in), many of the beneficiary households live on the main community road. This locale may make it easier for their children to get to school and might also mean they don’t have to travel as far for water as other families. Therefore, it is important to keep in mind the impact of living on the main road when we look at the additional impact of water systems.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: