Guest post by Sarah Boeckmann, our M&E Consultant
I have been SHE’s Monitoring and Evaluation consultant for about 2 years. That means I have helped the team develop a logical framework for SHE’s education and advocacy work. I also developed tools to help them collect data that, in the long term, will help SHE understand the impact of its work on women and girls in Rwanda. Recently I had the chance to go to Rwanda to work with the team there to test some data collection tools that are being used to collect data from girls in 10 rural, primary schools.
SHE is at a very exciting moment right now because menstrual hygiene management (MHM) clubs are starting this term in 10 schools in Kayonza. The clubs are run by teachers trained by Nadia Hitmana, SHE’s Health and Hygiene Manager, to teach an MHM curriculum to students. Now that these clubs are about to kick off, and pads will be reaching these same schools, it is important for SHE to collect baseline data on the students in these clubs to understand the effect the training has on them.
SHE is interested both in how the training affects their knowledge of menstruation and healthy menstrual hygiene practices, but also if and how the training affects their behavior around menstruation. This includes everything from their use and disposal of pads, to how confident they feel during their period, to how often they are absent from school because of their period.
Data like this is tricky and complex to gather accurately and a good way to try to get the best data possible is to use a combination of quantitative (like surveys) and qualitative (like focus groups) data collection tools. It is hard to get these tools right without testing them in the setting where they are meant to be used.
While I was in Rwanda we spent a day in one of the schools testing the various tools with a classroom of girls who volunteered. These are the kinds of things that we were looking for:
- Were their questions that the girls didn’t understand?
- Were we missing any possible answer choices on multiple choice questions?
- How long did it take to complete a survey?
We also asked the students what they thought of the tools and how they were administered. Sometimes asking those kinds of open-ended questions point out problems you never would have considered.
Testing the tools in a real life setting always makes glaringly obvious issues you never would have considered. You might find out the tool you thought would take 30 minutes takes 2 hours, or that a question you thought was great makes absolutely no sense.
As for us, we had decided that to test the students’ knowledge of menstruation and MHM we would have them close their eyes and raise their hands yes or no/true or false in respond to a question. We thought this would solve the problem of having students of different ages and literacy levels taking one test. We also wanted to make the data collection process more fun for the students and not start the club off with formal test that might make the club seem boring or scary. What we hadn’t considered was that there would be so many kids in a small classroom that they could barely raise a hand and could definitely feel what the kids on either side of them were doing.
While piloting data collection tools highlight problems, it also makes solutions obvious as well. The teacher who was with us pointed out that afterschool clubs are often held outside and that if the students were outside they could be more spread out, solving the problem. Piloting the tools and seeing what issues arise helped us to make the changes needed to finalize the tools.
Nadia and Flora are collecting lots of great baseline data from more than 200 girls right now before they receive pads or start learning about MHM. Then we will collect data from a similar number of girls after they spend a term learning about MHM from SHE-Trained Teachers and see what changes or doesn’t change and why.
It is especially important for SHE to get this kind of rich, detailed information from the girls in these 10 schools so that the team can learn and make changes based on lessons learned as it scales-up to providing pads and teacher training throughout Rwanda.