In a previous article, we shared a high level introduction to ILP’s Metrics & Evaluation system that we use to assess the effectiveness of our work. We use two tools to collect most of our data:
- Atlan COLLECT Mobile App: We use this when it is practical to collect and digitize data directly in the field. An example of this is to track Students’ attendance
- Spreadsheets: We use these when digitizing the data directly in the field is not practical. An example of this is when we have to evaluate the answer sheets of hundreds of students, and then enter the results in a database for further analysis.
ILP works with 25+ NGO partners across multiple states in India. These NGO partners collectively have 100+ employees putting ILP investments to work on the ground. They are our change makers working in the remote and rural parts of India. However, some of them may be less familiar / comfortable with use of softwares, apps etc. So, one of the most critical themes that applies to our work around data collection and metrics is the need for simplicity of our Metrics & Evaluation system.
In this article, we will start with a seemingly simple metric of Students’ attendance in the preschools and schools, and explain how we have defined and refined it based on the experiences of ILP and our partner NGOs.
As mentioned above, we use the Atlan’s COLLECT Mobile app to track students’ attendance. When an ILP / NGO team member visits a School, s/he has to respond to 5 – 7 questions. This enables ILP to collate, analyze and summarize the information. The questions in our survey are explained below:
- School: User selects a school name from a pre-populated searchable database of School Names. This is a critical step to minimize errors.
- Date of Data Collection: User enters date on which s/he visited the school to collect this data
- Number of Students Enrolled: User obtains the total enrollment number from the records in the school. A positive working partnership with the head of the school makes this easier to do.
- Number of Students Attending Today: User counts the total number of students that s/he can see attending the classes at that time. This is one example where we “simplified” our questionnaire. It was very tempting for us – initially at least – to seek the counts by class, by gender and so on. However, it became clear that this was not a scalable solution that could apply across thousands of schools in multiple states.
- Do you want to submit duplicate enrollment or migrant children information today?:
- Number of currently enrolled students who are attending other schools
- Number of currently enrolled children who have migrated out temporarily or permanently
Questions 5, 6 and 7 illustrate how the ILP team’s knowledge of the ground situation helps us to design a robust data collection process. In some of the schools that we work in, it is not uncommon to have students still on the roll of the Government school, even though they may have either migrated away from that village or joined a different (private) school. In such situations, we do not want to – artificially and incorrectly – calculate the attendance to be low. So, when a user answers YES to Question E, then s/he has to answer questions F & G as well.
Every instance of data collected using the above questions has the attendance calculated for that school for that day with a simple formula using the above points: % Attendance = (4) / (3 – 6 – 5) * 100
We typically try to collect 2 to 4 data points every month for every school. This helps us to measure student attendance over the course of the entire year, instead of just taking one or two snapshots. Collecting this data frequently and independently as an NGO also helps us to correlate our data with the data tracked by the school. In scenarios – thankfully not too many! – where we notice significant differences, we engage in a discussion with the school administration, the parents and the community. Very interesting patterns also emerge from this frequently collected data! For example, we see how harvest season brings an inevitable drop in the attendance rates in schools; as students take some days off to help their parents in the fields.
As the data is collected, ILP visualizes it into easy-to-use, actionable dashboards. These dashboards help ILP to monitor the progress and evaluate how this metric is progressing. However, more importantly, we are providing this data back to the NGOs that are collecting it so that they can use the data immediately to take corrective actions. Our goal this year and the next is to streamline this process to a high degree of efficiency – so that we derive the maximum benefit of near-real-time tracking and remedial actions.